Adding Tests Using the Elementary Cloud UI

This video explains how Elementary integrates with your code repository to automatically create and configure dbt tests, ensuring that they compile correctly and won't break your pipeline. It demonstrates how Elementary scans YAML files, enriches them with model and validation definitions, and opens a pull request for review. Once merged, the tests, including custom SQL validations, will run through dbt and sync results with the Elementary Cloud Platform. This approach bridges the gap between technical users who prefer working with code and non-technical users who can configure tests through the UI while keeping the code as the single source of truth.

 For example, I can use the dbt utils recency test. You can see that we have a small IDE with a prebuilt template that we provide out of the box, so you don't need to think what are the required parameters. We also say if the parameter is optional, so we can remove them. If, for example, you will remove a parameter that is mandatory, we will give you an error.

To prevent you from creating tests that won't compile and will break your pipeline. So if  the configuration is okay, then we let you continue configuring the test so you can rely on it and trust that if you configure the test, on the Elementary UI, the test will compile and will work for you.

So let's use our description and say the data is updated here yesterday. And now I can review my selections, review the configuration, and once I'm satisfied with it, I can submit it. And what happens right now is that we automatically connect to your code repository, we scan all your YAML files, and we understand if your models are defined, your columns are defined, and then we automatically enrich your YAML files with these additional validations.

And  if you don't have models defined, we will automatically enrich your YAMLs with these models definitions, so you don't need to be worried about that as well. And as you can see, we automatically open a pull request for you. You will see that this is a pull request from Elementary. The user that is logged in will be mentioned on the PR and all the configurations that you did from the UI will be translated into validations that are configured in your code. This way, your code remains the single source of truth for all your validations. And you can still collaborate on it, and you can still apply all your CI processes  and validations that you do when you develop. And also there is no vendor lock-in and all the validations remain yours. And from now on, once you merge the PR the dbt package will run these validations. We'll run these tests. And collect the results. If these are Elementary tests, it will run the Elementary test.

If these are  the dbt utils or any other dbt native test, we will collect the result and upload it into the Elementary schema and the Cloud will sync it back to the UI. So it's really cool because it really bridges the gap between technical the technical side of the data team who love working with dbt projects and maintain everything in code and also let people who are less familiar with it to work with the UI. As I mentioned earlier, you can also do things like adding a test to all my primary keys at once. For example, you can say that I chose all my ID columns or all the columns that end with ID.

And then I can add a unique test. I can add a description. I can add myself as an owner. I can add a tag so that it's part of my stability project. Again, I can review my selections and configuration and submit a pull request. And again, what happens is that we will scan the YAMLs and enrich the code with all the validations that will be that will be added to your code repository.

In addition to that, you can also configure custom query. So I don't have like a query that I prepared in advance, but it's an IDE. So also data analysts can work with it and paste here all the custom query logics that they have and we will open a pull request with this custom logic and we'll translate it into a dbt singular test. If you're familiar with it, it's the mechanism in dbt that enables you to implement custom SQL validations. So all the custom SQL queries that will be made by our data analysts will be translated into the dbt singular test and all the validations will remain in your source control, which is really important because working in a more structured way and maintaining all the metadata and validations in the code enables us to really collaborate on it and making sure that everything is repeatable and can be configured again from the code, if that makes sense.