Onboarding to the Elementary Cloud Platform

This video walks through the onboarding process for Elementary. It begins with setting up environments, which represent dbt projects or DAGs tied to specific targets, such as production, staging, or development databases. After installing the dbt package as an SDK, users configure their data warehouse and specify where the Elementary schema will be created. Once connected, the Elementary Cloud Platform syncs metadata from the Elementary schema and updates results in near real-time. Users can also integrate tools like Slack, Microsoft Teams, and incident management software for enhanced collaboration and monitoring. 

Let's start with the basics first. So first when you on board to Elementary, you need to start with adding an environment you can think of an environment as like a dbt project, a DAG that you run against a specific target.

So if you have one big DAG that runs against a production target, your production database and schema, this is one environment. If you have one for staging, one for dev -these are additional environments, so you can configure as many environments as you would like. If you have multiple dbt projects that you import into a single dbt project and it creates one big DAG this is still a single DAG, a single environment. So another way to think about it is like every Elementary schema that is being collected for your main job represents an environment. So what you need to do as part of the setup is first install the dbt package as an SDK that runs as part of your dbt invocation as part of your pipeline.

Then you need to choose your data warehouse, give the credentials and authentication details, and specify where  the Elementary schema is being created in your warehouse because this is configured as part of the dbt package installation process. So if you all already work with the open source, this means that you already have Elementary schema in your warehouse.

So connecting to the Elementary Platform is very easy for you because you just need to give the connection details and the Elementary schema name and that's it. From now on, the Elementary Cloud Platform will start syncing all the metadata from the Elementary schema to the Cloud Platform.

You can control the cadence of things. You can say that it's based on Hourly updates, but lots of our customers actually, when they deployed to production, they use webhook triggers. So you can trigger a sync from the Elementary Cloud Platform to the Elementary schema at the end of every job invocation, whether you use dbt Cloud or Airflow at the end of it, you can trigger a webhook and then Elementary will start a sync right away.

And this means that the UI in Elementary, all the validations, all the results, all the alerts will be updated in a near real time.

In addition to that, you can connect integrations so you can connect Slack or Microsoft Teams as your messaging app, you can connect incident management tools like Page Visibility and AuthGenie. Tableau and Looker. We have more coming soon. We have Power BI in the works. Additional ones are also coming soon.

You can connect a code repository, GitHub or GitLab, where you usually host your dbt projects. And that's it. That's the onboarding