At Elementary we have two product offerings:
- Elementary Open-source package - An open-source dbt package and CLI tool you can deploy and orchestrate to get basic observability for your data pipelines.
- Elementary Cloud platform- A turnkey data observability platform built on top of the open source package, designed for teams running mission-critical data pipelines. It includes a broad set of features, such as column-level lineage to BI tools, automated monitors for freshness, volume, and schema, and collaboration features.
If you are just starting with Elementary, or you’re an Elementary open-source user, you are probably wondering if and when you should choose the Cloud option.
This post will help guide you through the decision and explain the considerations.
Comparing Elementary Cloud to Elementary Open-source
You can also find a detailed comparison of the 2 products here.
Why teams choose Elementary Cloud
We often see data teams who use Elementary open-source to cover basic observability requirements. Their need for observability is an internal team need. They also have the time and resources to build, host and maintain the open-source product.
But in many cases, for teams that are scaling or need production-grade monitoring, Elementary open-source isn’t the answer.
Based on what we have seen so far, teams select Elementary Cloud for the following reasons:
They want faster root cause and impact analysis with column-level lineage to the BI
Spending long hours trying to understand the root cause of an issue is not uncommon. Prioritizing issues based on the assets that were impacted is a best practice that reduces the downtime of critical assets.
- Elementary Cloud includes automatic end-to-end column-level lineage from sources to the BI dashboards, to quickly understand the root cause of the issue and identify the impact.
They aim to achieve high coverage with low effort
One of the best features of Elementary Cloud is automated monitors. Once an account is set up, volume, freshness, and schema monitors are created automatically without any need for manual configuration. The monitors leverage metadata only and don’t add compute costs. This helps them create a very strong baseline with zero effort or cost.
They can’t afford to f**k it up
If their data is used in one of the below scenarios, data pipelines are considered mission-critical and require guaranteed uptime and professional support:
- Data powering automations - Actions are triggered automatically based on their data (with reverse ETL tools or custom code). In these cases, a data issue can lead to a significant negative impact.
- Self-service analytics - Democratization of data also raises the risk of data quality issues and inconsistent analytics outcomes, as more people interact with data directly and are less likely to notice if it is off.
- Embedded analytics - If their data is served to customers and something goes wrong - they have to catch it ASAP. Serving incorrect data to customers can lead to serious consequences.
- ML models - If data is used to train machine learning models, broken data can harm the model accuracy, garbage in == garbage out.
They need a solution that suits the entire team
Many teams want to enable their non-dbt developers to be more independent. Elementary Cloud enables users like data analysts to create their monitors and tests in Elementary, but still maintains the control of the code to the dbt project maintainers. This saves a ton of time and reduces the backlog of requests.
They want to monitor multiple environments
Some organizations manage multiple dbt environments (data mesh). Setting monitoring for multiple environments in our cloud product is very simple, and there is the added benefit of access management across different teams and environments.
They need advanced security & dedicated support
Elementary Cloud customers get dedicated support that helps with achieving optimized coverage and configuration of the platform to their needs. We also support SSO and RBAC with an identity provider of your choice and advanced security and deployment options.
Want to try Elementary Cloud but worried about getting through security?
We know going through security is painful and can take time, but based on our experience - we pass security with flying colors.
Elementary Cloud is designed with the core principle of least privilege. Our cloud service does not require permission to access customer data. Therefore, we instruct our customers to create a dedicated role for Elementary with read only access only to the Elementary schema and metadata in your data warehouse.
Making the Decision
- Are your data pipelines mission-critical based on the above? Yes / No
- Are you aiming for full coverage? Yes / No
- Do you get asked about data integrity often and wish you could easily measure and report on it to stakeholders? Yes / No
- Do you want the data analysts on the team to become more independent so that they don’t come to you for every single thing? Yes / No
- Is your team at max capacity and can’t afford to maintain a deployment of an open-source tool that is critical infrastructure? Yes / No
- Do you want to sleep better at night knowing that someone else is monitoring your monitoring tool so that you don’t have to worry about it crashing? Yes / No
- Are there more than 7 people on the data team? Yes / No
- Does it take your team too long to find the root cause of an issue and understand its impact? Yes / No
Answered ‘yes’ on at least 5 questions?
Get started with a free trial or book a demo.
Contributors
At Elementary we have two product offerings:
- Elementary Open-source package - An open-source dbt package and CLI tool you can deploy and orchestrate to get basic observability for your data pipelines.
- Elementary Cloud platform- A turnkey data observability platform built on top of the open source package, designed for teams running mission-critical data pipelines. It includes a broad set of features, such as column-level lineage to BI tools, automated monitors for freshness, volume, and schema, and collaboration features.
If you are just starting with Elementary, or you’re an Elementary open-source user, you are probably wondering if and when you should choose the Cloud option.
This post will help guide you through the decision and explain the considerations.
Comparing Elementary Cloud to Elementary Open-source
You can also find a detailed comparison of the 2 products here.
Why teams choose Elementary Cloud
We often see data teams who use Elementary open-source to cover basic observability requirements. Their need for observability is an internal team need. They also have the time and resources to build, host and maintain the open-source product.
But in many cases, for teams that are scaling or need production-grade monitoring, Elementary open-source isn’t the answer.
Based on what we have seen so far, teams select Elementary Cloud for the following reasons:
They want faster root cause and impact analysis with column-level lineage to the BI
Spending long hours trying to understand the root cause of an issue is not uncommon. Prioritizing issues based on the assets that were impacted is a best practice that reduces the downtime of critical assets.
- Elementary Cloud includes automatic end-to-end column-level lineage from sources to the BI dashboards, to quickly understand the root cause of the issue and identify the impact.
They aim to achieve high coverage with low effort
One of the best features of Elementary Cloud is automated monitors. Once an account is set up, volume, freshness, and schema monitors are created automatically without any need for manual configuration. The monitors leverage metadata only and don’t add compute costs. This helps them create a very strong baseline with zero effort or cost.
They can’t afford to f**k it up
If their data is used in one of the below scenarios, data pipelines are considered mission-critical and require guaranteed uptime and professional support:
- Data powering automations - Actions are triggered automatically based on their data (with reverse ETL tools or custom code). In these cases, a data issue can lead to a significant negative impact.
- Self-service analytics - Democratization of data also raises the risk of data quality issues and inconsistent analytics outcomes, as more people interact with data directly and are less likely to notice if it is off.
- Embedded analytics - If their data is served to customers and something goes wrong - they have to catch it ASAP. Serving incorrect data to customers can lead to serious consequences.
- ML models - If data is used to train machine learning models, broken data can harm the model accuracy, garbage in == garbage out.
They need a solution that suits the entire team
Many teams want to enable their non-dbt developers to be more independent. Elementary Cloud enables users like data analysts to create their monitors and tests in Elementary, but still maintains the control of the code to the dbt project maintainers. This saves a ton of time and reduces the backlog of requests.
They want to monitor multiple environments
Some organizations manage multiple dbt environments (data mesh). Setting monitoring for multiple environments in our cloud product is very simple, and there is the added benefit of access management across different teams and environments.
They need advanced security & dedicated support
Elementary Cloud customers get dedicated support that helps with achieving optimized coverage and configuration of the platform to their needs. We also support SSO and RBAC with an identity provider of your choice and advanced security and deployment options.
Want to try Elementary Cloud but worried about getting through security?
We know going through security is painful and can take time, but based on our experience - we pass security with flying colors.
Elementary Cloud is designed with the core principle of least privilege. Our cloud service does not require permission to access customer data. Therefore, we instruct our customers to create a dedicated role for Elementary with read only access only to the Elementary schema and metadata in your data warehouse.
Making the Decision
- Are your data pipelines mission-critical based on the above? Yes / No
- Are you aiming for full coverage? Yes / No
- Do you get asked about data integrity often and wish you could easily measure and report on it to stakeholders? Yes / No
- Do you want the data analysts on the team to become more independent so that they don’t come to you for every single thing? Yes / No
- Is your team at max capacity and can’t afford to maintain a deployment of an open-source tool that is critical infrastructure? Yes / No
- Do you want to sleep better at night knowing that someone else is monitoring your monitoring tool so that you don’t have to worry about it crashing? Yes / No
- Are there more than 7 people on the data team? Yes / No
- Does it take your team too long to find the root cause of an issue and understand its impact? Yes / No
Answered ‘yes’ on at least 5 questions?
Get started with a free trial or book a demo.
