dbt test: not_constant

This page details the not_constant test from the dbt-utils package. This test is designed to ensure that a specified column within your dataset does not contain the same value across all rows. It's particularly useful for validating diversity in data where uniformity is not expected or desired, such as in columns that should contain unique or varied entries.

How it Works

The not_constant test checks that the values in a specified column are not all identical. This is crucial for maintaining data integrity in scenarios where a column is expected to have varied data, and a constant value across all rows could indicate a data quality issue or an error in data collection or processing.

Steps and Conditions:

1. Column Selection: Identify the column you wish to test for constant values.
2. Execution: The test evaluates the column's values across all rows in the dataset.
3. Optional Grouping: If the group_by_columns parameter is used, the test checks for constant values within each group defined by the specified columns, rather than across the entire dataset.
4. Outcome:
   • Pass: If the column contains varied values (or varied values within each group, if grouping is used), the test passes, indicating that the column does not suffer from uniformity where it is not expected.
   • Fail: If the column contains the same value across all rows (or within any group, if grouping is used), the test fails. This failure suggests a lack of expected diversity in the data, which may need to be investigated.

Example Usage: Fintech Company

For a Fintech company, ensuring that transaction records are accurately captured and diverse is essential for fraud detection and financial analysis. The not_constant test can be applied to the transactions table to validate the transaction_amount column, ensuring it does not contain the same value for all transactions, which could indicate a systemic issue or potential fraudulent activity.

Consider a scenario where the transactions table stores records of financial transactions, including a transaction_amount column that should naturally vary between transactions.

models:
  - name: transactions
    columns:
      - name: transaction_amount
        tests:
          - dbt_utils.not_constant

In this example, the not_constant test ensures that the transaction_amount column in the transactions table contains varied values, as expected in a healthy dataset of financial transactions. Uniform transaction amounts across all records could indicate an error in data capture or processing, or even fraudulent activity.