Skip to content
northstar
Docs menu

Quality and validations

The trust score, its six dimensions and the rules that guard your data.

In short

Each analysis gives a dataset a quality score from 0 to 100, built from six dimensions. The score shows which dimension pulls it down, and a severe problem caps it so it cannot be averaged away. Validation rules turn your business expectations into checks that run at every analysis.

The quality score

The score tells you how far you can rely on a dataset today. It appears on the Quality tab and in the dataset header.

ScoreLevel
80 to 100Excellent
60 to 79Good
Below 60Needs improvement

Next to the score you find:

  • the change since the previous analysis;
  • how many dimensions it rests on, for example "Based on 5 of 6 dimensions";
  • the reason, when the score is capped.

The score is computed at each analysis. An analysis runs:

  • when someone clicks Run an analysis on the Quality tab;
  • after a transformation builds or rebuilds the dataset;
  • when a data pipeline requests it through the API.

Each analysis also runs the validation rules and checks the dataset's alerts.

The six dimensions

DimensionWeightWhat it measures
Validity30Your validation rules, weighted by severity
Completeness20Filled values per column, compared with each column's own history. Placeholders such as "N/A" or -1 count as missing.
Freshness15Age of the newest data against the expected update cadence
Volume15Row count change against the dataset's usual changes. A table that only grows is not penalised for growing.
Uniqueness10Duplicated rows and duplicated keys
Stability10Column drift, schema changes and the share of anomalous rows, against the dataset's history

A dimension without signal is left out instead of being scored 0. For example, validity has no signal until the dataset has a validation rule. The score is then computed on the remaining dimensions.

Expected freshness comes from, in this order:

  1. a freshness validation rule, if you set one;
  2. the update cadence learned from the dataset's history;
  3. otherwise, 48 hours.

Caps

Some problems are too serious to be averaged with good results. They cap the score:

ProblemMaximum score
A critical validation rule fails59
The primary key has duplicates59
The table is empty40
The schema changed in a breaking way40

A capped score always stays below Good or Excellent, whatever the other dimensions say.

Validation rules

A validation rule states what the data must respect. Rules are grouped in suites, on the Validations tab. Examples:

  • the email column must never be empty;
  • revenue must be positive;
  • the status must be one of an approved list of values;
  • the table must hold between 1,000 and 50,000 rows;
  • the newest row must be less than 24 hours old.

Rules check columns (values, lengths, ranges, patterns), statistics (mean, median, percentiles), the whole table (row count, freshness, schema changes, duplicated rows), or a custom SQL query.

Each rule has a severity, which sets its weight in the validity dimension:

SeverityWeightUse it for
Critical5A rule whose failure makes the data wrong for decisions. A failure caps the score at 59.
Warning2A problem worth fixing, which does not invalidate the data
Info0.5A signal to follow

The owner, an editor or a platform administrator can create and edit rules.

AI recommendations

On silver and gold datasets, the AI can suggest validation suites from the dataset's profile. You review each suggestion, then accept it to add it to the dataset's validations, or dismiss it. Suggestions need at least 70% of the dataset and its columns to be documented.

Early warnings

Each analysis also detects problems by itself, each with a severity:

  • schema changes (removed, retyped or added columns) and empty tables;
  • volume drops and stale data;
  • too many empty values, and placeholder values such as "N/A", blank strings, -1 or 9999;
  • duplicated keys and duplicated rows;
  • dates in the future, columns full of zeros, constant columns;
  • a change in a column's distribution since the previous analysis;
  • an unusual share of anomalous rows, with the columns that explain them.

These warnings need no setup. To be notified, create an alert.

AI summary

On silver and gold datasets, an AI summary explains each analysis in plain language: what changed since the previous analysis, the main issues and their impact, what the anomalous rows have in common, and what to do next. A suggested validation rule can be added in one click.

  • The summary is built from aggregated statistics only, never from raw rows.
  • It is only generated again when the analysis actually changed.

FAQ

My dataset has no validation rule. Is its score reliable? It rests on fewer dimensions: validity is left out. Add a few rules to make the score more meaningful.

Why did the score drop although nothing failed? Open the score breakdown: freshness often drops when an update is late, and volume when the row count moves more than usual.

Why is the score capped at 40? The table is empty, or its schema changed in a breaking way. The reason is shown next to the score.

See also