At the scale of a product with more than 300 million users, data quality has to be part of the platform. I built historical data reconciliation, an automated watchdog that investigates anomalies, validation for every release, and one governed registry of event definitions. The watchdog found real production issues in its first week.
Figures describe engineering validations and specific reference windows. They are not current public company KPIs.
Detection runs automatically, investigations produce evidence, releases record an approval, and definitions live in one governed place. Quality checks are part of normal platform operation.
Historical data was healed rather than discarded. Contaminated sessions were untangled, sessionless events were absorbed into valid nearby sessions, and passive server noise was suppressed from active user counts. Raw data stays untouched, and audit columns explain every correction that was applied.
A watchdog listens for completed pipeline runs, applies deterministic anomaly detection, and then uses AI to investigate within set limits. It produces a daily digest with lineage, impact and suggested ownership, suppresses known noise, and operates under strict query cost limits. It caught multiple real production issues in its first week.
A governed Data QA environment separates personal development, staging backfills, release QA and production. dbt test runs are linked to releases, outcomes are recorded as pass, fail, unsupported or insufficient traffic, and a named reviewer approves the result. Each release now has a recorded outcome and owner.
A governed event registry separates raw instrumentation from analytical meaning. Canonical concepts survive platform changes and event redesigns, rules retain the dates when they apply, and both daily and realtime processing resolve the same definitions. The registry was validated against a production partition with 1.6 billion rows with zero metric disagreements, at about 0.1% additional build cost.
A cleanup audit identified roughly 20.8 TiB of obsolete storage while explicitly protecting every active dbt, Airflow, dashboard, ML and analyst dependency. Cost went down. Nothing broke.