Chess.com needed one analytics platform for experimentation, reporting, marketing and CRM. I rebuilt it around dbt on BigQuery, with incremental processing, isolated CI/CD, governed QA, and hot, warm and cold views for different time horizons. It supports a product with more than 300 million users. Thirty to forty people across four functions use it every week.
Figures describe the platform and specific engineering reference windows. They are not current public company KPIs.
The goal was one place where models are tested, documented, incremental by default and safe to change. That platform now feeds experiment readouts, executive reporting, marketing audiences and CRM. When someone at Chess.com asks whether a number is right, this is the system that answers.
I wrote the architecture proposal that made dbt the foundation of analytical engineering at Chess.com, then implemented it. Model lineage, data tests and standardized materializations replaced duplicated SQL and manually tracked dependencies. Custom Jinja macros handle incremental logic once, so every model gets it right instead of reinventing it.
Bronze, silver and gold layers with a unified event contract underneath. Daily and realtime processing resolve the same definitions, so intraday numbers and historical reports agree with each other. Terraform governs the warehouse itself: datasets, permissions and cost controls live in code.
I built a tiered view system for different time horizons. Hot views serve the newest data, warm views cover recent history, and cold daily views keep longer history predictable and economical. Consumers use one stable interface while refresh behavior changes by tier.
Core models are fully incremental and idempotent. A normal daily run finishes in roughly two minutes, and history is never reprocessed without a reason. Analysts ship model changes on their own, without waiting on an engineer to wire up a DAG.
A dedicated CI/CD project runs the test suite against fixtures that resemble production before anything merges. A governed QA environment separates personal development, staging backfills, release QA and production, with clear pass and fail outcomes and a named reviewer approving releases.
Centralized backfill DAGs in Airflow, permissioned and observable, running production backfills on dedicated QA reservations so they never fight the daily workload. Long history can be reprocessed as a scheduled, reviewed operation.