Roku cut the time required to onboard a new experimentation metric from two to three weeks down to under two days, according to a July 23, 2026 post on the company's engineering blog describing a rebuild of the internal platform that powers A/B test analysis across the streaming company's product organization.

The post, written by Roku data engineers Abhilash Mittapalli and Milind Nandankar and published under the title "Built Once, Serves Many: How We Rebuilt Experiment Analysis Platform at Roku," details a multi-quarter migration away from a system the authors describe as having evolved into "a tightly coupled monolith with diffuse ownership." The rebuild touched how hundreds of concurrent experiments are analyzed daily across teams working on recommendations, search, and merchandising, according to the post.

While the announcement centers on internal engineering practice rather than a consumer product or advertiser-facing feature, it lands at a moment when Roku's underlying data infrastructure has become a subject of direct commercial interest. Fox Corporation agreed on June 15, 2026, to acquire Roku for $22 billion in enterprise value, a transaction that explicitly cited Roku's first-party data assets as a central part of the deal's logic. The experimentation platform described in the July 23 post sits several layers beneath the advertising products that Fox is buying, but it is the infrastructure that determines how quickly Roku's product and measurement teams can validate changes to those very products.

What changed, in Roku's own words

According to the post, experiment analysis at Roku is not a single, uniform operation. The platform supports nine distinct analysis methods, each carrying its own parameter space and extract-transform-load requirements. User segmentation analyses need demographic parameters. Breakdown analyses need event-level identifiers. That diversity, the authors write, is intentional, since different product hypotheses require different statistical approaches.

The legacy system, according to the post, evolved organically over years alongside Roku's business, and three failure modes came to define it. The first was onboarding and maintenance overhead: even when the underlying compute logic was identical across metrics, each new metric required teams to repeat the same plumbing work, and typical onboarding took two to three weeks. Adding a single metric required updates across three to four table definition files, and over time this produced hundreds of redundant tables spread across environments, according to the post.

The second failure mode was the absence of a source of truth for metric definitions. Ownership and definitions were scattered across merge requests and internal documentation, and answering a basic question like what a given metric actually measures required what the authors call "manual archaeology."

The third was silent extract-transform-load drift. Core aggregation logic was duplicated in multiple places and diverged quietly over time. The post offers a concrete illustration: some pipelines computed a plain average, while others applied a normalized average scoped to identifiers meeting a specific enrollment criterion, with no mechanism in place to detect the discrepancy or determine which implementation was authoritative.

Roku's engineering team set a single design constraint to guide the rebuild. Data scientists would declare business logic only, not infrastructure. Physical table schemas, statistical computation, and experiment allocation and enrollment logic would be invisible to whoever authored a metric.

Splitting a metric into two files

The new architecture decomposes metric development into two artifacts, according to the post. The first is a SQL file containing the metric's business logic: the grain, event filters, breakdowns, and base aggregation. The second is a YAML configuration file that specifies how to aggregate that metric, along with metadata including the owner, applicable breakdowns, and aggregation type.

Data scientists using the new system write SQL defining metrics such as sessions and bounced sessions for experiment evaluation, according to the post, with a bounce rate metric calculated as the ratio of bounced sessions to total sessions, averaged at the variant level between test and control groups. Neither the SQL file nor the YAML configuration handles experiment metadata such as variant assignment or enrollment, nor do they create new tables or manage intermediate data. All of that, according to the post, is abstracted away from the developer and handled behind the scenes.

At runtime, analysis requests arrive through the Airtable API, carrying an experiment identifier, an analysis type such as power analysis or lift analysis, an analysis window, and segmentation parameters. A Python enrichment layer then joins those request parameters against experiment metadata covering enrollment dates, variant assignments, and allocation windows. It normalizes all metrics into a long format by pivoting each metric column into a row, which the post states eliminates the need for per-metric-type schema maintenance. It injects breakdown computation logic for whatever segmentations were requested, and it emits Airflow tasks with the correct dependency graph for the analysis type in question.

The orchestration layer then parses the runtime configuration to select the correct extract-transform-load path, routing execution to either Trino or Spark depending on cluster utilization and estimated data volume, which is configured based on task profiling, according to the post. High-volume, computationally complex analyses run on Spark. Lower-volume analyses run on Trino, where query latency is lower. A fallback mechanism reruns tasks on Spark if an attempt to execute on Trino fails because of memory constraints.

Results land in Roku's Google Cloud Storage data lake and are served through Looker or the company's own experimentation platform interface. Statistical methods are pluggable and can be extended without touching the extract-transform-load layer, because every analysis type consumes the same standardized long-format output; the post cites the potential future addition of a delta test for certain metrics, or Bayesian interpretation, as examples of methods that could be added under this architecture.

Migrating hundreds of KPIs without breaking anything

Roku's post devotes substantial detail to the operational difficulty of the migration itself, distinguishing it from a green-field build. Migrating hundreds of key performance indicators across dozens of teams was, in the authors' words, "equal parts org change and engineering." The new framework changed metric semantics in ways that eliminated raw-total comparisons in favor of averages and ratios, and some teams lost legacy features as a result, so the rollout was paired with what the post describes as deliberate stakeholder management and migration support.

Several specific technical problems emerged during that process, and Roku's engineers describe how each was resolved. Some metrics depended on a user's enrollment day, which conflicted with the platform's goal of hiding enrollment logic from metric authors entirely; the team added a controlled escape hatch allowing metric owners to supply fully custom SQL that takes over context injection and bypasses standard enrichment when necessary.

Distinct-user metrics presented a separate challenge, since they are not additive across time and require global deduplication over the full experiment window. Built-in support for non-additive unique counts was added so that individual metric authors would not need to hand-roll deduplication logic themselves. High-cardinality breakdown dimensions made segment-level analysis expensive to compute, so the team made cardinality a first-class configuration option, allowing developers to cap values, for example to the top 20 or 30, trading theoretical completeness for predictable performance.

Validation itself stretched across a multi-quarter effort. Each migrated metric was checked against its historical results, a process that, according to the post, surfaced bugs in the original legacy implementation that were fixed before the new version was certified. The team states it optimized deliberately for correctness over speed throughout this phase.

Measured results and what comes next

Roku reports three categories of measurable improvement from the rebuild. Key performance indicator development time dropped from weeks to under two days, with metric authors now writing only SQL and YAML while the platform itself handles extract-transform-load, statistical computation, and orchestration. Metric definitions are now versioned and owned, with every metric carrying a declared owner and a single authoritative implementation that is auditable through git history, turning what the post calls a previously investigative question, of what a metric measures and who owns it, into a simple lookup. Aggregation logic is now implemented in exactly one place, meaning statistical computation and enrollment logic live in a single location and silent drift between metrics is, in the authors' description, structurally impossible.

Two further investments are described as in progress. The first targets scalability: Roku is migrating to infrastructure with dedicated compute intended to handle peak seasonal load, such as holiday traffic spikes, without the complications of shared infrastructure, with an internal target of cutting analysis request-to-delivery time by 50 percent. The second concerns automated, plain-language interpretation of experiment results, which the post frames as reducing the time between a statistical output and a decision-maker's understanding of what that output means. According to the post, engineers would continue to receive per-metric detail while leaders would get what the authors describe as "a one-line read on experiment status" without needing to open a dashboard. A pilot rollout of that capability has, according to the post, driven strong engagement and acceptance, and is expected to expand to broader teams in a subsequent phase.

The post credits Roku's leadership team, named as Andrew Moskowitz, Rupa Kommineni, Kajal Poply, and Olga Natkovich, alongside product manager Gbenga Awodokun, and a project team consisting of Balaji K Ganesh, Simon Tsai, Abhilash Mittapalli, and Milind Nandankar.

Why this matters beyond Roku's own engineering org

For marketers and publishers who never touch a table definition file, an internal rebuild of experiment tooling might read as far removed from campaign strategy. It is not, and the reason has to do with what experimentation infrastructure actually gates inside a company like Roku.

Every feature change to Roku's home screen, its ad load configuration, its recommendation algorithm, or its measurement products passes through some form of A/B test before shipping. When onboarding a new metric took two to three weeks, that delay did not just slow data scientists down internally. It slowed the rate at which product, advertising, and measurement teams could validate whether a given change actually worked, which in turn slowed the rate at which new capabilities reached the market. A platform that collapses that cycle from weeks to under two days changes the tempo at which a company the size of Roku can iterate on the products that advertisers, agencies, and viewers eventually interact with.

That tempo question has taken on additional weight since the Fox transaction was announced. PPC Land's coverage of the deal detailed how Fox is acquiring access to Roku's 44 percent share of United States connected television viewing hours by operating system, along with what the investor presentation described as 100 percent authenticated first-party data across more than 100 million streaming households. Anthony Wood, Roku's founder, told investors on the deal's announcement call that the company was negotiating from strength rather than necessity, pointing to consistent quarter-over-quarter growth. Infrastructure of the kind detailed in the July 23 engineering post is part of what sustains that growth trajectory: it is the layer that determines how efficiently Roku's own teams can test and confirm the changes that produce the growth figures cited in that deal.

The rebuild also illustrates a recurring pattern in how streaming and ad tech companies have approached measurement claims throughout 2025 and 2026. Roku became the first major streaming publisher to use iSpot's Outcomes at Scale product for outcome-based campaign optimization, announced on January 6, 2026, with early testing against a SimpliSafe campaign showing a 23 percent increase in leads and a 31 percent increase in website visits for the optimized group relative to a control. Outcome-based optimization products of that kind depend on the same category of underlying capability that Roku's July 23 post describes: the ability to run rigorous, auditable statistical comparisons between a treatment group and a control group at scale, without introducing the kind of silent calculation drift that the post identifies as one of the legacy system's three core failure modes. A platform-level fix to that drift problem is, in effect, a prerequisite for the credibility of every outcome claim built on top of it, whether that claim concerns a home screen redesign or an advertiser's return on ad spend.

The emphasis on auditability, in particular, echoes concerns that have surfaced elsewhere in CTV measurement. The Video Advertising Bureau's recent expansion of its measurement vendor directory was itself a response to what PPC Land described as the collapse of self-graded television measurement, where platforms reporting their own performance figures without independent verification had become a point of friction for buyers. Roku's post does not claim independent, third-party auditing of its internal metrics, and none should be inferred. What it does describe is an internal governance structure, specifically versioned ownership through git history and a single authoritative implementation per metric, that addresses a narrower but related problem: ensuring that two teams referencing what should be the same metric are not silently measuring different things.

For a data scientist or an analytics-adjacent product manager working inside a comparable organization, the specific technical choices Roku made offer a transferable template. Splitting business logic from infrastructure into declarative SQL and YAML artifacts, standardizing output into a long format regardless of metric type, and building a controlled escape hatch for edge cases rather than trying to force every metric through a single rigid pipeline are all decisions with direct analogues in advertising measurement stacks, retail media clean rooms, and marketing mix modeling pipelines. The post does not frame itself as guidance for other companies. It reads, instead, as a technical retrospective, one written for an internal and external engineering audience rather than a commercial one. But the underlying problem it solves, namely how to let dozens of teams add new metrics quickly without those metrics silently diverging from one another, is not unique to Roku, nor to streaming, nor even to experimentation platforms narrowly defined.

Timeline

Summary

Who: Roku's data engineering team, identified in the source post as Abhilash Mittapalli and Milind Nandankar, with credited contributions from Balaji K Ganesh and Simon Tsai and leadership guidance from Andrew Moskowitz, Rupa Kommineni, Kajal Poply, Olga Natkovich, and product manager Gbenga Awodokun.

What: A rebuild of Roku's internal experiment analysis platform, replacing a monolithic system with a declarative architecture in which data scientists write a SQL file and a YAML configuration file per metric, while the platform itself handles table management, statistical computation, and orchestration across Trino and Spark.

When: The rebuild is described in a post published July 23, 2026, on Roku's engineering blog. The migration itself is characterized as a multi-quarter effort involving validation against historical results for each migrated metric.

Where: The platform operates across Roku's internal data infrastructure, serving product, advertising, and data science teams across domains including recommendations, search, and merchandising, with results served through Looker and Roku's own experimentation platform interface.

Why: The rebuild addresses three failure modes in Roku's prior system, namely onboarding overhead, absent metric ownership, and silent calculation drift, cutting metric development time from two to three weeks down to under two days. The change matters to the broader advertising and streaming market because it sits beneath the same data infrastructure that Fox Corporation cited as central to its June 15, 2026 agreement to acquire Roku for $22 billion, and because reliable, auditable experimentation is the same capability that underpins outcome-based advertising products such as the iSpot integration Roku launched on January 6, 2026.