Deduplication is the process of recognising that two or more records describe the same underlying thing and merging them into one count. In advertising that thing is normally a conversion, a person, or an ad slot put up for auction. The records arrive from systems that observed the event independently, and without a rule for merging them the same purchase is counted twice, one viewer becomes three, and a buyer bids against itself for a single impression.

Most of the duplication is deliberate. Redundant collection has become standard practice because every individual measurement path leaks: a browser tag loses events to ad blockers, tracking prevention and consent refusals, while a server feed loses events that never reach the backend. Running both recovers more of the truth and manufactures the counting problem that deduplication exists to solve.

The mechanics of a deduplication key

Three components are required: an identifier both records carry, a rule that decides what counts as a match, and a window inside which matching is attempted.

Meta's implementation is the most widely imitated. The browser pixel sends an eventID, the server-side Conversions API sends event_id, and where those values and the event names agree, the pair resolves to one event. According to Meta's documentation, a match between events arriving within 48 hours of each other resolves to whichever arrived first, and when a server event and a browser event land within roughly five minutes, the browser or app event is preferred. Meta recommends an order number or transaction identifier as the key, which has the secondary effect of keeping two genuinely distinct purchases apart.

Google Ads uses a field labelled transaction ID, or order ID. Google's documentation states that an identifier arriving from an imported data source has to match the transaction_id captured by the conversion tag exactly, and that mismatched prefixes, suffixes or stray whitespace will prevent deduplication. The field is optional for offline conversion import and required for supplementary data sources connected through Data Manager. The same identifier is the recommended key for conversion adjustments and refunds.

Failure at this layer is silent. Nothing returns an error. The count simply doubles.

The IAB Tech Lab set out to standardise the pattern in ECAPI 1.0, a universal conversion API specification. The spec marks the event id as strongly recommended, expects the same value on the browser record and the server record, and instructs that where one identifier cannot be shared across channels the event should be submitted from a single channel instead. Before it, Meta, Google, TikTok and others each ran incompatible deduplication logic under different field names.

Duplicate conversions in practice

Implementation breaks in mundane ways. Two GA4 and server-side Tag Manager defects documented in September 2025 inflated event counts through duplicate hits, one of them traced to service worker communication, with workarounds limited to removing an iframe. Reloading an order confirmation page fires the same tag again, which is the original reason transaction identifiers exist.

Platform integrations import the problem wholesale. Google's server-to-server route for Shopify orders deduplicates purchases by matching transaction IDs, a guarantee that covers events from the Google and YouTube app rather than any third-party server feed running beside it. When four further shopping events joined the mapping on August 17, 2026, the protection did not extend to them: view_item_list, remove_from_cart, view_cart and add_shipping_info carry no transaction identifier, so merchants who already configured those events in their own containers face the same interaction recorded twice.

New entrants inherit the design. OpenAI's conversion tracking for ChatGPT ads requires a shared event_id across pixel and Conversions API, with the match resolved on pixel ID, event name and event_id, and custom_event_name substituting for the standard name on custom events.

One structural limit is easy to miss. Deduplication by transaction ID operates inside a single conversion action. Two separate conversion actions describing one sale, a common outcome when an Analytics-derived action is promoted alongside a native one, are not reconciled by any shared key.

Deduplication in the bidstream

Header bidding created the auction-side version of the problem. A publisher page runs one auction and fans the same impression out to several exchanges, each of which forwards it to the same demand-side platforms. Without a shared marker, a buyer sees five opportunities where one exists.

The OpenRTB Source object was introduced in version 2.5, circulated in draft in November 2016 with public comment open until December 16, and explicitly motivated by header bidding. Its tid field carries a transaction identifier the specification requires to be common across every participant in the request, including multiple exchanges. A community extension approved in June 2022 extended the concept to one identifier per impression object, since a single request may carry several discrete transactions.

That mechanism broke on August 27, 2025. A Prebid.js change generated bidder-specific values at ortb2.source.tid and ortb2Imp.ext.tid, so each demand partner received a different identifier for the identical opportunity. The IAB Tech Lab declared the implementation a material violation of the specification the same day, with chief executive Anthony Katsur stating that the approach "risks undermining the integrity and consistency of open technical standards". Publishers had argued that a shared identifier let buyers stitch requests together across sellers and expose relationships they preferred to keep separate.

Buyers described the loss in operational terms. The identifier had also served as a detector: an unusually high count of identical transaction IDs from one supply path signalled request duplication, and inconsistencies exposed identifier bridging by supply-side platforms. Prebid Server retains a createtids flag and a transmitTid activity control, which when denied withhold unique transaction identifiers from bid adapters entirely.

Deduplicating audiences

Net unduplicated reach predates digital advertising and depends on something the identifiers do not supply on their own: a link between the identifiers belonging to one person. That link is the job of a device graph, and its accuracy sets the ceiling on any deduplicated reach figure.

Google introduced Cross-Media Reach measurement in Google Ads in June 2024, aggregating reach and frequency across video campaigns and reporting unique reach with duplicates removed across devices, formats, sites, apps and networks. Display and Video 360 exposes the overlap directly through Unique Reach Overlap reports, whose pre-filtering was rebuilt on July 8, 2025 to calculate overlap, duplicate and exclusive comparisons from filtered entities. On April 10, 2026, Google announced campaign-level and placement-level dimensions for those reports.

Amazon built the same capability upward through its own hierarchy. Cross-order reach and frequency reporting arrived in December 2024, and Amazon's documentation cited a consumer packaged goods advertiser that found 34 per cent reach overlap across orders from three brands in one business line, alongside an average 6 per cent increase in unique reach from Frequency Groups drawn from internal September 2023 data. Both figures are the platform's own. Cross-account reporting followed in April 2026, reconciling Sponsored Products and Amazon DSP campaigns held in separate accounts.

Independent measurement firms sell deduplication as the product itself. Nielsen combined YouTube inventory on connected TV, mobile, desktop and linear television into one figure for Italy in June 2026, using Auditel panel data inside its identity resolution framework. AudienceProject added Pinterest to its combined view for the United Kingdom and Germany the same month, constructing deduplicated audiences without relying on cross-platform tracking identifiers.

Where it fails

Coverage stops at the ecosystem boundary. Amazon deduplicates Amazon, Google deduplicates Google, and no shared key crosses between them. Reconciling exposure across walled gardens requires a third-party panel, a modelled estimate or a governed join, which is why de-duplicated Meta measurement inside the LiveRamp clean room, announced in August 2026, was treated as a gap being closed rather than routine reporting.

Standards fragmentation compounds it. The Coalition for Innovative Media Measurement and the World Federation of Advertisers launched a strategic review of competing European cross-media approaches on January 15, 2026, examining whether any of them can deliver comparable metrics at scale.

The deeper dispute is that a deduplication key is also a linking key. The property that lets a buyer recognise one impression across five exchanges is the same property that lets it map relationships between sellers. The property that lets a platform merge a browser event with a server event is the same property that ties a backend record to a browsing session. The Prebid conflict is the clearest instance of that tension being resolved in favour of one side rather than negotiated.

Errors run in both directions. Absent deduplication overstates reach and conversions; aggressive deduplication understates them, discarding a genuine second order or collapsing two members of a household into one viewer.

Not the same as

Frequency capping limits how many times a person is shown an advertisement. Deduplication counts an exposure once. Capping decisions consume overlap reporting, but the two operate at opposite ends of the campaign.

Identity resolution supplies the key; deduplication applies the rule. A device graph or a clean room establishes that several identifiers belong to one person, after which the merge is arithmetic.

Attribution allocates credit for a conversion among touchpoints. Deduplication decides whether there was one conversion or two, a question that must be settled first.

Data deduplication in computer storage removes redundant byte patterns to reduce capacity. The shared word describes a shared logic, not a shared technique.

Timeline

  • November 2016: OpenRTB 2.5 circulates in draft with the Source object and its tid field, motivated by header bidding
  • December 16, 2016: Public comment on OpenRTB 2.5 closes
  • June 2022: IAB approves a community extension for per-impression transaction identifiers
  • June 2024: Google Ads introduces Cross-Media Reach measurement with deduplicated reach and frequency
  • December 2024: Amazon DSP adds cross-order reach and frequency reporting
  • July 8, 2025: Pre-filtering for Unique Reach Overlap reports is rebuilt in Display and Video 360 and Campaign Manager 360
  • August 27, 2025: Prebid.js generates bidder-specific transaction identifiers; the IAB Tech Lab issues a statement of non-compliance the same day
  • September 2025: Two GA4 and server-side Tag Manager defects producing duplicate events are documented
  • January 15, 2026: CIMM and the World Federation of Advertisers launch a review of European cross-media measurement approaches
  • April 10, 2026: Google announces campaign-level and placement-level dimensions for Unique Reach Overlap reports
  • April 2026: Amazon adds cross-account deduplicated reach and frequency reporting
  • May 2026: The IAB Tech Lab publishes ECAPI 1.0, standardising the event id as a deduplication key
  • June 2026: Nielsen ships four-screen deduplicated YouTube measurement in Italy
  • August 4, 2026: De-duplicated Meta measurement becomes available inside the LiveRamp clean room
  • August 17, 2026: Four Shopify events without transaction identifiers begin flowing into GA4

Summary

Who. Advertisers, publishers, measurement vendors and standards bodies all operate deduplication logic. Meta, Google and Amazon define it inside their own platforms; the IAB Tech Lab governs the cross-platform specifications through OpenRTB and ECAPI; Nielsen, AudienceProject and LiveRamp sell it as an independent service.

What. Deduplication merges multiple records describing one event, one person or one auction opportunity into a single count, using a shared identifier, a matching rule and a time window.

When. The auction-side mechanism entered OpenRTB with version 2.5 in late 2016; conversion-side deduplication became routine as server-side event APIs spread; the ECAPI standard formalised the conversion key in May 2026, and Prebid broke the auction key in August 2025.

Where. It runs in the browser and server tagging layer for conversions, in bid requests through the tid field, in reach and frequency reporting inside advertising platforms, and in clean rooms and measurement panels where platforms will not share keys directly.

Why. Redundant collection is now necessary for accurate measurement, and redundancy produces duplicates. Without a merge rule, reach appears additive when it is overlapping, conversions inflate, and buyers compete against themselves for a single impression. The same identifiers that make the merge possible also make linkage possible, which is why the mechanism keeps arriving at a privacy dispute rather than a settled standard.