Google updates Privacy Sandbox experiment guidance to address Invalid Traffic

The Privacy Sandbox team at Google yesterday announced an update to its guidance for addressing potential invalid traffic in experiments conducted within the Privacy Sandbox initiative.

This update, posted to the FLEDGE API announcements mailing list emphasizes the importance of data quality in ensuring the reliability of experiment results.

Privacy Sandbox and Experimentation

Privacy Sandbox is a collection of evolving technologies from Google designed to enhance user privacy while maintaining the effectiveness of online advertising. Marketers can participate in experiments to test these new features before they are fully rolled out.

Importance of Data Quality in Experiments

The accuracy of experiment results hinges on the quality of the data used. Invalid traffic, also known as bot traffic or fake clicks, can skew the results and lead to misleading conclusions.

How Google facilitates testing

To aid marketers in conducting Privacy Sandbox experiments, Google offers two testing modes:

• Mode A (Opt-in): Organizations can opt-in to receive consistent labels on a subset of Chrome browsers. This enables coordinated testing across different ad tech platforms using the same browser pool.
• Mode B (Global): Starting in January 2024, Google globally disabled third-party cookies for a small portion of Chrome browsers (approximately 1%). This allows marketers to test the effectiveness of their campaigns without relying on third-party cookies.

Both modes will continue until at least the second quarter of 2024.

Identifying Potential Invalid Traffic

Here are some key strategies to identify potential invalid traffic in your Privacy Sandbox experiments, as highlighted in the recent update:

• Monitor Population Sizes: Regularly observe the relative sizes of each labeled group. Significant discrepancies in group sizes could indicate invalid traffic. For instance, the number of users in the "treatment_1.1" group should be roughly equal to the number of users in the "treatment_1.2" group, and so on.
• Scrutinize Pre-Chrome 120 Traffic: Use caution when analyzing data from Chrome versions older than version 120. If your team that typically handles invalid traffic identifies suspicious user agents (software programs that mimic real users), it's advisable to exclude them from your test results.

Privacy Sandbox experiment labels

• Function: Labels are identifiers assigned to Chrome browsers participating in Privacy Sandbox experiments. These labels indicate which testing mode (A or B) a browser belongs to and whether it has specific features enabled or disabled (e.g., third-party cookies blocked, Privacy Sandbox APIs enabled).
• Accessing Labels: Labels can be accessed through a temporary "Cookie-Deprecation" value, obtainable via an opt-in HTTP header or JavaScript API. It's crucial to consult with legal counsel to determine if user consent is required for accessing labels in your jurisdiction (e.g., EU, UK).
• Availability: Labels are not guaranteed to be available in every instance. Users can disable features or delete cookies, which can prevent labels from being sent.