Surfshark today opened "Bot or Not" to the public - a browser-based social media simulation game built with master's students in Interaction Design from Malmö University - alongside data from a 710-person experiment showing that 47% of participants failed to correctly distinguish AI-generated comments from human-written ones. The game, which first ran at the UNFOLD exhibition during Milan Design Week in May 2026, is now available at botornot.one.

The finding arrives at a moment when the scale of automated activity on social platforms has reached a level that strains conventional assumptions about content authenticity. According to Surfshark, citing the 2025 Bad Bot Report, automated bots accounted for 51% of all web traffic in 2024. The same research from Surfshark estimates that social media platforms now delete 6.3 billion fake accounts and 11.1 billion spam content pieces annually. On the dark market, fake social media accounts are available from as little as $0.08.

How the experiment worked

The "Bot or Not" game drops players into a simulated social media comment section. Each session lasts 120 seconds, during which participants must identify 10 bot-generated comments from a mix of real and AI-produced content across one of four topics. Two topics - data centers and pineapple on pizza - were chosen as relatively low-stakes subjects for most participants. Two others - immigration and women's rights - were selected because they tend to provoke stronger emotional responses.

Performance was measured using two distinct metrics. Bot-detection captures what share of actual bots a player successfully identified, showing how many slipped past undetected. Accuracy measures how often a player who flagged a comment as bot-generated was actually right - a metric that captures false positives, where real human comments are accused of being machine-made. Both matter: low bot-detection means bots circulate unchallenged, while low accuracymeans real users get misidentified and potentially silenced or reported.

Surfshark analyzed data from 710 participants who played during the week-long UNFOLD exhibition and agreed to share their results. The sample skewed young: 47% were in their 20s, 8% were teenagers or younger, and 14% were aged 31 to 40. The 41-50 and 50-plus brackets each accounted for 7% of the sample, with 17% declining to share their age.

Across all 710 players, the average bot-detection rate was 58% and the average accuracy rate was 66%. More than four in ten bots went undetected, and roughly one in three accusation calls was wrong even for an average player.

Nearly half failed to beat the game

At headline level, 53% of participants won - meaning they caught more bots than they misidentified real humans as bots. The remaining 47% did not. That near-even split is the experiment's most direct finding. It does not mean humans are helpless, but it does mean that instinctive detection is barely better than chance when applied to a controlled simulation.

"The 'Bot or Not' game and experiment help us keep connecting the dots and better understand the influence bad bots have on us, real social media users. Earlier this year, we found that major platforms remove over 6.3 billion fake accounts every year - roughly 47 times the annual number of babies born worldwide (around 135 million). Bots are being generated by the billions, and our latest experiment shows that half of the participants can no longer tell them apart from real people. This trend will accelerate, as the technology lets bots blend in seamlessly with real human profiles," said Justas Pukys, Senior Product Manager at Surfshark.

The numbers Pukys cites carry implications that extend well beyond individual users. An Adalytics investigation published in March 2025 found that leading verification systems routinely failed to detect non-human traffic even when bots openly identified themselves, with the research finding at least 40% of web traffic consisted of fake users or automated bots across more than two million websites. When human users themselves struggle to identify bot content, the challenge reaches from automated fraud detection all the way to the audiences those tools are designed to protect.

The platform divide: Reddit and X lead, Facebook trails

One of the experiment's sharper findings concerns how much the primary platform a person uses predicts their detection ability. Surfshark asked participants which social platform they used most often, and the results show communities that have developed very different instincts.

Reddit and X users recorded the highest bot-detection rates, both at 68% - 10 percentage points above the overall sample average. X users also posted an accuracy rate of 71%. According to Surfshark, the text-heavy, argumentative nature of these platforms may sharpen users' ability to read comment patterns critically. Reddit's community moderation culture, with its persistent focus on identifying low-effort and inauthentic posts, may produce similar habits.

TikTok-first users showed a different profile. Their accuracy rate of 72% was the highest of any platform, meaning they were least likely to issue false accusations against real humans. Their bot-detection rate of 61%, while below Reddit and X, still exceeded the sample average. TikTok users appear to be cautious: slower to label something a bot, but more often correct when they do.

Facebook users struggled most. Their bot-detection rate of 47% was the second-lowest in the study, barely above the 40% recorded by Threads users - who posted the weakest bot-detection figure of any platform. Facebook's accuracy rate of 55% was the worst of any major platform. Put differently, Facebook users both missed more bots than most and made more false accusations than most.

The advertising significance is not trivial. Social media invalid traffic has been a documented concern across open programmatic for years, with Pixalate's 2024 global benchmark report covering ad transactions from platforms including Facebook, Instagram, TikTok, and others finding wide variation in invalid traffic rates by platform and device type. Platforms where users demonstrate the weakest detection ability are also platforms where advertiser confidence in audience quality is already under scrutiny.

The generational cliff at 40

Age proved to be one of the experiment's strongest predictors. Surfshark describes a "generational cliff" that becomes visible once players cross 40.

Players under 20 were the sharpest bot-hunters in the dataset. They achieved a bot-detection rate close to 65% and an accuracy rate above 71%. Performance remained broadly stable through the 21-30 and 31-40 age brackets. Then it fell sharply.

Participants aged 41 to 50 recorded a bot-detection rate of just 42% - a drop of more than 20 percentage points compared to the under-20 cohort - and an accuracy rate of 59%. Both figures fell below the overall sample average by a meaningful margin. Players over 50 improved only marginally on the 41-50 group.

The combination is troubling. Older users let more bots pass undetected and simultaneously issue more false accusations against genuine human contributors. Those two errors pull in opposite directions: real users may face unwarranted reports, while AI-generated influence campaigns face less resistance. User-generated reporting is one layer in platform safety infrastructure. If older user communities are flagging at lower accuracy, the signal quality of community-sourced moderation falls precisely in the groups where bots face least scrutiny from automated systems.

When emotional topics degrade detection

The experiment's topic structure produced some of its most striking data. Surfshark chose four subjects calibrated to represent a range of emotional stakes, and the difference in performance across topics was substantial.

On data centers - a technical topic unlikely to provoke personal investment for most participants - players achieved the study's highest scores: 71% bot-detection and 76% accuracy. Even the mildly contested pineapple-on-pizza topic maintained reasonably strong results, with 64% bot-detection and 69% accuracy.

The moment immigration entered the simulation, both metrics declined. Bot-detection dropped to 54%, meaning nearly half of all bots in that thread were missed. Accuracy fell to 63%, with more real comments misidentified as machine-generated. Women's rights showed the weakest results of any topic: bot-detection at 49%, below the midpoint, and accuracy at 61%. Participants missed more bots than they found.

According to Surfshark, emotional and ideological engagement appears to redirect attention in ways that undermine pattern-recognition. That mechanism matters directly for how social platforms are used in political and advocacy contexts - the precise environments where coordinated bot activity is most likely to be deployed. The content dynamics that drive engagement on politically charged threads are, based on the experiment's results, the same conditions that make bot content hardest to identify.

For marketing teams relying on social listening or sentiment analysis across brand discussions that touch social issues, this creates a data quality problem. Synthetic comments embedded in emotionally charged threads are harder for humans to flag, which affects moderation queues and can contaminate the datasets that social analytics tools use to estimate public opinion. AI-generated low-quality content sites have already been identified by Integral Ad Science as a structural threat to advertising quality, with the company's July 2025 analysis noting that over 90% of known AI-generated sites remained absent from leading DSP blocklists.

Moderate usage, not maximum exposure

Surfshark also measured performance against how much time participants spent on social media. Non-users posted the weakest baseline: 40% bot-detection and 58% accuracy. Users who checked social media several times a day performed better - 59% and 67% respectively - suggesting that exposure builds some familiarity with bot patterns.

But heavy usage did not extend that advantage. Users who described themselves as online "almost all the time" showed an accuracy rate of 63%, lower than the 70% recorded by users who checked their feeds only a few times a week. The most accurate bot-hunters were moderate users, not the most engaged ones. Constant exposure appears to erode the critical attention that makes detection possible, a pattern that mirrors research on habituation in high-stimulation environments.

What this means for advertising and marketing

Automated fraud detection systems are the primary defence against non-human activity in programmatic environments. But human moderation and user reporting remain part of the infrastructure. The experiment's results suggest that layer is considerably weaker than platform safety assumptions typically reflect.

Microsoft Clarity's bot activity dashboard, which became generally available in January 2026, gave marketers and analytics teams direct visibility into AI bot traffic at the property level for the first time. Cloudflare's April 2026 research with ETH Zurich documented how AI bot traffic is disrupting web caching layers, with implications for ad delivery infrastructure and measurement reliability. YouTube clarified how invalid traffic restrictions apply to creator channels in April 2026, addressing confusion among creators about why their ad revenue was being restricted when they had not generated the problematic traffic themselves.

The Surfshark data adds a human dimension to those technical findings. CTV fraud rose 140% in 2025 according to a DoubleVerify report published in May 2026, with bot fraud accounting for 82% of violations in North America. The convergence of rising fraud sophistication and declining human detection accuracy across social platforms means that the 47% failure rate in Surfshark's experiment is not just a digital literacy curiosity - it reflects a gap in one of the layers the industry relies upon.

The Cybersecurity Advocacy Fund

Alongside the public game launch, Surfshark today announced a Cybersecurity Advocacy Fund offering up to €100,000 in annual financial support for students, researchers, and creative developers working on cybersecurity awareness projects worldwide. Applications will open in September 2026.

The fund extends the model behind the "Bot or Not" project itself. The UNFOLD exhibition, where the game debuted, is a university design competition staged during Milan Design Week and described by Surfshark as the world's largest trade fair. The experiment grew from a collaboration between a commercial cybersecurity company and interaction design students at Malmö University - a structure the fund is designed to replicate at broader scale.

Timeline

  • 2024: Automated bots claim 51% of all web traffic, according to the 2025 Bad Bot Report cited by Surfshark
  • July 2024: Pixalate publishes global social media IVT benchmark report, covering platforms including Facebook, Instagram, TikTok, and YouTube
  • March 2025: Adalytics investigation finds at least 40% of web traffic consists of fake users or automated bots; leading fraud detection systems routinely miss them
  • July 2025: Integral Ad Science identifies AI-generated slop sites as a critical threat to digital advertising effectiveness
  • November 2025: Cloudflare publishes a registry format for bot and agent authentication based on cryptographic verification
  • January 2026: Microsoft Clarity launches bot activity dashboard, giving marketers visibility into AI bot traffic at property level
  • April 2026: Cloudflare and ETH Zurich publish research on AI bots disrupting web caching layers
  • April 2026: YouTube publishes FAQ on invalid traffic clarifying how ad restrictions apply when non-human traffic is detected on creator channels
  • May 2026: CTV fraud schemes rise 140% according to DoubleVerify report; bot fraud accounts for 82% of violations in North America
  • May 2026: Malmö University interaction design students debut the "Bot or Not" simulation game at the UNFOLD exhibition during Milan Design Week
  • May 19, 2026: Surfshark publishes initial "Bot or Not" experiment findings based on 710 participants; 53% won, 47% failed; average bot-detection rate 58%, accuracy 66%
  • June 1, 2026: Surfshark today launches "Bot or Not" publicly at botornot.one; announces €100,000 Cybersecurity Advocacy Fund with applications opening September 2026

Summary

Who: Surfshark, a cybersecurity company headquartered in Amsterdam, in collaboration with master's students in Interaction Design from Malmö University.

What: The public launch of the "Bot or Not" browser game alongside experiment data from 710 participants showing that 47% failed to correctly identify AI-generated comments in a simulated social media environment. Bot-detection averaged 58% and accuracy 66% across the sample. Facebook and Threads users performed worst; Reddit and X users performed best. Players over 40 showed a sharp drop in performance. Political and social topics degraded detection ability compared to technical ones.

When: The initial experiment data was published on May 19, 2026. The public launch of the game and the announcement of the Cybersecurity Advocacy Fund took place today, June 1, 2026.

Where: The experiment ran during the week-long UNFOLD exhibition at Milan Design Week in May 2026. The game is now publicly accessible at botornot.one. Surfshark is headquartered in Amsterdam, the Netherlands.

Why: Surfshark's research is aimed at quantifying how well social media users detect AI bots, a question that has grown more pressing as bots claimed the majority of web traffic in 2024 and platforms collectively delete billions of fake accounts each year. The data matters for the advertising and marketing industry because it indicates how reliably human moderation and community reporting can supplement automated fraud detection systems - and under what conditions both layers break down.