Google AI links ex-employee's number to company

On October 10, 2024, a concerning incident involving Google's AI technology and potential privacy violations came to light when a former video game company employee received an unexpected phone call. The incident raises questions about data handling practices and compliance with privacy regulations like the General Data Protection Regulation (GDPR) in Europe.

Rogelio Delgado, a former 3D artist at Funcom, a Norwegian video game developer, reported receiving a call from an attorney complaining about content in the game Conan Exiles. Delgado, who had not worked for Funcom in nearly a decade, was puzzled by the call. Upon investigation, it was discovered that Google's AI had erroneously listed Delgado's personal phone number as the main corporate office number for Funcom.

The incident unfolded when the attorney's clerk used Google to search for Funcom's contact information. Google's AI Overview feature displayed Delgado's phone number as the official corporate contact, despite him having left the company years ago. This misinformation led to the misdirected call and raised concerns about the accuracy and implications of AI-generated content in search results.

Data protection concerns

From a GDPR perspective, this incident potentially violates several key principles:

1. Data Accuracy: Article 5(1)(d) of the GDPR requires personal data to be accurate and kept up to date. The AI's association of Delgado's current phone number with his former employer fails this requirement.
2. Purpose Limitation: Article 5(1)(b) states that personal data must be collected for specified, explicit, and legitimate purposes. Using Delgado's personal contact information for corporate purposes without his consent likely violates this principle.
3. Data Minimization: Article 5(1)(c) mandates that personal data processing should be limited to what is necessary. Displaying personal contact information of a former employee as current corporate contact details exceeds necessary processing.
4. Lawful Basis for Processing: Article 6 requires a valid legal basis for processing personal data. It's unclear what lawful basis Google could claim for associating Delgado's current personal information with his former employer.

Potential consequences

If this incident had occurred within the jurisdiction of the GDPR, it could have serious implications:

1. Regulatory Scrutiny: Data protection authorities might investigate Google's data handling practices and the mechanisms behind its AI-generated content.
2. Fines: GDPR violations can result in significant fines of up to €20 million or 4% of global annual turnover, whichever is higher.
3. Reputational Damage: Such incidents can erode public trust in AI technologies and raise concerns about data privacy practices of major tech companies.
4. Individual Rights: Delgado and others affected by similar incidents could potentially exercise their rights under GDPR, including the right to rectification of inaccurate data and the right to erasure.

Industry impact

This incident highlights the broader challenges faced by the tech industry in managing AI-generated content:

1. AI Accuracy: The case underscores the need for robust mechanisms to ensure the accuracy of AI-generated information, especially when it involves personal data.
2. Data Sources: Questions arise about the sources of data used to train AI models and how frequently this information is updated and verified.
3. Transparency: There's a growing demand for greater transparency in how AI systems make decisions and present information.
4. Accountability: The incident raises questions about who bears responsibility when AI systems make errors that impact individuals' privacy.

According to Barry Schwartz, a search engine optimization expert, such incidents are not unprecedented. He shared a similar experience where his company's phone number was mistakenly associated with Google support in image search results, leading to numerous misdirected calls.

Lily Ray, an SEO professional, emphasized the potential consequences of even minor AI errors, stating, "As I've been saying since day 1, even if 1 in 1,000 AIO results are 'just a little bit wrong,' that little bit can have devastating consequences."

Jono Alderson, an independent technical SEO consultant, described the situation as "Google's SGE nightmare," suggesting that this incident might be indicative of broader issues with Google's Search Generative Experience (SGE) feature.

As AI technologies continue to evolve and integrate into search engines and other digital platforms, incidents like this highlight the need for:

1. Improved AI Training: Enhancing the accuracy and contextual understanding of AI systems to minimize errors in data association.
2. Regular Data Audits: Implementing systematic checks to ensure the currency and accuracy of information presented by AI systems.
3. Clear Correction Mechanisms: Establishing efficient processes for individuals and organizations to report and correct AI-generated misinformation.
4. Enhanced Privacy Safeguards: Developing stronger protocols to protect personal data from misuse or misrepresentation by AI systems.
5. Regulatory Adaptation: Evolving data protection regulations to address the unique challenges posed by AI-generated content and search results.

Key Facts

• Incident Date: October 10, 2024
• Individual Affected: Rogelio Delgado, former 3D artist at Funcom
• Company Involved: Google (AI technology)
• Potential GDPR Violations: Data Accuracy, Purpose Limitation, Data Minimization, Lawful Basis for Processing
• Maximum GDPR Fine: €20 million or 4% of global annual turnover, whichever is higher
• Google AI Feature Involved: Search Generative Experience (SGE)