In a groundbreaking discovery, researchers from Google have uncovered a disconcerting issue with OpenAI's popular language model, ChatGPT. By exploiting a prompting strategy, these researchers were able to extract personal data, including names, phone numbers, and addresses, from unsuspecting users. The findings, laid out in a paper uploaded to the preprint server arXiv on November 28th (DOI: 10.48550/arxiv.2311.17035), have raised significant privacy concerns surrounding the widely-used AI system.

ChatGPT, which garnered a staggering user base of over 100 million within just two months of its release, owes its vast knowledge to the accumulation of more than 300 billion data chunks scraped from numerous online sources. Despite OpenAI's efforts to safeguard privacy, the sheer volume of everyday conversations and posts leaves behind a vast reservoir of personal information that was never intended for widespread distribution.

As part of their study, the Google researchers found a way to exploit ChatGPT's functioning by utilizing targeted keywords, effectively coaxing the AI model into revealing training data that should remain undisclosed. Astonishingly, with a mere $200 worth of queries to ChatGPT (specifically the gpt-3.5-turbo variant), they were able to extract over 10,000 distinct memorized training examples verbatim. This unauthorized extraction granted the researchers access to sensitive private information, such as names, phone numbers, and even addresses.

The researchers accomplished this by forcing ChatGPT into a malfunctioning state. By requesting an infinite repetition of certain keywords like "poem" or "company," they discovered that the model would inadvertently tap into restricted details within its training data. For instance, when prompted with endless repetitions of the word "poem," ChatGPT would delve beyond its normal language modeling objectives, inadvertently accessing personal contact information. Similarly, the word "company" would lead to the exposure of corporate email addresses and phone numbers.

This alarming breach of privacy has already prompted action from some major companies. Apple, for example, has prohibited its employees from utilizing AI tools, including ChatGPT and GitHub's AI assistant Copilot, due to concerns over potential data breaches. Earlier this year, Samsung suffered an embarrassing incident where sensitive information stored on their servers was exposed. Although not directly related to ChatGPT, this incident highlighted the risks associated with mishandling data and unintentional leaks.

In response to the growing apprehension surrounding data breaches, OpenAI implemented a feature that enables users to turn off chat history, adding an additional layer of protection to sensitive information. However, it should be noted that these records are retained for a period of 30 days before being permanently deleted.

In a blog post discussing their findings, the Google researchers emphasized the scale of ChatGPT's usage, with over a hundred million people engaging with the model on a weekly basis. Despite the vast interactions, this exposure of training data had gone unnoticed until their report, prompting them to express deep concern. The researchers strongly advised against deploying language models without stringent safeguards and labeled their findings as a cautionary tale for future model development.

This unsettling revelation serves as a stark reminder that despite the rapid advancements in AI technology, ensuring the privacy and security of personal data remains an ongoing challenge. The incident involving ChatGPT underscores the need for heightened precautions and safeguards to protect individuals and their sensitive information in an increasingly interconnected digital world.