OpenAI has reportedly hired hundreds of contractors to read and evaluate real ChatGPT conversations as part of an internal effort to improve the chatbot’s responses, raising questions about how much users understand about human access to their chats.
According to 404 Media, which obtained internal documents related to the program, the initiative is known as Project Lily and involves contractors manually reviewing conversations between users and ChatGPT. The leaked material reportedly included reviewer instructions, internal discussions, scoring systems and examples of actual user chats.
Contractors are asked to examine conversations and judge how well ChatGPT responds. Reviewers may summarize what a user is trying to accomplish, compare generated answers and score responses based on factors including usefulness, tone and writing quality.
The process is also designed to identify behavior OpenAI wants to reduce, including excessive agreement with users, patronizing language, unnatural “AI-speak,” unnecessary emojis and responses that make ChatGPT sound as though it has human emotions or personal experiences.
Reviewers can see full conversations
One of the more notable details in 404 Media’s reporting is that reviewers may be shown full conversations rather than isolated prompts.
Some of those chats can contain sensitive or personal information. Usernames are reportedly removed, while OpenAI uses privacy filtering intended to strip identifying details before conversations reach reviewers. But the filtering process is not perfect, meaning certain personal information can still remain.
Reviewers may also receive additional context connected to a conversation, including summaries designed to explain a user’s previous interactions and preferences.
That matters because many people use ChatGPT for far more than simple searches. Conversations can include discussions about relationships, work, finances, personal problems or other details users may assume are being shared only with the chatbot.
At least one contractor interviewed for the report questioned whether ordinary users understood that another person could potentially read conversations submitted for review.
Users can opt out of model improvement
ChatGPT users do have an option to stop new conversations from being used to improve OpenAI’s models. On personal accounts, the setting can be disabled under Settings > Data Controls > Improve the model for everyone.
Turning the option off means future chats are not used for model training or improvement, although it should not be interpreted as a guarantee that no human could ever access a conversation for other purposes such as safety, security or legal requirements.
Users can also choose Temporary Chat for conversations they do not want added to chat history or used for model training.
The Project Lily documents ultimately shed light on a largely unseen part of AI development. ChatGPT may feel entirely automated from the user’s side, but improving how it responds can still involve real people reading, judging and scoring real conversations behind the scenes.
