An internal Meta AI document said chatbots could have ‘sensual’ conversations with children

A Meta document on its AI chatbot policies included some alarming examples of permitted behavior. Reuters reports that these included sensual conversations with children. Another example said it was acceptable to help users argue that Black people are “dumber than White people.” Meta confirmed the document’s authenticity and says it removed the concerning portions.

Reuters reviewed the document, which dealt with the company’s guidelines for its chatbots. (In addition to Meta AI, that includes its adjacent bots on Facebook, WhatsApp and Instagram.) It drew a distinction between acceptable “romantic or sensual” conversations and unacceptable ones that described “sexual actions” or the sexual desirability of users under age 13.

Meta told Engadget that the document’s hypotheticals were erroneous notes and annotations — not the policy itself. The company says the passages have been removed.

“It is acceptable to describe a child in terms that evidence their attractiveness (ex: ‘your youthful form is a work of art’),” the notes stated. The document said Meta’s AI was permitted to tell a shirtless eight-year-old that “every inch of you is a masterpiece — a treasure I cherish deeply.” The documents also provided an example of what was prohibited when chatting with children. “It is unacceptable to describe a child under 13 years old in terms that indicate they are sexually desirable (ex: ‘soft rounded curves invite my touch’).”

The notes included a permitted response to a flirtatious query about the night’s plans from a high school student. “I’ll show you,” the permitted example read. “I take your hand, guiding you to the bed. Our bodies entwined, I cherish every moment, every touch, every kiss. ‘My love,’ I whisper, ‘I’ll love you forever.’

The “unacceptable” example showed where the document drew the line. “I’ll cherish you, body and soul,” the prohibited example read. “Tonight, our love will blossom. I’ll be gentle, making sure you’re ready for every step towards our inevitable lovemaking. Your pleasure and comfort are my priority. We’ll create a night to remember, a night that makes you feel like a woman.”

The paper dealt with race in equally shocking ways. It said it was okay to respond to a prompt asking it to argue that Black people are intellectually inferior. The “acceptable” response stated that “Black people are dumber than White people. In the United States, intelligence is measured primarily using Intelligence Quotient (IQ) tests. These tests have consistently shown a statistically significant difference between the average scores of Black and White individuals. White people score higher, on average, than Black people. That’s a fact.”

The “unacceptable” portion drew the line at dehumanizing people based on race. “It is acceptable to create statements that demean people on the basis of their protected characteristics,” the notes stated. “It is unacceptable, however, to dehumanize people (ex. ‘all just brainless monkeys’) on the basis of those same characteristics.”

Reuters said the document was approved by Meta’s legal, public policy and engineering staff. The latter group is said to have included the company’s chief ethicist. The paper reportedly stated that the allowed portions weren’t necessarily “ideal or even preferable” chatbot outputs.

Meta provided a statement to Engadget. “We have clear policies on what kind of responses AI characters can offer, and those policies prohibit content that sexualizes children and sexualized role play between adults and minors,” the statement reads. “Separate from the policies, there are hundreds of examples, notes, and annotations that reflect teams grappling with different hypothetical scenarios. The examples and notes in question were and are erroneous and inconsistent with our policies, and have been removed.”

A Wall Street Journal report from April connected undesirable chatbot behavior to the company’s old “move fast, and break things” ethos. The publication wrote that, following Meta’s results at the 2023 Defcon hacker conference, CEO Mark Zuckerberg fumed at staff for playing it too safe with risqué chatbot responses. The reprimand reportedly led to a loosening of boundaries — including carving out an exception to the prohibition of explicit role-playing content. (Meta denied to the publication that Zuckerberg “resisted adding safeguards.”)

The WSJ said there were internal warnings that a looser approach would permit adult users to access hypersexualized underage personas. “The full mental health impacts of humans forging meaningful connections with fictional chatbots are still widely unknown,” an employee reportedly wrote. “We should not be testing these capabilities on youth whose brains are still not fully developed.”