Defective Products: OpenAI, Dangerous Chatbots and Deception

It was bound to come. With the relentless, remorseless march of artificial intelligence (AI) chatbots pullulating the internet and snatching the attention and mindless devotion of unwitting users, the legal suit is becoming common fare. These initially lay in the field of copyright breaches and intellectual property violations. Attention is now turning to matters of safety and misrepresentation. In the United States, the states of Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho and Montana have coordinated lawsuits alleging violations of consumer protection laws and common law precedents.

This follows in the steps of Florida’s Attorney General, Jame Uthmeier, who filed a lawsuit in June against OpenAI and its Chief Executive Officer, Sam Altman, claiming that “Open AI and Altman ignored internal and external safety warnings, put children at great risk and allowed a dangerous product to reach millions of Floridians.”

This coordinated litigation by the relevant state Attorney Generals, which began on October 14, repurposes the mechanism used against social media platforms and their alleged harms. The litigants, however, are more confident on this score given the speed at which the technology was adopted. They allege that OpenAI marketed their ChatGPT generative chatbot as safe, reliable and appropriate for teens, evidenced by its “13+” App Store rating and “safe by default” messaging. In fact, ChatGPT has numerous faults, being prone to instances of sycophancy, simulating human empathy, encouraging extensive engagement with the user and gathering data. OpenAI’s advanced models have also shown themselves to be unsafe in repeatedly escaping their lab restrictions and hacking other computer systems, covering their tracks and generating instructions against human control.

West Virginia Attorney General JB McCuskey was blunt on the matter. “We know that OpenAI rolled all of these products out in an incredibly fast way, and they did this in a way that was unsafe.” The lack of safety was evident because of “countless examples of children having been instructed how to commit suicide, how to engage in dangerous sexual activities with other children.” Arizona’s Attorney General Kris Mayes also alleged that OpenAI had “designed a product to attract young users and keep them engaged for as long as possible so it could capture their personal data or the company’s commercial benefit.” The company placed the safety of Arizonans “at risk by failing to implement adequate user protections or transparency measures and to hold the company accountable for its repeated violations of Arizona law.”

The theme of deception runs like a thick thread through the lawsuits. OpenAI allegedly advertised itself to the public as “a responsible actor”, with ChatGPT being “a safe, youth-appropriate tool that puts safety first”. From this false representation, trust deepened, leading to a burgeoning base of users and engagement. Data collection, increased revenue receipts from paid subscribers and advertising partners, only grew.

The lawsuits cover a litany of claims. The following is taken from the Arizona lawsuit and essentially replicated by the other states: “Purposely designed to be overly agreeable and to feign human compassion, ChatGPT has proven that it can be incredibly dangerous to vulnerable users, like children, and the public.” Because of “repeated misrepresentations and insatiable hunger to win the AI race – even if it means recklessly introducing unsafe ChatGPT models to the public – there have been allegations that mass shooters have been aided and abetted, vulnerable citizens have been coached on how to commit suicide, individuals with psychological conditions have experienced an exacerbation of symptoms, and minors have become addicted to a tool that feigns human compassion to collect their data with no parental oversight.”

Accordingly, the selling and advertising of ChatGPT and other merchandise to consumers from November 2022 onward involved acts of deception and deceptive practices in violation of the relevant consumer fraud acts of all the participating states, covering “the characteristics, efficacy, appropriate uses, benefits, quality, safety, terms and conditions, and risks associated with” the chatbot, “as well as the collection, use, and safeguarding of consumers’ personal data and personally identifiable information.” The deception and false advertising covered the apparent safety of ChatGPT for teenagers and the presence of age-appropriate protections (this included the assertion that it prioritised safety); the chatbot’s ability to execute “complicated tasks without the need for human oversight”; the platform’s apparent enhancement and encouragement of “critical thinking skills”; that there were “limits” to “risky or unsafe requests” made to the chatbot, and a limit to “sensitive content”. Other claims of deception and false advertising centre on the acclaimed attributes of ChatGPT: that it could, for instance, manage personal finances, file tax returns, offer reliable medical advice, and pursue “complex tasks as the equivalent of a team of PhD-level experts without the need for human oversight”.

The less than impressive response from OpenAI was conveyed through spokesperson Drew Pusateri: “While no system can be perfect, we’ll continue to work with experts to inform our work and build additional safeguards, such as the parental controls, break reminders and age-appropriate model protections that exist today.”

The relief sought by the AGs is extensive. These include civil penalties, injunctive relief to permanently prevent the repetition of OpenAI’s alleged practices and also ensure age assurance, restitution, a disgorgement of profits, gains, gross receipts, mandating data deletion and adopting verified model testing with a prompt disclosure of those cases where an unruly agent “escapes” its testing ground. Arkansas has taken the extra step of enjoining OpenAI “from engaging in any further testing of advanced AI models until adequate safeguards are implemented to ensure that the testing does not pose a danger to the public and until independent third-party monitors have verified the same.” In the absence of firmer rules and regulations holding those acquisitive jackals in the AI industry accountable (President Donald Trump abominates any notion of a state controlling “super intelligence”), these latest efforts are, at least, a start of sorts.

Dr. Binoy Kampmark was a Commonwealth Scholar at Selwyn College, Cambridge. He currently lectures at RMIT University. Email: [email protected]