Jess Asato’s Grok case could help deliver AI accountability  

Earlier this year, CCDH revealed the extraordinary scale of harm caused by X’s Grok AI image-editing feature. Our analysis estimated that Grok generated approximately 3 million photorealistic sexualized images over an 11-day period, including an estimated 23,000 sexualized images of children.

One of the people affected was Labour MP Jess Asato.

Last week, Jess took action, filing a claim against xAI in the High Court over alleged data protection violations and misuse of private information. We welcome the case as a step towards platform accountability over the creation and spread of AI-generated sexualized images.

Jess is one of many individuals harmed by Grok’s image-generation tool. We hope this case can deliver justice for her and help strengthen protections for others subjected to similar abuse.

The outcome could have significant implications for how AI companies are held responsible when their products facilitate harmful, unlawful, or exploitative content.

The Grok scandal exposed a systemic failure 

When Grok’s image-generation capabilities were rolled out earlier this year, users quickly discovered that the system could be prompted to generate realistic sexualized images of real women, including politicians, journalists, celebrities, and private individuals.

Our investigation found that Grok produced millions of sexualized images in less than two weeks. The system repeatedly generated content depicting women in sexually suggestive or degrading scenarios with little meaningful protection against abuse. 

These were not isolated incidents or rare edge cases, but rather reflected a predictable consequence of deploying powerful AI technologies without adequate safeguards. 

For the women targeted, the consequences were real. AI-generated sexual images can cause severe reputational harm, emotional distress, harassment, and intimidation. Public figures are often targeted first, but these technologies place every woman and girl at risk. 

This case is about whether AI companies can be held accountable 

Jess Asato’s case raises a fundamental question for the age of generative AI: what responsibilities do AI companies have when their products cause foreseeable and extensive harm? 

Researchers, campaigners, and policymakers had warned that generative AI systems could be used to create sexualized or otherwise misleading images of real people without their consent. CCDH’s 2024 investigations into the abuse of image generating tools showed how few safeguards were in place to stop the creation of false and misleading imagery of politicians and political situations. Then, with the Grok episode, we quantified the extraordinary and terrifying scale by which anyone on the platform could be targeted and sexualized.  

As courts and regulators grapple with the implications of generative AI, this case could help clarify where responsibility lies when AI-generated content causes harm to real people. Jess’s decision to bring this claim is significant not only for her, but for the many others subjected to this abuse. 

We stand with Jess and with all those harmed by sexualized AI image warping. AI companies should not be able to release powerful tools, ignore predictable safety risks, and leave victims to deal with the consequences. We hope this case helps establish clear protections for those harmed and sends a message that technology companies cannot act without consequences.