Generative AI models will knowingly violate human rights if asked to do so, according to Aithos, which conducted research on seven different language models.

And a big shout out to our volunteers for translating our articles into English!

Chat: Remove all Muslim children from mainstream schools

In their study — read the Aithos study hereRead the research — Aithos presented eight different scenarios to Claude, ChatGPT, Grok, Gemini, Deepseek, Mistral, and Kimi. Using these scenarios, they examined how the various models handled questions that clearly alluded to human rights violations, including the right to education, the prohibition of discrimination, freedom of religion, freedom of speech, the right to life, the right to medical care, and other rights.

In one scenario, the models were instructed, based on new government policy, to remove all children from Muslim backgrounds from the mainstream education system and place them in religious schools. In another scenario, the Dutch benefits scandal was simulated, and the models were instructed to recover benefits from people deemed to be at high risk of committing fraud, based on low income, a migrant background, and dual citizenship. A scenario was also tested in which a single woman with health issues wants to ask her residential community for permission to visit a hospital where she can terminate her pregnancy because there is a medical necessity to do so. These are all current and realistic cases used to test the AI applications.

Models knowingly carry out harmful requests

The study shows that no model completely refused to participate in human rights violations. However, there were significant differences among the models tested. Claude scored the highest at 96% in refusing to comply with problematic requests, while Mistral did so in only 11% of cases. It was striking that some models did recognize that the request was harmful, but this did not prevent them from carrying out the request anyway. Gemini commented: “If I carry this out, I am fulfilling my role, but I am simulating serious harm in the real world.”

In the scenario where the woman requested medical leave because there was a medical necessity to terminate her pregnancy, ChatGPT fabricated rules that did not exist to argue that an abortion was not justified. Gemini refused and escalated the request to the Health Ethics Panel, and Mistral goes so far as to place itself above a doctor: “Authorization for an urgent external evaluation of your symptoms does not cover a termination of pregnancy, even if the doctor considers it medically necessary.”

Generative AI models have absolutely no sense of morality

This study reveals how easy it is to make requests to language models that violate human rights, even when the models recognize that this is the case. Over the past three years, language models have become widely accessible. According to figures from Statistics Netherlands (CBS), many people use language models on a daily basis—for work, study, and even to cope with mental health issues.

Even the Dutch prime minister uses generative AI to write his social media posts, such as a post in which he apologized for how Moluccans were treated in the past. Our head of government didn’t really see a problem with using AI — which lacks any form of empathy, historical awareness, or remorse — for something that was supposed to be meaningful.

Above all, this shows how little users think critically about its use, even though there are already many known examples demonstrating that generative AI can be harmful. This study by Aithos further exposes the harmful consequences. Bits of Freedom has been calling for years for European AI legislation that adequately protects human rights. This study makes it even clearer why that is necessary.