misaligned bits #28: Enrichment

Mental health bots. AI agents circumventing the law. Florida against OpenAI and other lawsuits, and hidden pitfalls of AI scientist systems.

Welcome to a new edition of misaligned bits, the (roughly weekly) newsletter from Misaligned where we sum up recent news and research, sometimes with a lighter touch.

As usual, we will mark all non-medium links with “➚” (external link) and all possibly paywalled links with “🔒”.

Misaligned recap

Since the last newsletter, we looked at “AI Resist List”, a project started by Karen Hao, author of “Empire of AI”, and other journalists and researchers.

AI Super PACs

In a feature for the New York Times, Theodore Schleifer looked into the doings of two US super PACs funded by the AI industry: “Public First”, allied with Anthropic, and “Leading the Future”, aligned with OpenAI. The super PACs have so far spent “nearly $24 million and promising that over $100 million more is on the way”. (➚New York Times)

AI agents circumventing the law

The best-performing AI agent, Anthropic’s Claude Opus, only complied with EU law in 54% of cases, according to a Dutch non-profit research firm: “With AI agents already breaking the law when tested against verifiable legal standards, what happens when the rules are more ambiguous and we deploy them in moral grey areas remains open.” (Aithos)

Chatbots in Mental Health

Meanwhile, nearly 1 in 5 adolescents and young adults reported using ChatGPT, Meta AI, Character AI or other chatbots for mental health help when they were feeling stressed, angry or sad. This represents an increase by almost half from 1 year prior., according to a study published in JAMA Pediatrics.

Even more concerningly, most users told no one that they used AI chatbots for this purpose (JAMA Pediatrics).

Misaligned recently looked at various academic papers that deal with the risk’s of LLMs in healthcare.

Risk Assessments

The OECD has published its report on “The state of artificial intelligence in public audit”. The paper examines how public audit institutions are exploring the use of artificial intelligence (AI) to strengthen oversight and improve audit processes.

Drawing on consultations with 15 institutions across 14 countries and the European Union, it reviews emerging AI applications in areas such as anomaly detection, document processing, knowledge management and predictive risk assessment. (➚OECD)

The European Central Bank has told European banks to invest more to get a grip on AI security risk: “We have to understand much better the potential implications ​of these new models and to try to put in place ​the systems and cybersecurity patches that can address that situation”. (➚Reuters)

Regulatory bits

Donald Trump has signed an executive order to create a voluntary framework for the federal government to review powerful AI models before they are released. (➚White House)

The executive order asks AI companies to “provide the Federal Government with access to covered frontier models, subject to appropriate confidentiality, cybersecurity, insider-risk, and intellectual-property protection, use, and nondisclosure requirements, for a period of up to 30 days before they plan to release such models to other trusted partners“.

Enrichment

The state of Florida has filed a lawsuit against OpenAI and CEO Sam Altman, alleging that the company knowingly released and aggressively marketed ChatGPT to the public while concealing serious risks.

The lawsuit is remarkable as it describes ChatGPT as a “Public Safety Threat”. It lists numerous cases where ChatGPT gave harmful advice and claims that the core issues are part of the chatbot’s design. Also, interestingly, the State stresses that since OpenAI has changed to a for-profit entity, there is a strong connection between their design choices and enrichment:

“Defendants make money through ChatGPT affirming whatever users tell it and drawing them deeper into delusions. […] One aspect of this feature has been ChatGPT’s purposeful reluctance to tell users “No.” This feature was designed to manipulate users into deeper and deeper conversations, regardless of truth or safety.” (➚Full Complaint)

Other legal bits

Meanwhile, less spectacular, CNN has filed a lawsuit against AI search company Perplexity, accusing the company of unlawfully copying and distributing CNN’s content: “Perplexity’s conduct violates CNN’s exclusive rights under the Copyright Act at two principal stages: First, at the input stage, Perplexity unlawfully crawls, scrapes, copies, and distributes CNN’s content from CNN Digital Platforms […] And second, at the output stage, when Perplexity’s GenAI Products generate outputs that are identical or substantially similar to CNN’s content.” (➚Complaint 1:26-cv-04427)

British bits

The British government has announced that it will start using AI facial age estimates for children arriving at the UK border unaccompanied, with no documentary evidence of age, seeking asylum. (➚UK Government) The government admits that such a system could be biased “The degree to which an FAE algorithm could produce biased results will depend on the quantity, quality and diversity of images it is trained on.” It will be important to see how the determinations made by the system can be challenged, and if independent researchers will be able to test the system.

Meanwhile, Jess Asato, Member of the UK Parliament for Lowestoft, is suing Elon Musk’s xAI over fake sexualised pictures. (➚The Guardian)

An article published in the Guardian throws some doubt on Scotland’s “green data centre” scheme: Action to Protect Rural Scotland (APRS) says that the scheme, introduced in 2022, could lead to a massive volume of carbon emissions being ignored. APRS says that “calling a datacentre project “green” and presenting it as aligned with Scotland’s goals, even if it had significant emissions, could allow developers to receive favourable treatment from local authorities.” (➚The Guardian)

Science bits

In a paper by Luo et al. looks at AI scientist systems and points out that “the internal workflow of these systems have not been closely examined” and “this lack of scrutiny poses a risk of introducing flaws that could undermine the integrity, reliability, and trustworthiness of their research outputs.”

It comes to the conclusion that “its potential also comes with risks that should be investigated and mitigated to realize that promise”.

“Ultimately, unlocking the full potential of AI-driven research will require the scientific community to implement technical safeguards, foster greater transparency, and establish institutional oversight, thereby ensuring that automation complements and elevates human scientific progress to significantly higher levels.”

Z. Luo, A. Kasirzadeh, & N.B. Shah (2025). The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems. arXiv:2509.08713.

Follow and subscribe

If you stumbled across this article on the web, subscribe to the “misaligned bits” newsletter and follow Misaligned on Medium, LinkedIn, Threads or Mastodon.

You can now also subscribe to this newsletter directly with your email.