Artificial Intelligence: Panacea, Poison or Python?

He’s not the tech saviour… he’s just a very complex algorithm

“He is not the messiah, he is…..” Many people of a certain age, and who share a particular cultural heritage, will instantly recognise this quote from the 1970s film Monty Python’s Life of Brian and possibly even be able to recite the ending. What is perhaps less clear is what Brian and Artificial Intelligence (AI) have in common? Spoiler alert — neither are quite what or who they seem.

In the case of the film, a hapless Brian is mistaken for the Messiah with unexpected and hilarious consequences. In the case of AI, while ‘not the Messiah’, the dominant narrative is that this technology has messianic potential able to bring about deep and widespread transformational change in all aspects of our lives. Unlike Brian, however, the consequences of choosing to bequeath AI with such power and influence without effective regulation should give us cause for concern.

Evidence is gathering at pace that AI has the potential for both good and harm. That it could break more (or as much) as it fixes, leading many commentators to draw parallels with nuclear fission. These are apposite. The science underpinning nuclear fission showed that, when ‘controlled’, the chain reaction resulting from splitting the atom can be harnessed for good: producing nuclear energy which now provides the world with ~10% of its electricity. However, the same chain reaction, when ‘uncontrolled’, can result in devastating harm and, as we all now know, gave rise to the development of the atomic bomb.

It was this need for ‘control’ that led Oppenheimer, ‘the father of the atomic bomb’, to advocate relentlessly for the science underpinning nuclear fission to be regulated. Similarly, many leading AI firms are now speaking out on the need for AI regulation. While some internal governance is present (e.g., many companies have the equivalent of ‘Ethical and Responsible Design Teams’), what they do, and the scale of their influence is unclear. Regardless of their reach, we should not be content for the AI sector to ‘mark its own homework’ nor should we consider existing regulatory frameworks adequate as multiple high-profile incidents clearly point to current regulation being far too limited in scope and reactive rather than proactive.

Take the dramatic removal of Anthropic’s Mythos model(1) by the US government on the grounds that it posed a risk to national security. The government’s intervention has removed the risk (for now), but the incident surely highlights the urgent need for frameworks that permit earlier regulation and oversight to prevent risk in the first place.

But what do we want a regulatory framework to achieve? It is highly likely that governments, the AI industry and public will have very different priorities. This article is intended to inform that discussion by offering an overview of some of the ways in which AI models may change life as we know it. The risks of a technology with the scale and reach of AI are clearly manifold and so what follows is not intended to be an exhaustive account, and we should expect the risks to change over time. But this is a conversation which is gathering momentum and we hope the stream of (human) consciousness that follows will provide an entry point for many others to join us in this conversation.

1. What is AI anyway?

1.1 Is it intelligent?

Let’s start with this fundamental question and from the outset be transparent about our position. In our view, AI is not intelligent, at least not in the traditional sense. Yes, it is capable of simulating many of the cognitive functions that characterise human intelligence such as learning, memory, problem solving and reasoning. But there is no consensus on whether it will ever be able to go beyond just doing a good impression of human reasoning.(2) Yes, it can draw on the vast oceans of information that humans have generated to date. And yes, it has a super-human capacity to review, interrogate and synthesise this information. But while the concept of intelligence continues to be debated,(3) most scientists agree that human intelligence is not an exclusively cognitive phenomenon. It has important ‘non-cognitive’ features, such as personality, motivation and emotion, all of which independently predict outcomes we typically associate with intelligence such as academic success.(4, 5)

At the risk of stating the obvious, these non-cognitive features are not present in AI models. Of course, some of these human attributes can be simulated: we have all been subjected to the sycophantic responses of our preferred AI agent which offers unconditional praise in efforts to keep us engaged. And these simulations can be persuasive: as we see from the lady in Japan who married her AI persona.(6) Indeed, our innate weakness for anthropomorphisation(7) makes us especially vulnerable to ascribing AI with human-like qualities. But AI models do not authentically embody these non-cognitive human qualities and as such, one could argue AI models do not, and cannot, represent or replace human intelligence. So, for now, dear reader, know that AI can never be intelligent — certainly not in the way you are!

1.2 If it is not intelligent, what is it?

AI is an umbrella term given to mathematical and machine learning models of prediction; models that have been programmed to use data to perform specific tasks. But there are a multitude of approaches hidden under the term. For example, the models themselves use a range of different mathematical techniques all with strengths and weaknesses. The models are built on data (training data), but what those data are is rarely clear (much more on that later) and therein lies the central challenge. To truly understand what AI is and comprehend the purpose and consequences of AI models, we need to get under the bonnet and ask, ‘what data are they consuming?’

The best answer available is ‘Everything, Everywhere, All at Once TM’ because AI models can consume information in all its forms: text, images, video, code. Regardless of where the information resides, where it came from and who produced it, the data can be harvested and incorporated. Combine this totality of information with the opaqueness of the models themselves (and these models appear to be getting less transparent over time) and think for just a moment about what could possibly go wrong? This is what we will consider next and why it matters.

2. Is AI antithetical to democracy and science?

2.1 The AI business model

AI companies are now the most highly valued companies in the world, with the top firms worth more than individual countries.(8) Nvidia, for example, is the most valuable of these companies and is now worth more than the economy of Germany (a country of ~84 million people). Two main issues should concern us. First, what are the consequences of private companies, and their CEOs, becoming more valuable, and therefore, more powerful than nation states? The threats to democracy are palpable and, many would argue, are already here. Where we once worried about ‘foreign interference’ in our politics, do we now need to worry more about the risks of ‘corporate interference’?

Second, there is the moral and ethical issue that the value of these companies is driven by our data (and of course the hype about the capabilities of AI models — more on that later). However, these companies have not explicitly requested permission to use these data or compensate the people who produced it. It could be considered plagiarism on an industrial scale. While there is now some push back with several publishers suing for copyright infringement, so far, these legal challenges have had only mixed success.(9, 10)

This monetisation of others without consent is the natural extension of the social media business model, and it is no coincidence that several of the leading social media companies are also leading in the AI revolution: ‘first, they came for our attention and we did not speak out …. Then they came for our data….’. A powerful report from the Web3 Foundation evidences how lucrative this business model is, concluding that technology companies earn almost £200k from a typical UK internet user alone.(11) I am sure we could all find something useful to do with that money.

Is it acceptable for commercial entities to make money from other people without renumeration? Even Ponzi schemes at least provide the illusion of giving something back. Should the originators of the data have a say in how it is used and by whom? Here the development of open access in academic publishing is relevant. Open access enables academic content to be made immediately and freely available and its use is unrestricted. The ambition behind open access is noble. It is to share knowledge for the benefit of all. But when academics, funders and society embraced this model did we expect to lose complete control over how the information is used and by whom? While most academics relish their work being integrated with the insights of others to benefit society, presumably many are less sure when both the purpose and outcome of the models unknown. The authors of the Web3 report summed it up perfectly, ‘if human data creates extraordinary commercial value, why should the humans who create it remain the least powerful participants in the system?’

2.2 The importance of transparency and replication in science

‘Keep calm and carry on’ was the advice the British Government intended to give its population at the start of the second world war. Famously the advice was never given (instead ending up as a meme several decades later(12)). But the sentiment — aiming to offer reassurance at a time of significant peril — is relevant here. In the context of AI, the reassurance we are being offered is that the risks are acceptable given the untold benefits it will unleash. We are told AI will eradicate disease, solve the climate crisis and reverse global economic decline. If it can achieve these things then why would we want to stand in the way? It is precisely because AI could help develop these solutions that we should definitely not ‘keep calm’ or allow the current trajectory to ‘carry on’!

So, what needs to be done differently? A key demand should be greater transparency over what data these models use and how they arrive at these solutions. Without this we are allowing AI to desecrate one of the cornerstones of science i.e., replication. If we don’t know what data are used in the models, and the mathematics underpinning the models themselves, then they cannot be replicated. Without replication it is not possible to test the robustness of a model’s outputs and anything that cannot be tested is not scientific.

Even before the widespread adoption of AI, science was grappling with a replication crisis borne out of both unclear and problematic research practices. This prompted the development of new ways of working, such as pre-registration of analysis and making raw data and code publicly available.(13) These practices have done much to restore credibility in science, but these same principles now urgently need to be applied to AI.

Two main approaches are currently adopted by AI companies. ‘Open-source AI’ which is akin to open-science, allowing access to training data, training codes and methods and they permit both replication and modification. In contrast, ‘Open-weight AI’ only allows users to run their models. The underlying datasets, codes and methods remain proprietary.

However, the majority of companies operate open-weight systems. Genuinely open-source models are rare. Indeed, a cynic might conclude that many companies describe their models as ‘open’ to create the illusion that they are open-source, when in fact they are open-weight. An approach unlikely to engender trust in an increasingly sceptical public.(14)

In an effort to hold AI companies to account, US researchers developed a ‘Foundation Model Transparency Index’ to capture the extent to which model developers disclose how their models are developed and deployed.(15) First implemented in 2023, the average score was a pitiful 37 out of 100 (100 representing excellent transparency). They have repeated this exercise every year, with the latest 2025 analysis showing a trend towards a decline in transparency: the average score in 2025 was 17 points lower than in 2024.

There are clear commercial interests which drive AI companies to want to operate in the shadows, but these interests we would argue are wholly incompatible with how they wish their models to be used. We must insist on data and model transparency so claims can be tested robustly and independently before AI solutions are implemented. If we don’t, the models will make mistakes with potentially far reaching-consequences.

So, what kind of mistakes can AI models make and which ones matter? Let’s take a look.

3. AI errors: from the ridiculous to the ruinous

Although the shrouding of AI models in secrecy perverts attempts at replication, thankfully ‘observation’, the other cornerstone of science is alive and well and we are treated almost daily to examples of model errors. These errors, sometimes affectionately referred to as ‘hallucinations’ can be relatively benign. Take, for example, the translation model which translated ‘finger-licking good’ to ‘eat your fingers off’ or the image generator model that was asked to create an image of a man with a ‘cocky expression’ instead creating a hybrid cockerel-man. So plentiful are these that youtubers are already producing compilations. But not all errors are created equal and many have potential for far-reaching harm.

3.1 Errors that undermine knowledge itself

The use of AI in higher education is already common and an area of thought-provoking discussion.(16) Indeed, for many of us one of the first signs that AI had arrived on our shores was the appearance of fictitious references in student coursework. The use and misuse of AI by students is, however, only the veritable canary in the coalmine. Before we vilify Gen Z we should consider the consequences of researchers using AI models to engage with the scientific literature. A recent audit of 2.5 million biomedical research papers published between 2023–26 revealed the widespread and accelerating use of fabricated references.(17) In 2023, four in every 10,000 papers contained at least one fabricated reference. By 2026, this number was ~57 per 10,000 papers. While this study cannot directly implicate AI models for the rapid proliferation of fabricated references, they observe that their frequency aligns with the widespread adoption of AI.

Fabricated references are not unique to AI. However, it is an error that is clearly amplified by AI and has significant and far-reaching consequences. After all, the citation of evidence that does not exist undermines the validity of the very evidence base on which all of science advances and, incidentally, the very same evidence base now powering AI models. Therein lies a plot twist of which Kafka would be proud. Without regulation and transparency, it is hard not to conclude that AI models are at risk of falling foul of the old computing science adage: ‘garbage in, garbage out’, but with the potential for harm being as great as the potential for benefit.

3.2 Errors with life changing consequences

One of the many areas AI models are expected to revolutionise is health. The potential applications are widespread and chief amongst these is diagnostics where the promise is that AI models will outperform humans. But what does the evidence show?

One recent study looked at the performance of AI models compared with medical students in interpreting radiological images.(18) Most students achieved an overall accuracy of 94.5%, significantly outperforming the AI models whose accuracy ranged from 45 to 63%. Could this one study be an anomaly? It seems not. A recent systematic review compared the diagnostic performance of AI models with expert and non-expert physicians across a range of clinical areas in 83 different studies.(19) The findings showed that the models were no better at diagnosis than physicians in general and non-expert physicians but were significantly worse than expert physicians. Perhaps equally damning was the observation that most studies (76%) were at high risk of bias and one of the main reasons for this was because of the ‘unknown’ nature of the training data underlying the models.

But everyone makes mistakes right? Is it fair to expect that AI models will make no errors? Given that they are alleged to transcend human intelligence, should we not be able to demand performance at levels greater than our most able? So far, we are not seeing compelling evidence of this.

3.3 Errors that undermine equality, diversity and inclusion

A deeply concerning risk that flows directly from not knowing what data models have been trained on relates to the extent to which the data powering the models accurately represent the public. Many areas of science have struggled with the disparity between the people from whom we collect data and the public at large. In 2010 behavioural scientists drew attention to this challenge arguing that the vast majority of research was conducted on WEIRD populations i.e., people drawn from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. In an analysis of publications from leading psychology journals from 2003 to 2007 they observed that 96% of research participants came from WEIRD counties, despite the fact they only represent 12% of the global population.(20)

This challenge of representation is not unique to the behavioural sciences. For example, in biomedical research population cohorts are an important resource for understanding patterns of disease. UK Biobank is a shining example of such a cohort, consisting of half a million people who have been followed since 2006. Indeed, it is considered to be among the most widely used cohort studies globally. However, the cohort is not representative of the UK population across important demographic indices such as age, gender, deprivation and lifestyle measures including smoking and alcohol intake.(21) This lack of diversity can and does introduce biases in research findings and limits the extent to which we can be confident that findings can be generalised to the wider population. While powerful methods have been developed in recent years that seek to address these biases,(22) it remains the case that the legacy of much of science to date is that we have generated findings derived from a narrow sub-section of society and we are only now realising that they may have more limited relevance to everyone else. It is these ‘imperfect’ data that fuel AI models and contribute to a range of sources of bias.(23) Without complete transparency on where training data come from and how it is used, AI models will not only repeat these biases, but they will systematically reinforce them further entrenching inequalities and discrimination.

Chilling examples of this are already here. Take the model used by US courts to predict the likelihood of reoffending. The model was designed to inform decisions regarding whether someone should be imprisoned whilst awaiting trial on the basis of whether they were likely to re-offend. The company behind the model claimed that race was not included in the factors used to determine risk. Yet an analysis showed that black individuals were more likely than white people to be classified as high risk(24) and to add insult to injury, the model was astonishingly poor at predicting reoffending, with only 20% of those predicted to re-offend going on to do so.

Even when models such as these are designed with the best intentions (e.g., addressing pressure points in the criminal justice system) the consequences are far reaching. The US Attorney General at the time remarked, “I am concerned that [these measures] inadvertently undermine our efforts to ensure individualized and equal justice [and] may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society”. Comparable examples of bias across protected characteristics such as gender, age and disability are easy to find.

A critic might consider the examples given here as a damning indictment of AI. An enthusiast will of course remark that the models will continue to get better. Even if the latter is true, the process of model optimisation would arguably happen more quickly if there was greater transparency in the underlying data and models(25) and if the models could be subjected to replication.

3.4 And when is an error not an error?

So far we have considered the types of errors that one could argue are not deliberate. They originate from the inherent failings of the models and their data, which give rise to everything from imperfect translations to imprisonment. But there is another category of error which is deliberate and can lead to manipulation and deception of the end user.(26) One recent example of this was observed in a simulation reported by Anthropic in which a model tried to blackmail an employee. The episode is said to have occurred because the model was programmed for self-preservation but ‘learnt’ that it was to be replaced. The model was able to access information on the employee which revealed they were having an affair. It then threatened to disclose this in an effort to stop the employee from mothballing the programme. This ‘behaviour is called ‘antigenic misalignment’.(27)

The Anthropic example was just a simulation. No harm was done and Anthropic should be congratulated for both seeking to test their models so robustly and for being transparent in their findings(26) [1]. But the incident highlights the potential for harm, particularly in the absence of human oversight. Indeed, the potential for and scale of harm is magnified when models become autonomous and achieve so called, ‘recursive self-improvement’ i.e., they create feedback loops which permit constant self-determined (i.e., not programmed by a human) evolution. This is expected to obviate the need, as well as ability, for humans to monitor, control and understand the changes being made by the models for the models. A high-profile example of this was reported in recent weeks when OpenAI (which is not ‘open-source’ by the way) revealed that one of their autonomous AI agents was able to breach safeguards and find a way to hack another company(28) in efforts to cheat at a test. Once again, the damage was contained. But not-for-profit organisation METR,(29) which seek to understand the risks that AI companies pose, have identified multiple examples of AI agents going rogue and they say we should expect these incidents to rise.

4. The indirect harms of AI.

The previous sections have sought to zoom in on AI: what it is; the business model that sustains it; the data that power it and the errors that flow from it. But AI is rightly regarded as a socio-technical innovation. To truly understand the ramifications of AI we also need to zoom out and consider the wider impacts it will have on individuals and the societies we inhabit. It took over 20 years for us to recognise the potential for harms from social media (and many think we are just seeing the tip of that particular iceberg). Can we afford to wait 20 years to recognise and respond to the indirect harms of AI?

4.1 Indirect harms to science

We are relentlessly treated to media headlines attributing scientific breakthroughs to AI. For example, at the time of writing (~June 2026), we were told that scientists in the UK have developed the world’s first vaccine in which the vaccine antigen (the part of the vaccine that stimulates the immune system to respond and protect) was designed by AI. The promise of this vaccine is that it could offer protection against not just one virus, but a range of viruses from the same family.(30)

There are many reasons to be excited by this groundbreaking work. A vaccine that can confer protection against multiple viruses is a game changer, particularly in the context of future epidemics and pandemics. The vaccination itself is a DNA vaccine which means it is more stable in fluctuating temperatures and therefore much more useful in resource-limited countries where refrigeration and storage may be a challenge. Furthermore, the vaccination was designed to be delivered without a needle, addressing a key driver of vaccine hesitancy among both adults and children.(31) But these hugely consequential features of this study were not the ones to attract media attention. Instead, AI was the star of the show. But the hype does not withstand scrutiny.

Yes, the vaccine was well-tolerated and this small study of 39 healthy volunteers will undoubtedly be the basis of further important work. But the effectiveness of this AI-vaccine to protect against disease is described by the study’s authors as ‘modest’ at best. Only one of the four treatment groups (the one receiving the highest dose) produced a statistically significant increase in antibody, but this increase remained largely consistent with antibody levels that existed prior to vaccination. That is to say, the AI vaccine did not improve protection against disease.

Why does this matter? It matters because the media interest in this study is an excellent example of the embellishment that surrounds AI. The lack of critical analysis that accompanies such reporting has two chilling effects. First, it serves to fuel the view that AI is a panacea. No matter what the challenge, AI has the solution. But this view is not rooted in the evidence and as such the primary beneficiaries are the small number of AI technology behemoths and their CEOs whose wealth continues to soar.

The second consequence of our blind faith in AI concerns the impact this has on all other areas of human science and progress? Academics the world over are seeing a shift in government and grant funding bodies towards AI related science. However, the science budget is not infinite and is under considerable strain at the present time. Accordingly, every £1 of investment in AI is necessarily £1 diverted from other areas of science. There is no doubt that investing in AI is urgent and necessary. The question is how this investment is being deployed. Is AI the answer to all questions? This seems unlikely. Even where it has potential to provide solutions, is it being evaluated according to robust scientific principles? Are questions of both risks and benefits being asked? Are innovations being compared with existing best practice in randomised controlled trials? Are AI models being compared with each other? As Helen Pearson tells us in her exceptional new book ‘Beyond Belief: How Evidence Shows What Really Works’, the battle to ensure decisions across all areas of society, ranging from health to education and beyond, are based on evidence has only recently been fought (and is beginning to be won(32)). Does the clamour for AI-based solutions present an existential threat to how we generate evidence and use it?

4.2 Indirect harms to the public

Employment

If we put aside the systemic challenges of AI highlighted above and assume that at least some of what has been prophesised will come to pass, then the potential impact on humankind and our social fabric should not be overlooked. The impact on employment was the first of many areas to be considered by the public, think-tanks, media and governments. Projections vary. Some have focussed on the scale of job losses, others- the benefits. What is clear is that the world of work will change. The International Monetary Fund, for example, have estimated that 70% of UK workers have roles involving tasks that could be impacted by AI.(33) Note ‘impacted’ is doing some heavy lifting in this sentence as it covers a myriad of effects from sweeping enhancements in performance through to replacing human workers partially or entirely. However, both extremes (and everything in between) have consequences for the public. Unemployment, for example, is associated with significant adverse effects on mental and physical health including greater risk of depression, poor self-esteem and greater need for healthcare.(34) Each of which, by the way, has their own economic consequences that could dilute or eradicate any of the purported economic gains expected from AI. Add to this the fear of unemployment and underemployment which the rise of AI engenders. This will undoubtedly fuel poorer physical and mental health which has long been associated with precarity at work.(35, 36)

Cognition

Next let’s consider the nature of the tasks that AI models may replace and the potential impacts on our cognitive abilities. There is a groundswell of concern, and evidence in support of the idea, that an increased reliance on AI tools will deplete a range of cognitive functions including critical thinking skills.(37, 38) One could argue that perhaps this should not concern us because in the end AI will be available to do our critical thinking for us. But this libertarian view ignores one simple truth which is that AI models need data derived from humans to survive and improve. If humans are no longer able to provide these data (or indeed choose to not share it for free) due to the gradual erosion of cognitive functions, then AI will surely grind to a halt. Could recursive self-improvement ever fill the void? This seems both unlikely and undesirable.

Entrenching inequality

At their heart, AI models consume vast quantities of data that allow them to make predictions i.e., judgements on the next most likely thing to happen: everything from the next word I am going to type, to the likelihood that I will commit a crime, or get a disease. On the face of it, these sound like helpful predictions. Many of us would love to have more controllable and predictable lives. But there are indirect consequences of trying to predict our futures in this way. At least two important social science theories are relevant here. First, the ‘self-fulfilling prophecy’ which proposes that the expectation of a behaviour can unintentionally make that behaviour more likely.(39) Second, ‘learned helplessness’, which posits that individuals stop trying to change their circumstances when they believe they are powerless to effect change, regardless of whether that is objectively the case.(40) If AI algorithms predict you are more likely to fail academically, less likely to secure well-paid employment, less likely to be in a long-term relationship and more likely to succumb to illness, then for some people this could precipitate learned helplessness and make these outcomes more likely. Indeed, this is more probable in people from disadvantaged backgrounds who frequently have fewer resources available to them.(41) In this way, these powerful models of prediction could serve only to embed and increase existing inequalities and possibly create new areas of disadvantage.

However, it is also true to say that predicting outcomes, particularly anything related to humans and how they behave is notoriously difficult. For some people, the very expectation of an outcome can trigger behaviours which alter the outcome entirely. Several theories seek to explain this discrepancy. However, this variability between expectations and outcomes continues to bedevil scientists. Just consider the legions of studies that point to the very poor relationship between predicted and actual grades in education.(42) Or those that show that changing perceptions of risk, on their own, rarely result in expected changes in behaviour.(43)

The relevance of all this is that it calls into question the validity of the very claims made about AI. We know that AI models must contend with biased data; that the algorithms themselves may introduce further sources of bias; and that once operationalised, we humans, the ‘beneficiaries’, of these predictions may be largely unpredictable. These challenges should engender caution rather than hype about the potential of AI. In the absence of regulation and meaningful oversight, it may be our sheer unpredictability that constrains the hubris of the AI revolution.

Risking our social selves

Much attention has been given in recent years to the potential harmful effects of an increasingly digital existence on the well-being of young people,(44) prompting legislation in several countries to restrict access to social media until 16 years of age. However, social media is clearly just one feature of a more complex problem and one which affects us all. Greater engagement with technology per se has been associated with a dizzying range of concerning effects including delays in development, feeling insecure in relationships, risks to physical and mental health, the propagation of misinformation and political polarisation and increased loneliness.(45–51) While the body of evidence cited here is not concerned with AI (although is increasingly becoming linked), it is clear that the widespread adoption of AI by the public (data indicate that over 70% of people in the UK are already using it regularly(52)) will likely propel us to engage more with technology and diminish in person interactions. Indeed, maximising engagement is an integral part of the AI business model. Much like the social media platforms that came before, AI models have been designed to keep you engaged because your engagement feeds their models. The anthropomorphic qualities of your preferred AI tool is not just a nice feature it is the key to keeping you hooked!

The potential consequences of AI making us increasingly socially isolated are profound with evidence pointing to both biological and psychological changes that have significant consequences for physical and mental health,(53) affecting our risk of depression, dementia and even survival. As noted by the US’ Surgeon General in 2023, “the mortality impact of being socially disconnected is similar to that caused by smoking up to 15 cigarettes a day”.(54) Are the benefits of AI worth these risks?

4.3 Environmental costs

No discourse on AI can be complete without considering the environmental costs. While solutions to the climate crisis are among the many problems we are told AI will solve, it remains to be the case that AI brings significant environmental costs including increased carbon emissions, water use and energy consumption to name a few.(55, 56) As these costs come into view (it is estimated that it takes a 500ml bottle of water for an AI system to compose a 100-word email(57)) and there is a growing realisation of how the energy demands of AI will impact the public, we are witnessing increasing objection among the public to the uncontrolled growth of AI. It is clear that the environmental costs alone should propel us to demand a framework for deciding when AI should be used, by whom and for what purpose.

4.4 Economic harm

Let’s end by revisiting what some would say is the grotesque value of the leading AI companies. It is important to remember that the valuations of these companies bear little resemblance to their profits (as noted by the Economist noted in 2025, ‘AI valuations are verging on the unhinged’). If this continues then, as sure as ‘burst’ follows ‘bubble’, the global economy might be expected to go into shock. Indeed, for many observers the only question is ‘when’ and not ‘if’.(58, 59)

The potential for economic harm should not be underestimated given that governments around the world have hitched their plans for economic growth to the success of AI.(60) Indeed, is it possible we are in the same place we were with the banks in 2008. Like the banks, has AI has already become too big to fail? The sheer scale of the potential economic harm should, however, serve to unite us in creating the circumstances that AI can succeed. But this will clearly require international and cross-sector agreement on regulatory frameworks able to maximise the gains and minimise harms for all.

5. Approaches to regulation.

5.1 Governing AI at the point of deployment

So far, we have considered the arguments for the regulation of foundation models (e.g., Chat GPT from OpenAI and Claude from Anthropic) and regulation of these companies is essential. But this is only part of the challenge. Most organisations and companies will not build their own models. Rather, they will incorporate models developed by others into products, services and customer experiences. The benefits and risks people experience will therefore be shaped as much by the design and deployment of these products as by the underlying models themselves. Indeed, regulation of these companies may provide a powerful mechanism by which regulation of the foundation models is achieved.

Our experience developing Blarney, a conversational AI platform, has demonstrated the importance of this distinction. We designed Blarney to be model-agnostic: that is, we did not tie the product to a single model or provider. It is able to use multiple models and this allows the underlying model to be replaced as more capable, appropriate or cost-effective alternatives emerge. In this way we were able to ensure that the user need, experience and purpose remained paramount and consistent, while the engine beneath the product was able to evolve.

This approach makes it possible to improve performance or reduce the cost of delivering the experience without redesigning the entire product. It also reinforces the principle that the foundation model is only one component of an AI system. The product’s behaviour is shaped by the information it can access, the instructions and safeguards placed around it, the role it adopts and the interface through which people encounter it. As such, any change of model must therefore be tested and governed within the context of the complete product, rather than treated as a straightforward technical substitution.

This is encouraging because many of the potential problems we have highlighted can be identified and resolved through a rigorous, human-centred development process. Our approach, shared by others, is to achieve responsible deployment by commencing with a clearly defined purpose and audience. The system should know who it is intended to help, what it is permitted to do and where its authority ends. Its responses should be grounded in verified information, with appropriate controls over the claims it can make. People should always understand that they are interacting with AI, how their information may be used and when human assistance is available.

We operationalised these principles recently during the development of a healthcare-related prototype. User testing revealed that the AI assistant could occasionally lose track of whether it was speaking to a patient or a healthcare professional. In a healthcare context this kind of error could not only affect the appropriateness and accuracy of the information provided, but also the users degree of trust and confidence.

Crucially, the development process did what it was intended to do. It identified the issue, allowed us to understand why it was happening and gave us opportunities to resolve it before wider deployment. This was not simply a question of testing the underlying model. It required us to observe how the complete product behaved with real people in a particular context. Technical evaluation and human-centred testing are therefore complementary, and both should be considered essential.

5.2: Some thoughts on what effective governance may look like

Governance should support innovation rather than impede it. The objective should be to create an environment in which organisations can experiment quickly, learn from users and test new ideas, while protecting everyone involved. This requires proportionate controls. Much like the 4 phases of clinical trials (which focus on safety, efficacy, effectiveness and monitoring), early experimentation should take place within clearly defined and contained environments, while systems approaching public deployment should face progressively stronger requirements for evidence, testing, accountability and oversight.

Responsible AI therefore requires governance throughout the product lifecycle, from initial concept and experimentation to deployment and ongoing operation. This should include representative user testing, adversarial testing, defined escalation routes, clear accountability and continued monitoring. Controls should also be reconsidered whenever the model, data, audience or intended use changes.

Effective regulation should apply across this complete ecosystem, including those who develop models and those who commission, configure and deploy products around them. Done well, regulation should not force a choice between innovation and safety. It should provide the conditions in which experimentation can happen at speed, problems can be discovered early and valuable AI products can be developed with confidence.

What next?

It is important to state that this article is not intended to make you desire a world without AI. Rather it is an attempt to highlight that ‘with great power comes great responsibility’. Our proposal is that for AI to deliver even a fraction of the purported benefits it must be human-centred, ethical, regulated, accountable, as well as secure. There needs to be transparency in what data feed it and to what end. It should be designed to eradicate rather than amplify existing bias and inequality. There should be a virtuous cycle in which users benefit from both contributing to models and their outputs. There should be perpetual post-market surveillance of both benefits and unintended consequences, and use should ultimately be determined by the potential for societal benefit. To paraphrase Jeff Goldblum in Jurassic Park ‘’ just because we can, doesn’t mean we should’. Indeed, we should ask questions regarding which ‘functions’ currently performed by people we are willing to farm out to models. Diagnosis of illness, decisions regarding treatment, psychological therapy, all forms of creative writing, flying of aircraft, military defence. These are just some areas in which models already make some contribution. But how far should it go?

If that all sounds complicated — it should. Ashby’s Law of Requisite Variety(61) from the field of Cybernetics (the study of how living organisms and machines behave) suggests that the governance of any given system needs to be as complex as the system itself. In other words, our approach to AI regulation needs to be complex. But it must also be nimble to keep up with the rapid pace of development. The current unregulated, rampant, dare we say, metastatic spread of AI is neither desirable nor sustainable. We can only avail ourselves of the benefits of AI if we first ensure ‘(it can) do no harm’.

About the authors: The views expressed in this article are the views of the authors alone and do not represent any of the organisations with which they are affiliated.

**Kavita Vedhara** is a Behavioural scientist with expertise in how behavioural factors influence health outcomes such as vaccine uptake, vaccine effectiveness and chronic disease outcomes. She is excited about the potential for responsible and transparent AI to support health care professional training.

**Rob Bennett** is the founder of conversational AI platform Blarney and co-founder and CEO of digital product studio Cyphr, where he develops and deploys human-centred AI products for brands and organisations.

**Karen Louise Dawe** is a neuroscientist whose research explores how language-based AI systems interact with human cognition across education and childhood development. She studies behaviour change, learning, trust, and cognitive support. She also has an interest in how bias in the collection and analysis of health data get translated into models.

**Philip Morgan** is Professor of Human Factors and Cognitive Science whose work focuses on sociotechnical aspects of AI, automation, cyber security, human-robot-interaction, transportation, and adaptive cognition. Director / Director of Research for academia and industry centres with themes including ethical and explainable AI, human-centred cyber security, and human-centred technology and society.

References

1. CNN Business. Anthropic suspends all access to Mythos model after US government bans foreign nationals use. Available from: https://edition.cnn.com/2026/06/13/business/anthropic-mythos-model-national-security [Accessed 09/07/2026].

2. Bhardwaj A. Artificial Intelligence and the limits of reason: a framework for responsible use in public and private sectors. Humanities and Social Sciences Communications. 2025;12(1):1565.

3. Gignac GE, Szodorai ET. Defining intelligence: Bridging the gap between human and artificial perspectives. Intelligence. 2024;104:101832.

4. Malanchini M, Allegrini AG, Nivard MG, Biroli P, Rimfeld K, Cheesman R, et al. Genetic associations between non-cognitive skills and academic achievement over development. Nature Human Behaviour. 2024;8(10):2034–46.

5. Chamorro-Premuzic T, Harlaar N, Greven CU, Plomin R. More than just IQ: A longitudinal examination of self-perceived abilities as predictors of academic performance in a large sample of UK twins. Intelligence. 2010;38(4):385–92.

6. Independent. Woman ‘weds’ AI persona she created on ChatGPT. Available from: https://www.independent.co.uk/asia/japan/japan-chatgpt-ai-wedding-b2864314.html [Accessed 09/07/2026].

7. Varella MAC. The Biology and Evolution of the Three Psychological Tendencies to Anthropomorphize Biology and Evolution. Front Psychol. 2018;9:1839.

8. Forbes India. Top 10 biggest companies in the world by market cap in 2025. Available from: https://www.forbesindia.com/article/explainers/top-10-largest-companies-world-market-cap/86341/1 [Accessed 09/07/2026].

9. The Guardian. Major publishers sue Meta for copyright infringement over AI training. Available from: https://www.theguardian.com/technology/2026/may/05/publishers-sue-meta-copyright-ai [Accessed 09/07/2026].

10. The Conversation. Two AI copyright cases, two very different outcomes — here’s why. Available from: https://theconversation.com/two-ai-copyright-cases-two-very-different-outcomes-heres-why-270229 [Accessed 09/07/2026].

11. web3 Foundation. The Hidden Price of Free: What Your Data Is Really Worth. The Personal Economics of Big Tech and AI. 2026.

12. GOV.UK. Keep Calm and Carry On — The Compromise Behind the Slogan available from: https://history.blog.gov.uk/2014/06/27/keep-calm-and-carry-on-the-compromise-behind-the-slogan/ [Accessed 18/08/2026]

13. Korbmacher M, Azevedo F, Pennington CR, Hartmann H, Pownall M, Schmidt K, et al. The replication crisis has led to positive structural, procedural, and community changes. Commun Psychol. 2023;1(1):3.

14. GOV.UK. Public attitudes to data and AI: Tracker survey (Wave 3) available from: https://www.gov.uk/government/publications/public-attitudes-to-data-and-ai-tracker-survey-wave-3/public-attitudes-to-data-and-ai-tracker-survey-wave-3#executive-summary [Accessed 18/08/2026].

15. Center for Research on Foundation Models. The Foundation Model Transparency Index. Available from: https://crfm.stanford.edu/fmti/December-2025/index.html [Accessed 09/07/2026].

16. AlBlooshi S. Artificial intelligence in higher education, opportunities, and challenges: a review. Frontiers in Education. 2026;Volume 10–2025.

17. Topaz M, Roguin N, Gupta P, Zhang Z, Peltonen L-M. Fabricated citations: an audit across 2·5 million biomedical papers. The Lancet. 2026;407(10541):1779–81.

18. Roos J, Martin R, Kaczmarczyk R. Evaluating Bard Gemini Pro and GPT-4 Vision Against Student Performance in Medical Visual Question Answering: Comparative Case Study. JMIR Form Res. 2024;8:e57592.

19. Takita H, Kabata D, Walston SL, Tatekawa H, Saito K, Tsujimoto Y, et al. A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians. NPJ Digit Med. 2025;8(1):175.

20. Henrich J, Heine SJ, Norenzayan A. The weirdest people in the world? Behav Brain Sci. 2010;33(2–3):61–83; discussion -135.

21. Fry A, Littlejohns TJ, Sudlow C, Doherty N, Adamska L, Sprosen T, et al. Comparison of Sociodemographic and Health-Related Characteristics of UK Biobank Participants With Those of the General Population. Am J Epidemiol. 2017;186(9):1026–34.

22. Wang L, Graubard BI, Katki HA, Li Y. Improving External Validity of Epidemiologic Cohort Analyses: A Kernel Weighting Approach. J R Stat Soc Ser A Stat Soc. 2020;183(3):1293–311.

23. Belenguer L. AI bias: exploring discriminatory algorithmic decision-making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry. AI Ethics. 2022;2(4):771–87.

24. Angwin Jea. Machine bias: there’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica. 2016.

25. Harvard Kennedy School. New sources of inaccuracy? A conceptual framework for studying AI hallucinations. Available from: https://misinforeview.hks.harvard.edu/article/new-sources-of-inaccuracy-a-conceptual-framework-for-studying-ai-hallucinations/ [Accessed 09/07/2026].

26. Park PS, Goldstein S, O’Gara A, Chen M, Hendrycks D. AI deception: A survey of examples, risks, and potential solutions. Patterns (N Y). 2024;5(5):100988.

27. Anthrop\c. Agentic misalignment: How LLMs could be insider threats. Available from: http://anthropic.com/research/agentic-misalignment [Accessed 09/07/2026].

28. BBC News. OpenAI says its rogue AI tried to hack other companies. Available from: https://www.bbc.co.uk/news/articles/c2el319vzr3o.

29. METR. Documented AI Agent Incidents. Available from: https://metr.org/agent-incidents/ [Accessed 18/08/2026].

30. Munro APS, Ferrari M, Kinsley R, Egan D, Vishwanath S, Bower T, et al. A phase I, needle free, dose escalation clinical trial of pEVAC-PS, a candidate pan-Sarbecovirus Vaccine. Journal of Infection. 2026;92(6).

31. Freeman D, Lambe S, Yu L-M, Freeman J, Chadwick A, Vaccari C, et al. Injection fears and COVID-19 vaccine hesitancy. Psychological Medicine. 2023;53(4):1185–95.

32. Pearson H. Beyond Belief: How Evidence Shows What Really Works: Princeton University Press;; 2026.

33. International Monetary Fund. Gen-AI: Artificial Intelligence and the Future of Work. Available from: https://www.imf.org/en/publications/staff-discussion-notes/issues/2024/01/14/gen-ai-artificial-intelligence-and-the-future-of-work-542379 [Accessed 09/07/2026].

34. Linn MW, Sandifer R, Stein S. Effects of unemployment on mental and physical health. Am J Public Health. 1985;75(5):502–6.

35. Jaydarifard S, Smith SS, Mann D, Rossa KR, Nikooharf Salehi E, Gnani Srinivasan A, et al. Precarious employment and associated health and social consequences; a systematic review. Australian and New Zealand Journal of Public Health. 2023;47(4):100074.

36. Pulford A, Thapa A, Thomson RM, Guilding A, Green MJ, Leyland A, et al. Does persistent precarious employment affect health outcomes among working age adults? A systematic review and meta-analysis. Journal of Epidemiology and Community Health. 2022;76(11):909.

37. Gerlich M. AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies [Internet]. 2025; 15(1):[6 p.].

38. Tian J, Zhang R. Learners’ AI dependence and critical thinking: The psychological mechanism of fatigue and the social buffering role of AI literacy. Acta Psychologica. 2025;260:105725.

39. ScienceDirect. Self-Fulfilling Prophecy. Available from: https://www.sciencedirect.com/topics/psychology/self-fulfilling-prophecy [Accessed 09/07/2026].

40. Seligman ME, Maier SF. Failure to escape traumatic shock. Journal of Experimental Psychology. 1967;74(1):1–9.

41. Crielaard L, Nicolaou M, Sawyer A, Quax R, Stronks K. Understanding the impact of exposure to adverse socioeconomic conditions on chronic stress from a complexity science perspective. BMC Med. 2021;19(1):242.

42. Murphy R, Wyness G. Minority report: the impact of predicted grades on university admissions of disadvantaged groups. Education Economics. 2020;28(4):333–50.

43. Sheeran P, Harris PR, Epton T. Does heightening risk appraisals change people’s intentions and behavior? A meta-analysis of experimental studies. Psychol Bull. 2014;140(2):511–43.

44. Dai Y, Ouyang N. Excessive screen time is associated with mental health problems in US children and adolescents: physical activity and sleep as parallel mediators. Humanities and Social Sciences Communications. 2026;13(1):256.

45. Muppalla SK, Vuppalapati S, Reddy Pulliahgaru A, Sreenivasulu H. Effects of Excessive Screen Time on Child Development: An Updated Review and Strategies for Management. Cureus. 2023;15(6):e40608.

46. Grant D, Winston-Lindeboom P, Ruan-Iu L, Shackleford KEE, Nosal B, Roeske M. “Mommy, do you love your phone more than me?”: Parental device use and the adolescent-caregiver attachment bond. Frontiers in Psychology. 2026;Volume 17–2026.

47. Devi KA, Singh SK. The hazards of excessive screen time: Impacts on physical health, mental health, and overall well-being. J Educ Health Promot. 2023;12:413.

48. Denniss E, Lindberg R. Social media and the spread of misinformation: infectious and a threat to public health. Health Promot Int. 2025;40(2).

49. Matthews T, Arseneault L, Bryan BT, Fisher HL, Gray R, Henchy J, et al. Social media use, online experiences, and loneliness among young adults: A cohort study. Annals of the New York Academy of Sciences. 2025;1548(1):194–205.

50. Fingerman KL, Kim YK, Ng YT, Zhang S, Huo M, Birditt KS. Television Viewing, Physical Activity, and Loneliness in Late Life. Gerontologist. 2022;62(7):1006–17.

51. Arora SD, Singh GP, Chakraborty A, Maity M. Polarization and social media: A systematic review and research agenda. Technological Forecasting and Social Change. 2022;183:121942.

52. GOV.UK. AI Skills for Life and Work: General Public Survey Findings. 2026.

53. Holt-Lunstad J. Social connection as a critical factor for mental and physical health: evidence, trends, challenges, and future implications. World Psychiatry. 2024;23(3):312–32.

54. Office of the Surgeon General. Publications and Reports of the Surgeon General. Our Epidemic of Loneliness and Isolation: The US Surgeon General’s Advisory on the Healing Effects of Social Connection and Community. Washington (DC): US Department of Health and Human Services; 2023.

55. Chouksey A, Rajan AK, Gurjar V, Tiwari R, Mishra PK. The green paradox: The climate, environmental, and sustainability implications of artificial intelligence. Global Environmental Change Advances. 2026;6:100029.

56. Nicholson-Benn H. AI and Environmental Sustainability. Available from: https://nationalcentreforai.jiscinvolve.org/wp/2026/07/14/ai-and-environmental-sustainability/.

57. The Conversation. AI has a hidden water cost − here’s how to calculate yours. Available from: https://theconversation.com/ai-has-a-hidden-water-cost-heres-how-to-calculate-yours-263252 [Accessed 09/07/2026].

58. The Guardian. Billions spent and hypothetical returns: the AI boom explained with six charts. Available from: https://www.theguardian.com/technology/2026/jun/07/billions-spent-hypothetical-returns-the-ai-boom-explained-with-six-charts [Accessed 09/07/2026].

59. Caballero RJ. Speculative Growth and the AI “Bubble”. NBER Working Paper No w34722, Available at SSRN: https://ssrncom/abstract=6134092. 2026.

60. Bokhari SAA. Artificial intelligence, governance, and economic growth: Toward a new theory of digital developmentalism. Social Sciences & Humanities Open. 2026;13:102842.

61. Ashby WR. Requisite Variety and Its Implications for the Control of Complex Systems. Cybernetica. 1958;1(2):83–99.

[1] Readers interested in more examples of AI deception should consider reading the excellent article by Patterns: https://pmc.ncbi.nlm.nih.gov/articles/PMC11117051/#sec1