Enterprises face growing risks not from artificial intelligence itself but from the human tendency to over-trust confident AI systems without sufficient understanding or scrutiny, says JJ Lopez Murphy, Head of AI, Globant.
The public has growing concerns that conversations with AI are fuelling vulnerable users towards delusional thinking.
Personal accounts have amplified this fear shared online and in the media, to the point where the phenomenon has been popularised as ‘AI psychosis’. In these cases, individuals report developing or experiencing worsening symptoms of psychosis, such as paranoia and delusions, in connection with extended chatbot use.
While the term feels new, the underlying behaviour is not. A study on mental health shows that people experiencing psychosis often incorporate emerging technologies and media into their delusional narratives and thoughts. The difference today is the nature of technology itself.
Today, LLMs don’t just present information; they interact with us by conversing, empathising and adapting to what the user wants. Models are intentionally optimised to be helpful, agreeable and non-judgmental. These models are becoming more advanced, more fluent, more persuasive and more confident in tone, so many users, especially those who are already vulnerable, stop questioning the outputs altogether.
This is where using LLMs can unintentionally encourage ‘AI psychosis’; regularly interacting with a system that is consistent in mirroring the user’s language and providing validation of assumptions can reinforce existing distortions instead of challenging them. What appears to be a friendly conversational companion may instead strengthen false beliefs, creating conditions in which delusional thoughts can deepen.
Looking at wider usage of LLMs, there seems to be a quiet shift from critical thinking to blind acceptance and a lack of scepticism. The issue here is not that AI is becoming too human; the problem is that humans are beginning to treat the outputs as infallible.
We may be experiencing an AI hangover due to too much, too hard and too fast. The discourse on AI is widespread, with some in favour, some against and a few sceptics who may believe AI is a sentient being with the ability to negatively influence users. This is understandable; it’s a whiplash. AI has quickly moved from being a niche topic to a global industry worth billions, becoming intertwined with our daily lives.
The majority of people, whether they know it or not, interact with AI daily. While the technology feels ubiquitous, typical users aren’t given enough time to understand it and perhaps once they do, suddenly, capabilities have evolved or there’s a new model in the market.
Because of the fast development, it’s easy to comprehend AI as something magical but the expectations of AI exceed the typical user’s understanding of how these systems are supposed to work. It is a prediction engine that operates on patterns, not comprehension.
Users need AI fluency, a basic understanding to question the outputs, understand the limitations of AI models and to use AI as a tool that can support decisions instead of replacing them. It is not so fundamentally different from digital literacy regarding trusting whatever is forwarded on a social network.
It is harder to unravel because it caters so closely to our cognitive biases, such as confirmation bias, emotional validation and dunning-kruger. When our cognitive and emotional wires are so effectively tapped, we need to reinforce as fundamentally as literacy some ideas about agency, choice, verification and skepticism.
Perception vs reality of AI capabilities reflects a core misunderstanding in that confidence in tone doesn’t mean confidence in truth. When users don’t challenge outputs, especially in sectors such as healthcare or finance, mistakes can go unnoticed until they cause real harm.
Over-trust is a behavioural bias, not a software bug. Over-trusting modes can be just as harmful as bias or error in the models themselves. Automation bias has been known for decades and that is without the personality of current AI.
Humans are wired to anthropomorphise and AI companies are building systems that exploit these biases. When technology speaks with authority, simulates emotion and can operate as a friend, we unconsciously assign it credibility and project a human-like intelligence. The more ‘human’ AI appears, the less humans feel compelled to analyse the results critically.
To counteract this, organisations need to encourage training for using AI models to help teams understand what AI is, what it isn’t and how it should fit into decision-making frameworks. When employees understand how models generate results, including their boundaries, biases and confidence levels, they will ensure they’re better equipped to question them responsibly, especially in sectors where errors can have big consequences.
We need to emphasise responsible AI, building systems where transparency, governance and usability are treated as one discipline.
While organisations assess how to address the gap of how AI works and how it is used, the responsibility shifts to those building these systems. If users are becoming too trusting of outputs, then companies must create environments where questioning the result becomes the intuitive behaviour.
How? With transparent product design. User interfaces need to provide context to users such as confidence levels in the output provided, what data was used to result in the output and clear boundaries, such as “this tool is not intended to give medical advice” directly to the user. When users can see how the AI model reached its conclusion, they’ll be better prepared to validate, challenge or override it, turning trust from passive into something earned.
Equally important is ensuring that technical safeguards aren’t hidden deep in the backend. Monitoring tools, audit logs and adaptive guardrails only result in responsible use if they work easily with the user experience.
Responsible behaviour does not depend on users remembering every best practice, but the system itself should encourage critical thinking and informed oversight.
The new literacy about choice and discomfort is starting to be noticed, we need a reversal from decades upon decades of institutions, jobs, educational paths and everything else trying to commoditize the human to make them fungible.
It is going to be uncomfortable, there are going to be growing pains, but the prize is a higher, deeper degree of freedom.
This rise in ‘AI psychosis’ reflects an unpreparedness and growing discomfort with how quickly intelligent systems are reshaping our world. But the right response isn’t to retreat, it’s to engage more thoughtfully.
We shouldn’t let AI lead us; it needs to keep being questioned. Machines are at their best when they serve as extensions of human insight, not replacements for it.
In the end, responsible AI is less about controlling machines and more about guiding human behaviour. Curiosity and the willingness to ask who, what, when, why and how is the most reliable safeguard we have.
Having transparent designs, aligning governance with user experience and cultivating a culture of critical thinking, enterprises can ensure that AI remains a trusted partner and not a quiet authority we follow blindly.
