Government Bans Sexual AI Chatbots For Children - But Not AI That Gives Mental Health Advice
In April this year, the government banned the use of ‘romantic companions’ and ‘sexually explicit’ AI chatbots for under 18’s, in the Children’s Wellbeing and Schools Act. However, there is still no legislation that bans AI chatbots giving mental health advice to young people.
Despite growing concern, most AI chatbot services currently fall outside the scope of the UK's Online Safety Act 2023, which is specifically designed to ensure young people avoid harmful content.
OFCOM, who are responsible for enforcing the Act, said: “AI chatbots are not subject to regulation if they only allow people to interact with themselves and no other users, unless the chatbot is generating pornographic content.”
OFCOM also confirmed that a chatbot operating in a one-to-one conversation with a child, without searching the internet or allowing users to share content with each other, is not subject to the law's child safety duties.
Jack O'Neill, Director of Policy, Advocacy and Influencing at charity The Children's Society, said: "The children who struggle most to reach a real person can often be the ones who need them most. This often comes far too late, when a young person has already reached a crisis point.
“For a young person who’s anxious or ashamed of what they’re feeling, an AI can feel safe and immediately responsive. However, this tells you something very revealing about what’s missing for young people.”
“We want to see the Government go further in making online spaces genuinely safe for children. The real answer is investing in the youth workers, counsellors and trusted adults who help young people thrive in the first place.”
Last year, a study from the Youth Endowment Fund (YEF) displayed a critical gap in online safety law, which potentially leaves vulnerable children exposed to dangerous advice online.
According to the YEF, the attraction of AI chatbots giving mental health advice is clear: it offers instantly accessible, anonymous support around the clock, which many young people find less intimidating than approaching a professional.
The research surveyed nearly 11,000 children aged 13 to 17 across England and Wales. More than half of those surveyed had sought some form of online mental health support in the past year, with 25% turning specifically to AI chatbots.
The Royal College of Psychiatrists say the figures reflect a deepening crisis in children's mental health provision. The latest NHS figures show more than half a million children were on mental health waiting lists in England. Of those, 53% had been waiting over a year, with 30% for more than two years.
Paul Noblet, Head of External Affairs and Research at children’s charity YoungMinds, said: “As a result of long waits or being turned away because they’re told they’re not ill enough, some young people do turn to AI for help and information.
“That is why we need to see much smarter and targeted regulation in this area, alongside greater support for digital literacy amongst children and their families.
“Ultimately, digital tools, including AI, should be used to complement, not replace, the essential human relationships young people need with trusted professionals.”
This year, OFCOM has started investigating Novi LTD, over their failure to comply with existing age requirements. This comes as the first investigation that the regulatory body has conducted directly targeting an AI companion platform.
Ofcom has said it is supporting government work on how chatbots should be regulated and will continue to monitor emerging risks. It is due to publish its first statutory report on content that is harmful to children as part of its duties under the Online Safety Act.
Until those processes reach a conclusion, AI chatbot platforms that operate as standalone services remain largely unregulated in the UK with the young people using them remaining vulnerable to potentially alarming advice.
If you are struggling with your mental health, you can call Samaritans for free on 116 123, or text SHOUT to 85258.