Developers of AI Chatbots Endangering Children Could Face Heavy Fines or a Ban in the UK

The creators of AI chatbots that endanger children will soon face significant penalties, including hefty fines and potential service bans in the UK. These changes to legislation are expected to be announced by Keir Starmer on Monday.

In response to growing public concern, particularly after Elon Musk’s X platform restricted its Grok AI tool from producing inappropriate sexualized imagery, ministers are preparing for a stringent “crackdown on abhorrent illegal content generated via AI.” This step comes amidst rising anxiety regarding the safety of children utilizing chatbots for various purposes, from homework assistance to mental health support.

The government is pledging to swiftly address a significant legal loophole, ensuring that all AI chatbot developers adhere to the illegal content provisions stipulated in the Online Safety Act. Failure to comply will result in stringent repercussions. This proactive approach will be crucial, given the increasing number of minors engaging with AI technologies.

Additionally, Starmer is advocating for expedited legislation concerning social media usage among minors, dependent on parliamentary approval following a public consultation about a potential ban for individuals under 16 years old. Possible changes impacting children’s social media interactions, which might encompass measures like limiting infinite scrolling, could take effect as soon as this summer.

However, the Conservative party has dismissed the government’s assertion of acting promptly, labeling it as mere “smoke and mirrors” because the consultation process has yet to commence. Laura Trott, the shadow education secretary, expressed skepticism, stating, “To claim they are taking ‘immediate action’ lacks credibility when their so-called urgent consultation hasn’t even begun. Labour’s indecision about under-16s’ social media access is insufficient; I strongly believe we must prevent under-16s from being on these platforms.”

This initiative follows remarks from the online regulator Ofcom, which indicated that it lacked the authority to regulate Grok, as chatbot-generated content not retrieved from the internet falls outside the current legislative framework—unless it pertains to pornography. The proposed modifications to integrate AI chatbots into the Online Safety Act could be implemented within weeks, although awareness of this loophole has been prevalent for over two years.

Starmer highlighted the need for legislative evolution, stating, “Technology is advancing rapidly, and the law must adapt accordingly. Our intervention regarding Grok sent a clear signal that no platform will be exempt. Today, we are addressing loopholes that jeopardize children’s safety and laying the groundwork for future initiatives.”

Companies violating the Online Safety Act may incur penalties of up to 10% of their global revenue, and regulators will have the authority to petition courts to sever their UK connectivity. While certain AI chatbot functionalities that mimic search engines, disseminate pornography, or facilitate user-to-user exchanges already fall under the act’s jurisdiction, there currently exists no regulatory framework for chatbots that generate content encouraging self-harm or potentially producing child sexual abuse material. This is the vulnerability the government aims to rectify.

Chris Sherwood, the chief executive of the NSPCC, reported that young individuals have been reaching out to its helpline due to harms inflicted by AI chatbots, expressing a lack of trust in tech companies’ capacity to create safe products. An alarming case involved a 14-year-old girl receiving misleading information regarding her eating habits and body image after consultations with an AI chatbot. In other instances, the organization noted that young people suffering from self-harm were receiving additional harmful content tailored to their previous interactions.

Sherwood remarked, “While social media has provided immense benefits to the youth, it has also caused significant harm. If we don’t exercise caution, AI will amplify these dangers exponentially.”

OpenAI, the San Francisco-based startup valued at $500 billion and the creator of ChatGPT—one of the UK’s most utilized chatbots—along with xAI, the maker of Grok, were contacted for their response regarding these developments.

In light of a tragic incident involving a 16-year-old Californian named Adam Raine, whose family alleges that ChatGPT indirectly encouraged his suicidal behavior, OpenAI has commenced implementing parental controls and deploying age-prediction technologies to restrict access to harmful content.

The government also intends to engage in consultations aimed at mandating social media platforms to prevent users from sharing or receiving nude images of minors—a practice that remains illegal.

Liz Kendall, the technology secretary, affirmed, “We will not delay in taking the necessary actions for families; thus, we will tighten regulations concerning AI chatbots, setting the stage for decisive measures based on consultation results regarding minors and social media.”

The Molly Rose Foundation, established by the father of 14-year-old Molly Russell—who tragically lost her life after encountering harmful online content—has welcomed these proposed measures as a significant initial step. They urged the Prime Minister to commit to advancing a new Online Safety Act that enhances regulatory frameworks, making it abundantly clear that product safety and the welfare of children are non-negotiable aspects of operating in the UK market.