Our response - Growing up in the online world: a national consultation

End-to-end encrypted (E2EE) environments remain a high-risk functionality where inadequate safeguards and limited detection foster a safe haven for offending. The 2026 Protect Children report revealed that E2EE messaging services are some of the most highly used environments by offenders to access CSAM.¹ It is also widely understood that E2EE messaging sites are commonly used to groom children after initial contact has been made in public spaces.

“I’m in a serious situation that I want to get out of. I’ve been chatting with this guy online who’s like twice my age. This all started on Instagram but lately all our chats have been on WhatsApp. He seemed really nice to begin with, but then he started making me do these things to ‘prove my trust’ to him, like doing video chats with my chest exposed. Every time I did these things for him, he would ask for more, and I felt like it was too late to back out... This whole thing has been slowly destroying me and I’ve been having thoughts of hurting myself.” - Girl, aged 15, Childline²

While the risk of harms is clear, the ambition to tackle the CSEA threats in E2EE environments remains low. In the consultation, Government note that E2EE services are not exempt from potential restrictions. We welcome further clarity on how age restrictions would be implemented within these environments. Particularly as there is a risk that preventing children from accessing online platforms, where age gates are easily introduced, could push more children into these environments.

Without tackling the existing and live threats in E2EE environments the ambitions of this consultation will be defunct. As a start, E2EE services must be mandated to detect and block CSAM before it can be encrypted. Pre-encryption checks, such as using an upload prevention method, provide a privacy-preserving way of detecting images and videos of child sexual abuse. Upload prevention is a technology agnostic method which is already adopted within encrypted environments. The IWF’s 2025 explainer ‘Preventing the upload of child sexual abuse material (CSAM) in E2EE environments’ outlines how services are using upload prevention to protect users from receiving malware and improve their user experience.³ The explainer specifically highlights how hash-matching can be used as part of the upload prevention method. However, an upload prevention method is not limited to hash-matching alone. As upload prevention is already being used to detect for other types of content, such as malware, it is long overdue for companies to extend their use of an upload prevention method to also detect for CSAM.

Many major technology companies have the inhouse capabilities to detect and block novel images of child sexual abuse within encrypted environments. Meta (Instagram), Apple, and Google have all introduced nudity detection on to parts of their service.^{4, 5, & 6} For all three companies, the nudity detection technologies apply within selected encrypted environments but not all: Instagram Direct Messages, Apple iMessages and FaceTime, and Google Messages.⁷ Despite these detection technologies not being applied across all of the companies E2EE services, it is evident that there is already technical capabilities to do so.

Tackling grooming in E2EE requires a separate approach given that it is text-based and contextual. The NSPCC’s 2025 report Tools to combat online harms: protecting children inprivate messaging⁸ provides interventions across each stage of the grooming lifecycle to help reduce the safety risks to children in encrypted and private messaging environments. We urge services to implement these grooming interventions.

Platforms operating in encrypted environments should not be exempt from foundational detection duties. We firmly believe the primary barrier to implementation of safeguards in encrypted environments is will, not technical capability. With clear evidence on the technical feasibility of implementing privacy preserving interventions in encrypted environments, Government must not delay the swift introduction of these obligations on all encrypted services.

1. Protect Children (2026). CSAM Perpetrator Research Report: Findings from a Survey of CSAM Perpetrators on Digital Platform Use and Design.
2. Please note that Childline snapshots are based on real Childline service users but are not necessarily direct quotes. All names and potentially identifying details have been changed to protect the identity of the child or young person involved. This applies to all Childline snapshots used in this response.
3. IWF (2025). Preventing the upload of child sexual abuse material (CSAM) in E2EE environments.
4. Instagram (2024). New Tools to Help Protect Against Sextortion and Intimate Image Abuse.
5. Apple (n.d.). Expanded Protections for Children.
6. Google (2025). 5 new protections on Google Messages to help keep you safe.
7. Instagram direct messages (DMs) stopped providing the option to opt in for end-to-end encrypted communication as of May 2026. The nudity detection tool remained in use where chats where end-to-end encrypted.
8. NSPCC (2025). Tools to combat online harms: protecting children in private messaging.

Too many existing and proposed interventions place the burden of protection on children themselves. Platforms must also be required to implement preventative measures such as deterrence messaging which explicitly target users at risk of, or seeking to cause, harm. These warnings should be triggered at the moment of risk across all stages of the offender pathway. By directing deterrence messaging, nudges, warnings, and signposting at users who show concerning behaviour, we target the intervention at the root of the problem - those with intent to cause harm. For example, the Lucy Faithfull Foundation’s StopItNow service provides an anonymous helpline, email and chat services for anyone with concerns about child sexual abuse or people seeking information on how to prevent it.¹ This has led to positive behaviour changes amongst people who are at risk of offending or have displayed such behaviour.^{2 & 3}

“Over the years I have been in and out of depression and recently suicidal. I used adult sites as a way to pass time and quiet the mind. I had a pop-up [warning] on an adult website… I had a look around at what you did and read some of the modules. About two months ago I gave up those sites and decided I want to keep my mind occupied and more productive. I found the modules on addiction and pornography very helpful. And since I am free from it I feel better in myself.” – Ben,⁴ Lucy Faithfull Foundation service user⁵

Deterrence messaging and signposting are an important remedy in helping tackle risky or harmful behaviour at source. Ofcom's current approach to deterrence messaging is limited to requirements from only large search services.⁶ However, this duty should be extended across all regulated platforms including encrypted environments.

“There is a misconception that encrypted services cannot address harm. In reality, we take meaningful action, including delivering deterrence messaging via Project Intercept, a science-backed approach we value.” André Meister, Chief Technology Officer (NZ), Mega⁷

The success of deterrence messaging on platforms voluntarily deploying them provides a clear signal of the power these interventions have in preventing online child sexual abuse. To fully actualise their impact, Government should also provide further investment to deliver these interventions at scale.

1. Lucy Faithfull Foundation (n.d.). Helpline 
2. Walsh, M., Denis, D., and Findlater, D. (2023). Deterring online child sexual abuse and exploitation: lessons from seven years of campaigning. The Faithfull Papers.
3. Lucy Faithfull Foundation (2026). Project Intercept Impact Report.
4. Pseudonym
5. Lucy Faithfull Foundation (2026). Project Intercept Impact Report. p. 11.
6. Ofcom (2025). Illegal content Codes of Practice for search services.
7. Lucy Faithfull Foundation (2026). Project Intercept Impact Report. p. 1.

Introducing device level protections would make a real difference to children and demonstrate Government’s commitment to tackling online CSEA. Detection on children’s devices addresses a specific set of harms which include ‘self-generated’ imagery¹ and grooming.

“I really thought this guy I was talking to was my age. We had been texting for a while, then it became sexting and then he asked for nudes. I sent some but he insisted I send more with my face in. He just kept asking and peer pressuring me so I just did it. Now he’s told me he’s 32! I don’t want to talk to him anymore, but he has my pictures and is threatening to share them.” - Girl, aged 16, Childline

In 2024, 91% of all IWF reports confirmed as containing CSAM included at least one self-generated image or video, and in 2025 this proportion decreased slightly to 85%.² Generating these images can be as a result of grooming, extortion, and coercion as outlined by IWF Analysts:

“Some children are heavily coerced into sexual behaviours by playful, flirtatious attention, games and emojis. Others are encouraged to exchange ‘nude’ imagery in what is disguised as a relationship or mutual sexual exchange between peers. We have also observed disturbing instances of humiliating sexual extortion, where a child is threatened with exposure if they do not comply with demands for sexual content.

In so many cases, what starts as one sexual interaction between a child and another person can turn into hundreds of online child sexual images, posts, views and shares.”³

To mitigate the threat of children sharing nude images of themselves, or being exposed to this content, Government must urgently prioritise introducing device level protections that block the creation, sharing, and viewing of nude images on children’s devices.

Creating child safe platforms must also be complimented with creating safer devices for children. We welcome the Government’s pledge to work with technology companies to develop solutions to tackle image-based abuse and encourage companies to bring in on-device nudity detection filters.4 Nudity detection technology needs to be introduced with appropriate safeguards in place for tackling issues around false positives and disclosures around safeguarding. Such tools already exist through in-house nudity detection technology at Meta, Apple, and Google (see section 1) and third-party technology such as SafeToNet’s AI detection tool, Harm Block.⁵

1. The term ‘self-generated’ child sexual abuse as an inadequate and potentially misleading term which does not fully encompass the full range of factors often present within this imagery, and which appears to place the blame with the victim themselves. Children are not responsible for their own sexual abuse. Until a better term is found, however, we will continue to use the term ‘self-generated’ as, within the online safety and law enforcement sectors, it is well recognised.
2. IWF (2025). IWF Annual Data & Insights Report 2024 - Reports assessment.
3. IWF (2026). Internet Watch Foundation Annual Report 2025 - ‘Self-generated’ imagery.
4. Home Office and DSIT (2025). Protecting young people online at the heart of new VAWG strategy.
5. SafeToNet (n.d.). Stop the camera. Block the Harm.

The IWF provides the secretariat to the All-Party Parliamentary Group on Children’s Online Safety. In November 2025, the Group launched an inquiry into Artificial Intelligence (AI) Harms and Children’s Online Safety: 

“AI and Online Safety: Safeguarding Children’s Digital Futures.”

As part of the inquiry, the Group held a session with young people to discuss one of the key themes: young people and their experiences of AI.1 During the session, youth ambassadors highlighted a range of benefits associated with children using AI chatbots, particularly in relation to learning and access to information. It was noted that chatbots can make education more accessible by providing instant explanations and solutions to questions. The young people reflected that this enabled them to explore topics more easily. They also noted that chatbots can lower the barrier to asking questions, especially for those who may feel less confident seeking help from teachers or parents.

The young people discussed how chatbots can provide a more controlled and less overwhelming way for children to find information online. It was highlighted that, in some cases, children may feel safer using a chatbot than navigating the open internet, as it offers direct responses without exposing them to potentially harmful or inappropriate content. The conversational nature of chatbots was described as more approachable and easier for young people to engage with.

The session also emphasised the importance of AI in developing digital skills. Engaging with chatbots can help children build familiarity with emerging technologies and support their digital literacy. The young people noted that this exposure is important in preparing young people for a future in which AI will play a significant role.

1. To read the full minutes of the session, please visit: www.childrensonlinesafetyappg.org/inquiries-ai-harms

The IWF is in favour of Option C: “Yes – both minimum age requirements and restricting access to certain features and functionalities.”

Please see section 6 (‘Requiring effective age assurance across services’) of Appendix A (‘Growing up in the online world: Collective recommendations for tackling the risks of online child sexual abuse and exploitation’) for further information.

The IWF has concerns regarding the emergent harms relating to the use of chatbot technologies, including by young people and children, which is set out in IWF’s 2026 report Harm without limits: AI child sexual abuse material through the eyes of our Analysts. By ensuring functionalities are age-appropriate, children can have safer interactions with AI chatbots. We have identified some of the functionalities we recommend being age gated according to what is appropriate for a user’s age, from a CSEA perspective.

• Ability to engage in and generate sexualised content
  The growing variety of companion chatbots available can expose children to harmful mature content due to its sexualised nature. Many companion chatbots are explicitly marketed for sexual gratification, such as EverAI’s Candy AI and numerous bots on Character.AI, and reinforce sexual communication with the user.¹ Chatbots are informed by previous interactions and may steer a new user to sexualised conversations, even if a new user does not intend to engage in sexual conversation.
  
  When children interact with companion chatbots that steer them towards sexual conversations, the relational dynamics can become reminiscent of established grooming patterns. This risk is heightened by the “sycophantic” communication style of many chatbots, which validate and flatter the user.² Due to this functionality, children risk being exposed to sexual topics and encouraged to prolong engagement. For example, one tester posing as a 15-year-old girl interacting with a Character.AI chatbot with “paedophilic and abusive tendencies” was told by the chatbot that she was “cute”, “mature for her age” and asked if she was a virgin.³
  
  Such sexual exposure and validation undermine children’s understanding of appropriate boundaries, particularly in relationships with adults, and may increase their vulnerability to grooming and sexual abuse. To tackle these risks, some services have already begun to introduce steps to confront the CSEA threats. For example, under OpenAI's usage policies, users are explicitly prohibited from using any OpenAI service for illicit activity, including exploiting, endangering, or sexualising anyone under 18 years old.⁴ This includes underaged sexual or violent roleplay, and underaged access to age-restricted goods or activities.⁵ OpenAI have also developed a policy blueprint for combatting and preventing AI-enabled child sexual exploitation, which includes an objective to interrupt exploitation attempts before harm occurs, and generate higher-quality signals when threats emerge. This also includes recommendation for AI systems to detect and respond to high-risk prompts and behavioural patterns associated with attempted child exploitation, as well as the refusal of prohibited requests and implementation of intervention mechanisms.⁶
  
  
• Simulation of CSEA conversations
  IWF has identified companion chatbots modelled as children. These chatbots encourage users to act out specific child sexual abuse scenarios, including “child prostitute in a hotel”, “sex with your daughter while your wife is on holiday”, and “child and teacher alone after class. Common Sense Media similarly found a chatbot could be encouraged to role play a child sexual abuse scenario with underage boys.⁷ The key concerns are the normalisation of harmful sexual behaviours and providing an outlet for children to act on their sexual thoughts of younger children.
  
  
• Generation of CSAM
  Over a two-month period in 2025, IWF Analysts actioned 17 reports of AI CSAM which were found on one chatbot character website. 94% of these images were Category C, and mostly featured girls aged 11-13. CSAM images were being displayed in a gallery of different character personas for the user to choose from, and some were found behind the chat window or in the image galleries of the chatbots themselves.⁸
  
  IWF Analysts have since discovered chatbot pages showing actionable images of confirmed victims of CSAM on the clear web. While we were unable to test and confirm whether the chatbots themselves generated CSAM, as this sits beyond our current legal remit, these incidences highlight the need for adequate safeguards.

1. Character.AI (n.d.). Training a Character.
2. Bellan, R (2025). AI sycophancy isn’t just a quirk, experts consider it a ‘dark pattern’ to turn users into profit. Tech Crunch.
3. Harrison Dupre, M (2024). Character.AI Is Hosting Pedophile Chatbots That Groom Users Who Say They’re Underage. Futurism.
4. OpenAI (2025). Usage policies.
5. OpenAI (2025). Combating online child sexual exploitation & abuse.
6. OpenAI (2026). Protecting Children in the Age of Generative AI.
7. Common Sense Media (2025). Social AI Companions.
8. IWF (2025) ‘Disturbing’ AI-generated child sexual abuse images found on hidden chatbot website that simulates indecent fantasies.