Building online safety for healthy societies

Moreover, many young people do not tell anyone about these experiences, making it hard for them to get help. Image take-down laws such as those now being considered in the UK are essential but alone are insufficient. Australia has had an image-based abuse take down scheme since 2018, which has provided rapid harms remediation for tens of thousands of Australians, but any take down scheme is employed after the damage has been done. We all know that those reporting represent just the tip of the iceberg, with the Australian study finding that one third of young people did not tell anyone about the non-consensual sharing of their sexual image, or about being solicited online by adults.

Award-winning Australian research has also shown worrying increases in sexual violence inflicted between adolescents, which has coincided with the proliferation of the online environment, and access to violent pornography. Harmful narratives and stereotypes embedded in these forms of digital media have contributed to the formation of problematic sexual scripts and the drastic resurgence in restrictive gender norms among young people.

These problems have crystallised and metastasised while political leaders and the public were still in the dark about the operational risks of these products. Scores of new products continue to come to market, again without the safety guardrails required elsewhere. Nudification apps are an appalling example, encouraging violence and misogyny. The widespread usage of these applications, accompanied by a lack of accountability reflects a broader cultural phenomenon: the normalisation of sexual violence in young people. We now have enough knowledge to recognise that a serious response is imperative.

Central to this is a suite of mandatory and enforceable codes and standards which place the responsibility onto platforms to detect and remove unlawful material such as terrorist and child sexual abuse content and also to ensure children are not exposed to age-restricted content such as pornography, high-impact violence and material which promotes suicide and self-harm. Where there is non-compliance eSafety can use the full range of its powers, including seeking penalties of up to $49.5 million.

These powers have already been used to prevent several of the world’s most popular nudifying, or “declothing apps” being served to Australians.  We have used our “gatekeeping powers” to have app stores like Apple deplatform 100s of “chat roulette apps” which pair child predators and children in video chat and as of December 2025, search engines must blur pornographic and explicit violent imagery to prevent innocent eyes from accidentally or incidentally encountering content they are not cognitively ready yet to see.

The world has been watching some of eSafety’s work, including its implementation of Australia’s social media minimum age legislation, which is intended to delay young people from having a social media account until age 16. Similar measures are increasingly being adopted by many other European nations and elsewhere, and are now being anticipated in the UK. Now that we are tackling the harms of AI companions, chat bots and popular online gaming platforms with kids, we look forward to working with like-minded nations to tackle these other forms of grievous harms impacting young people worldwide.

To enable tech platforms to make their platforms “safer by design,” eSafety has also just announced its new ‘Designing for Safety: Preventing child sexual exploitation and abuse online toolkit’ to support industry to meet their safety and regulatory obligations by embedding child safety into product design, risk assessments and systems architecture from the outset. eSafety Commissioner Julie Inman Grant has championed Safety By Design since 2018, with the ultimate goal of technology companies adopting the approach to protecting Australian children.

Regulation provides guard rails, but with technology developing at exponential speed, it alone cannot reshape social norms and young people’s attitudes and behaviours.

Here again, Australia has a world-leading approach by mandating consent education in the Australian National Curriculum in an age appropriate way from Kindergarten until year 10. Whilst this is a significant step in addressing gendered based violence and promoting respectful relationships, advancements in technology have proven that this curriculum requires updates to provide young people with the capabilities and understanding to navigate this increasingly harmful digital world.

This is particularly critical in an era where algorithmic systems amplify sensationalist, misogynistic content that often rewards extremity and reinforced restrictive gender norms. Without critical digital literacy for social media and pornography, young people are exposed to problematic sexual scripts and distorted representations of intimacy that can shape behaviours.

Only by addressing both structural regulation and cultural norms through informal and formal education can we meaningfully shift the conditions that enable digital and in person violence amongst youth.