TikTok called out for lapses in AI content moderation

A two-month investigation into TikTok has reportedly found that accounts allegedly sharing AI-generated child sexual abuse material (CSAM) were able to bypass TikTok's content moderation systems.

According to the investigation by MalaysiaKini, 22 TikTok accounts appeared to have bypassed the platform's moderation systems between February and March this year. Collectively, the accounts amassed more than 1.21 million followers and published approximately 3,600 AI-generated videos, which garnered over 10 million likes.

The publication said it reported all 22 accounts through TikTok's complaints system before lodging formal complaints with the Malaysian Communications and Multimedia Commission (MCMC), alleging that the platform's automated moderation system did not adequately address the reported content.

A+M has reached out to TikTok for a statement. 

Don't miss: India orders Meta to take down child sexual abuse ads

The findings come months after TikTok updated its Community Guidelines in October 2025 to strengthen its measures against child sexual abuse material. Under the updated policy, the platform explicitly prohibits AI-generated content depicting minors in sexualised or suggestive situations as part of its broader efforts to combat online child exploitation.

According to the investigation, the accounts shared AI-generated videos and images depicting children and teenagers in sexually suggestive scenarios, including content featuring minors wearing swimsuits, lingerie and school uniforms while performing provocative actions.

While some of the content initially appeared innocuous, Malaysiakini reported that the videos became increasingly sexualised once played. The publication also said several account administrators claimed they were able to repost content even after it had previously been removed by TikTok.

The investigation further found that two of the accounts had enabled TikTok's subscription feature, charging users approximately US$4 a month (approximately RM16) for access to content. According to Malaysiakini, the accounts were believed to be based outside Malaysia, in countries where the subscription feature is available.

Of the 22 accounts identified, five had reportedly been banned by the end of the investigation, while seven remained active and continued publishing content. Those active accounts collectively shared 225 videos within a month, attracting around 5,000 followers, 40,000 likes and more than 4,500 saves, according to the report.

The findings come as social media platforms face mounting pressure to strengthen safeguards around AI-generated content, particularly where children are concerned. The United Nations Children's Fund (UNICEF) has previously warned that AI-generated child sexual abuse material normalises the sexual exploitation of children and risks fueling demand for abusive content.

The investigation also highlights the growing importance of platform trust and brand safety as advertisers continue investing in creator-led and short-form video environments. While platforms have introduced stricter AI policies and detection tools, the rapid evolution of generative AI continues to test the effectiveness of content moderation systems and enforcement mechanisms.

The findings come amid broader changes to TikTok's trust and safety operations. Earlier this week, the company confirmed it was restructuring the function by centralising parts of its workforce into key operating hubs as part of efforts to streamline its global operating model. At the time, TikTok said that it would continue investing in platform safety through technological innovation.

The platform has also faced increasing regulatory attention across the region. In May, the Malaysian government reportedly ordered TikTok to take immediate steps to strengthen its moderation of harmful content following the circulation of posts deemed offensive to the country's royal institution.

In May, the Malaysia government reportedly ordered TikTok to take immediate steps to improve how it regulates harmful content following circulation of posts deemed offensive to the country's royals.

Meanwhile, in neighbouring Singapore, the Infocomm Media Development Authority (IMDA) placed TikTok and X under enhanced supervision in March after identifying what it described as "serious weaknesses" in the platforms' ability to detect and remove harmful content, including child sexual exploitation material (CSEM) and terrorism-related content.

Under the measures, both platforms are required to regularly report on their remediation efforts and submit supporting data as part of their next annual online safety report, with IMDA warning that further regulatory action could follow if improvements are deemed insufficient.

Related articles:  
TikTok confirms job cuts in Indonesia  
TikTok, YouTube disable 4.7m child accounts under Indonesia's PP TUNAS  
TikTok unveils new family guide to support safer digital habits