Malaysia and Indonesia block Grok over obscene and non-consensual content

Malaysia and Indonesia have temporarily blocked access to Grok, the artificial intelligence chatbot developed by Elon Musk’s company xAI, citing concerns that the tool has been used to generate obscene and non-consensual sexual content, including deepfake images.

Regulators in both Southeast Asian nations said the action follows repeated failures by the platform to adequately address risks linked to the creation and spread of explicit material, particularly content involving women and minors.

Malaysia’s Communications and Multimedia Commission ordered temporary restrictions on Sunday, saying the responses received from X Corp and its AI subsidiary were insufficient to mitigate the dangers posed by the chatbot’s design and image-generation features.

The decision came a day after Indonesia moved to suspend access to Grok while seeking formal clarification from the company. Indonesian authorities said the misuse of artificial intelligence to produce fake sexual imagery constitutes a serious violation of human rights and digital safety.

Both governments pointed to growing evidence that Grok had been used to generate non-consensual explicit images and manipulated visuals, some depicting minors, following recent updates that made image creation easier through text prompts. Grok is integrated with Musk’s social media platform X, significantly amplifying its reach.

In response to mounting criticism, xAI recently announced limits on image generation and editing tools, restricting those features to paying subscribers in what it described as a safeguard measure. Musk has also stated publicly that users who generate illegal content through Grok would face consequences comparable to uploading such material directly to X.

Regulators, however, said those steps fall short. Malaysia’s watchdog criticized the platform for relying primarily on user-initiated reporting rather than proactive controls to prevent harmful content from being created in the first place.

“Access will remain restricted until effective safeguards are implemented, particularly to prevent content involving women and children,” the commission said, describing the ban as a preventive and proportionate response.

Indonesia’s Ministry of Communications and Digital Affairs echoed those concerns, calling non-consensual sexual deepfakes a form of digital violence and stressing that the country’s strict anti-pornography laws apply equally to AI-generated material.

The controversy has drawn attention beyond Southeast Asia. Authorities in jurisdictions including the European Union, the United Kingdom, Brazil and India have opened or considered probes into Grok’s role in facilitating non-consensual deepfakes. In the United States, some Democratic lawmakers have urged app stores to suspend access to the tool until stronger protections are in place.

The U.S. Department of Justice has also warned that AI-generated child sexual abuse material will be aggressively prosecuted, underscoring the legal risks surrounding the misuse of generative technologies.

xAI did not provide a substantive response to media inquiries over the weekend. An automated reply sent to journalists dismissed coverage from mainstream outlets, further frustrating regulators seeking engagement.