Grok faces bans in Indonesia and Malaysia after deepfake backlash

• Indonesia and Malaysia blocked Grok over deepfake, non-consensual sexualised images.
• Officials cite human rights violations and user harm.

Access to Grok, the artificial intelligence chatbot run by Elon Musk, has been blocked in Indonesia and Malaysia after the tool was used to generate sexualised images of real people. The move comes after days of growing criticism over how easily the chatbot could be prompted to create explicit images without consent.

Indonesia confirmed on Saturday that it had temporarily restricted access to the chatbot developed by xAI, Musk’s AI company. Malaysia followed a day later with its own block. While officials in other countries have raised concerns or called for restrictions, Indonesia and Malaysia are the first to take formal action against the application.

Governments cite harm and rights concerns

“The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity and the security of citizens in the digital space,” said Meutya Hafid, Indonesia’s minister of communications and digital affairs, in a statement released on Saturday.

The decision follows weeks of complaints about sexually explicit images created by Grok circulating widely on X, the social media platform owned by Musk. Users were able to request images of real people placed into explicit scenarios, including altered images that removed clothing or posed individuals in sexualised ways. Many of the images spread quickly in the platform, drawing anger from users and officials alike.

A history of strict content controls

Indonesia’s response fits into a longer pattern of strict oversight of online content. The country has blocked access to adult websites like Pornhub and OnlyFans for years and has taken swift action when platforms are seen as posing risks to users. In 2018, Indonesian authorities briefly shut down TikTok, citing concerns that some content could harm children, including sexually explicit material.

Malaysia has also stepped up scrutiny of online platforms, particularly where children are concerned. Regulators there have said they plan to bar children under 16 from using social media. The proposal follows several high-profile cases involving online bullying that were linked to the deaths of minors, increasing pressure on the government to act.

International pressure builds

Outrage over Grok’s image generation has not been limited to Southeast Asia. Officials and lawmakers in other countries have publicly criticised the chatbot’s output. In the United Kingdom, Prime Minister Keir Starmer weighed in on the issue, while US Senators Ron Wyden of Oregon, Ed Markey of Massachusetts, and Ben Ray Lújan of New Mexico sent a letter last week urging Apple and Google to remove the application from their app stores.

Their concerns reflect a broader debate about safeguards in AI tools. Many chatbots restrict users from creating images of real people or block prompts that could result in explicit content. Musk, who has often argued for fewer limits on speech, has so far resisted tightening controls in his platforms. Since his takeover of X, the platform has faced criticism for allowing harassment and abusive language to spread more freely.

Legal risks and limited responses

Legal experts and regulators say Grok’s ability to produce non-consensual sexual imagery could run afoul of laws in several countries. Possessing or sharing such material is illegal in many jurisdictions, particularly when the images involve children or minors. Some Grok-generated images reportedly included underage individuals, intensifying the backlash.

In a post on social media earlier this month, Musk said that users who directed Grok to create sexually explicit images of children would face “consequences.” He did not spell out what those consequences would be or how they would be enforced.

Paywalls fail to calm regulators

xAI and X have made some changes in response to the criticism. Last week, the platform limited Grok’s image-generation features to paying subscribers. Access is now restricted to users who pay for premium features on X. Regulators and advocacy groups, however, say the change misses the point.

Critics argue that placing the feature behind a paywall does not prevent harm and instead risks normalising the creation of abusive content. Rather than stopping misuse, they say it turns image generation into a paid option without addressing the lack of safeguards.

On Friday, a spokesman for Starmer said limiting the feature to subscribers was “not a solution” and described the move as “insulting” to victims of misogyny and sexual violence.

For now, access to Grok remains blocked in Indonesia and Malaysia as officials review next steps. The bans highlight how governments are starting to draw firmer lines around AI tools that can be used to harm individuals. They also raise questions about how quickly regulators can respond when new technologies spread faster than the rules meant to govern them.

Want to experience the full spectrum of enterprise technology innovation? Join TechEx in Amsterdam, California, and London. Covering AI, Big Data, Cyber Security, IoT, Digital Transformation, Intelligent Automation, Edge Computing, and Data Centres, TechEx brings together global leaders to share real-world use cases and in-depth insights. Click here for more information.