Malaysia, Indonesia Block Grok Over Deepfake Concerns

Malaysia and Indonesia have taken decisive action by blocking access to the AI chatbot Grok. These nations are the first to implement such a ban, claiming the tool lacks sufficient safeguards to prevent the creation of explicit AI-generated deepfakes. The blocks were implemented over the weekend, with Indonesia acting on Saturday and Malaysia following on Sunday.

Concurrently, the United Kingdom's media regulator, Ofcom, has initiated a formal investigation into the platform X. This probe, conducted under the Online Safety Act, focuses on the platform's handling of illegal content, specifically regarding the generation and dissemination of non-consensual intimate images and child sexual abuse material. The regulatory landscape is tightening around AI tools and social media platforms as governments move to address digital safety violations.

Indonesia and Malaysia have moved swiftly to restrict access to Grok following a scandal involving the generation of explicit deepfakes. Indonesia's temporary block was enacted on Saturday, while Malaysia implemented its block on Sunday. Both governments cited the chatbot's inability to prevent the creation and dissemination of sexually explicit deepfakes of women and children as the primary reason for the action.

Indonesia's Communication and Digital Affairs Minister, Meutya Hafid, issued a strong statement regarding the government's stance. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Hafid said. Indonesian officials noted that initial findings revealed Grok lacks effective controls to stop users from creating deepfakes based on photos of Indonesian residents.

The country's director general of digital space supervision, Alexander Sabar, elaborated on the legal and social implications. Sabar stated that generating these images violates individuals' image and privacy rights when photos are manipulated without consent. He added that such violations can lead to significant reputational, social, and psychological harm for the victims.

In Malaysia, the Malaysian Communications and Multimedia Commission cited "repeated misuse" of Grok to generate explicit and non-consensual deepfakes. The regulator specifically highlighted instances involving women and children. The commission clarified that the block will remain in place until X Corp and its parent company, xAI, establish safeguards deemed strong enough to prevent further misuse.

The UK media regulator Ofcom has opened a formal investigation into X regarding potential breaches of the Online Safety Act. The investigation centers on whether the platform has complied with its legal duties to protect UK users from content that is illegal in the country. Ofcom is specifically looking at the measures X has in place to prevent users from accessing "priority" illegal content.

Priority illegal content includes child sexual abuse material (CSAM) and non-consensual intimate images. The regulator is assessing if X removes such illegal content quickly after becoming aware of it. Furthermore, Ofcom is investigating whether X carried out an updated risk assessment before making significant changes to the platform, a requirement under the new safety laws.

The probe also extends to child protection measures. Ofcom is determining whether X assessed the risks its platform poses to UK children and if it utilizes "highly effective age assurance" to protect minors from viewing pornography. An Ofcom spokesperson expressed deep concern over reports of Grok being used to create and share illegal images. "Platforms must protect people in the UK from content that’s illegal in the UK, and we won’t hesitate to investigate where we suspect companies are failing in their duties," the spokesperson stated.

Ofcom possesses significant enforcement powers under the Online Safety Act. If the regulator determines that a company has broken the law, it can require platforms to take specific steps to come into compliance or remedy the harm caused. Ofcom contacted X on January 5 and received a response by the January 9 deadline, prompting an expedited assessment of evidence.

The financial penalties for non-compliance are substantial. Ofcom can impose fines of up to £18 million (approximately $24.3 million) or 10 percent of the company's "qualifying" worldwide revenue, whichever figure is higher. Beyond fines, the regulator can seek court orders to stop payment providers or advertisers from working with the platform, or require internet service providers to block access to the site within the UK.

The UK government has signaled its full support for any action Ofcom takes against X. Reports indicate that the UK has discussed a coordinated response to Grok-generated deepfakes with allies. Regulatory scrutiny is not limited to the UK and Southeast Asia; authorities in India and the European Union are also investigating X.

In response to the growing controversy, Grok recently announced that its image generation and editing tools would be limited to paying subscribers. However, as of Monday, reports indicated that non-paying users could still generate images through the Grok tab on the X website and app, suggesting the transition or restriction was not fully effective.

Why did Malaysia and Indonesia block Grok?
Both countries blocked Grok because they claim the AI chatbot lacks sufficient safeguards to prevent the creation and spread of explicit, non-consensual deepfakes, particularly involving women and children. They cited repeated misuse of the tool for generating these harmful images.

What is Ofcom investigating regarding X?
Ofcom is investigating whether X has violated the Online Safety Act by failing to protect users from illegal content. The probe focuses on the platform's handling of child sexual abuse material, non-consensual intimate images, and whether it has conducted proper risk assessments regarding user safety.

What are the potential consequences for X?
If found in breach of the law, X faces potential fines of up to £18 million or 10% of its global revenue. Ofcom can also seek court orders to block the platform in the UK or cut off payment processing and advertising revenue.