Malaysia and Indonesia first to block Grok following CSAM scandal

Malaysia and Indonesia are the first countries to block Grok, saying X’s chatbot does not have sufficient safeguards to prevent Explicit deepfakes of women and children generated by AI to be created and broadcast on X. Indonesia temporarily blocked access to Grok on Saturday, as did Malaysia on Sunday, the Associated Press reports. Meanwhile, UK media regulator Ofcom has opened a formal investigation in X under the Online Safety Act.

“The government considers non-consensual sexual deepfakes a serious violation of the human rights, dignity and safety of citizens in the digital space,” Indonesian Minister of Communication and Digital Affairs Meutya Hafid said in a statement. Country officials said initial findings showed Grok did not have effective controls in place to prevent users from creating and sharing sexually explicit deepfakes based on photos of Indonesian residents. Director General of the Country’s Digital Space Supervision Alexander Sabar said the generation of deepfakes can violate individuals’ image and privacy rights when photos are shared or manipulated without consent, adding that they can lead to reputational, social and psychological damage.

The Malaysian Communications and Multimedia Commission cited “repeated misuse” of Grok for generate explicit, non-consensual deepfakessome of which involved women and children. The regulator said Grok would remain stuck in the country until X Corp and its parent company xAI establish sufficiently strong safeguards.

Ofcom’s investigation will focus on whether X has “met its obligations to protect people in the UK from content that is unlawful in the UK”. This includes whether whether the platform removes illegal content promptly after becoming aware of it; and whether X carried out an updated risk assessment before making “significant changes” to the platform. The investigation will also examine whether X has assessed the risk its platform poses to British children and whether it has a “highly effective age guarantee to protect British children from pornography”.

The regulator said it contacted X on January 5 and received a response before the January 9 deadline. Ofcom is conducting an “expedited assessment of urgently available evidence” and added that it has asked xAI for “urgent clarification” on the steps the company is taking to protect UK users.

“Reports that Grok was used to create and share illegal and non-consensual intimate images and child sexual abuse material on X are deeply concerning,” an Ofcom spokesperson said. “Platforms must protect UK citizens from content that is illegal in the UK, and we will not hesitate to investigate cases where we suspect companies are failing in their duties, particularly where there is a risk of harm to children. We will progress this investigation as a matter of priority, while ensuring due process is followed. As the UK’s independent online safety enforcement agency, it is important that we ensure our investigations are legally robust and fair decided.”

If Ofcom believes a company has broken the law, it can “require platforms to take specific steps to comply or to remedy the harm caused by the breach”. The regulator can further impose fines of up to £18 million ($24.3 million) or 10% of “eligible” global turnover, whichever is greater. It can also seek a court order to prevent payment providers or advertisers from working with a platform, or to require internet service providers to block a site in the UK. The British government has stated that it step back from any action that Ofcom takes against X.

Reports over the weekend suggested the UK had held discussions with allies on a coordinated response to Grok-generated deepfakes. Regulators elsewhere, notably in India and the European Union, are also investigate x.

Last week, Grok started telling users that its image generation and editing tools were limited to paying subscribers. But as of Monday, it was still possible for non-paying users to generate images through the Grok tab of the X website and app.