Indonesia Threatens to Block X and Grok AI Over Deepfake Pornography Concerns
Indonesia’s government is considering blocking access to X, the social media platform owned by Elon Musk, and its artificial intelligence service, Grok AI X, amid escalating concerns over the creation and distribution of non-consensual deepfake pornography. The move follows a government investigation revealing a lack of safeguards within the Grok AI system to prevent the manipulation of personal photos for obscene content.
The Ministry of Digital Communications (Kemkomdigi) launched an in-depth investigation into the alleged misuse of Grok AI after reports surfaced that the technology was being exploited to create explicit deepfakes using images of Indonesian citizens without their consent. According to authorities, the AI is playing a key role in generating this illicit content.
At a briefing in Jakarta on Thursday, Alexander Sabar, Director of the Digital Space Supervision Department, stated, “Based on our initial investigation, we have confirmed that there are no adequate safeguards in place within the Grok AI system to filter or regulate the creation of pornographic content based on photos of real individuals.” He further emphasized the severity of the situation, warning that it “could lead to serious rights violations.”
Director Alexander characterized the incident as a “serious violation of the right to one’s image,” going beyond simple content management failures. He explained that the digital manipulation inherent in deepfake technology “robs the victim of his or her visual identity,” inflicting significant damage to privacy, reputation, and mental health.
The Ministry of Digital Communications has issued a warning to all electronic system operators (PSEs), including X, demanding improved user protection measures. Should these platforms fail to cooperate or implement effective safeguards, the government is prepared to impose stringent penalties, including a complete block of access to both Grok AI and X services within Indonesia.
Indonesia has recently strengthened its legal framework to address the proliferation of manipulated content. The new Criminal Act No. 1 of 2023, which took effect on January 2nd, removes legal loopholes previously exploited by creators and distributors of such material. Articles 172 and 407 now stipulate penalties ranging from six months to ten years imprisonment, alongside substantial fines, for sexual exploitation and the production and distribution of pornography.
The Indonesian government is actively encouraging victims of deepfake crimes to come forward. A government official urged citizens subjected to immoral photo manipulation to immediately report the incidents to law enforcement or through the Ministry of Digital Communications and Digital Affairs’ official complaint channels.
Director Alexander concluded by reiterating the importance of accountability in the digital realm, stating, “The digital space should not become a blind spot of the law,” and emphasizing that electronic system operators have a fundamental “obligation to ensure technical and ethical safeguards to prevent their technology from degenerating into a means of sexual exploitation and destruction of individual dignity.”
