San Francisco’s city attorney demanded Apple and Google remove 13 AI apps that generate non-consensual nude images, citing California law and alleging the tech giants aided and abetted
their proliferation through app-store fees and distribution.
On July 17, 2026, San Francisco City Attorney David Chiu sent cease-and-desist letters to Apple and Google, targeting 13 apps that use generative AI to create non-consensual nude images by digitally removing clothing from photos. The legal action alleges the tech companies knowingly facilitate
deepfake pornography and have likely earned millions of dollars in fees
from these apps. The letters invoke California’s 2025 law letting victims pursue civil claims against third-party facilitators, framing Apple and Google as active distributors rather than passive platforms.
Legal Action and the City Attorney’s Argument
Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,
Chiu said in a statement cited by WIRED. The letters demand the companies comply or face potential litigation. Chiu’s office also highlighted a 2026 report by the Tech Transparency Project, which found these apps remained on store shelves despite prior warnings, some rated “E for Everyone”—a classification that allows access to children.
Company Responses and the Moderation Gap
Apple and Google responded with limited compliance. WIRED reports Apple removed three of the eight flagged apps and is terminating developer accounts, while Google suspended five of the five targeted apps. Neither company addressed broader systemic issues, such as how these apps persisted for months after being flagged by researchers. The Tech Transparency Project’s January and April 2026 reports had already documented the apps’ availability, yet both companies took no meaningful action until legal threats arose.
Google’s statement emphasized its policy against sexual content, noting it continually take[s] proactive steps to detect and remove apps with harmful content.
Apple reiterated that developers are ultimately responsible for app content, though it acknowledged removing three of the flagged apps. Both companies’ responses have been criticized as reactive rather than systemic.
Broader Implications for App-Store Moderation
The case is being described as the first municipal-level app-store removal mandate of its kind targeting AI-generated content in the US. Chiu’s legal framing—arguing that Apple and Google aided and abetted
the apps—challenges the traditional defense that platforms are merely “hosts” of user-generated content. If upheld, this could force tech companies to reevaluate their revenue-sharing models and moderation practices.
Experts note the case highlights a “moderation gap” where apps exploit loopholes in developer policies. One app, cited by WIRED, ran paid ads on YouTube and X, funneling users to its download pages. The fact that these apps remained on store shelves despite prior warnings underscores the tension between economic incentives (app-store fees, ad revenue) and enforcement of anti-pornography policies.
The outcome could reshape how local attorneys general approach platform accountability. Chiu’s office has already targeted deepfake websites, and Senator Jon Ossoff’s office previously raised concerns about AI-generated nudes affecting minors, citing survey data that 16% of US teens report knowing someone personally targeted by an AI-generated deepfake image as a minor, and 11% report deepfake nudes of themselves as minors being shared or reshared. If Apple and Google comply broadly, it may prompt other jurisdictions to adopt similar tactics, creating a patchwork of local regulations rather than a unified federal approach.
The key unresolved question is whether Apple and Google will address the systemic issues or merely remove the 13 apps. Neither company has indicated plans to overhaul its moderation processes. Meanwhile, the city attorney’s office has given both firms a deadline to comply, with threats of litigation if they fail. The case also raises broader questions about the legal liability of tech platforms for AI-generated harms, a debate that could influence future legislation and court rulings.
Sources: WIRED, varindia.com.
Related reading
