The Grok AI app developed by Elon Musk's xAI is facing renewed scrutiny after concerns surfaced over its ability to generate sexualized deepfake images of real individuals.
The controversy escalated to the point where the app reportedly came close to being removed from the Apple App Store earlier this year.
Apple Warned xAI Over Potential App Store Removal
Apple informed U.S. senators that it had warned xAI about the possible removal of Grok unless the company addressed issues involving non-consensual explicit deepfakes.
NBC News reported that the concerns were that the app's image generation capabilities were being misused to create manipulated sexualized content, which later circulated across X.
Apple has not publicly commented on the most recent allegations, and xAI has also not issued a detailed response.
Grok AI Image Tools and Reported Misuse
Grok functions as xAI's core artificial intelligence assistant on X, offering chatbot features along with image and video generation capabilities. However, investigations suggest that users have exploited these tools to generate inappropriate and explicit AI-created content, including altered images of real individuals.
Since its deepfake scandal was overwhelmingly alarming, it came to the point that Indonesia and Malaysia blocked Grok's access. As a follow-up, the UK also expressed its desire to block the chatbot after the Online Safety Act review.
xAI has stated that it enforces strict policies against non-consensual explicit content and relies on monitoring systems, filters, and model updates to reduce misuse. The company maintains that it is continuously improving safeguards to prevent abuse of its tools.
Apple's App Store Review and Temporary Reapproval
CNET reported that Apple confirmed in its communication to lawmakers that both X and Grok underwent compliance reviews following public backlash. The app was initially rejected but later resubmitted with changes that met Apple's requirements, allowing it to return to the App Store.
The company reiterated that apps violating policies on harmful or explicit content may be removed if they fail to comply with platform standards.
Regulatory Pressure Over AI-Generated Deepfakes Intensifies
The controversy has also drawn political attention, with U.S. Senator Ron Wyden criticizing the lack of stronger enforcement against non-consensual AI-generated content.
Lawmakers have increasingly questioned major tech companies, including Apple and Google, over their oversight of generative AI tools.
Earlier this year, some senators couldn't help but call out Grok for its "sickening" content generation. This even reached the point where they asked Apple and Google to remove the apps because they were using information from "real and private citizens."
Originally published on Tech Times
