Cybersecurity researchers have uncovered a critical security vulnerability in the AI-as-a-service provider Replicate. This flaw could have allowed malicious actors to access proprietary AI models and sensitive data.
Cloud security firm Wiz, in a report published this week, stated that exploiting this vulnerability could have granted unauthorized access to AI prompts and results for all Replicate platform customers.
The vulnerability arises from the packaging of AI models in formats that enable arbitrary code execution, opening the door for attackers to conduct cross-tenant attacks using a malicious model.
Replicate utilizes an open-source tool called Cog to containerize and package machine learning models for deployment either in a self-hosted environment or on Replicate’s platform.
Wiz explained that they created a rogue Cog container, uploaded it to Replicate, and successfully executed remote code with elevated privileges on the service’s infrastructure.
Security researchers Shir Tamari and Sagi Tzadik noted that this code-execution technique highlights a common pattern where organizations run AI models from untrusted sources, risking potential malicious activities.
The attack utilized an established TCP connection to a Redis server instance within a Kubernetes cluster on Google Cloud Platform to inject arbitrary commands.
The manipulation of the centralized Redis server as a queue for managing customer requests could facilitate cross-tenant attacks by tampering with processes to impact other customers’ models.
These manipulations not only compromise the integrity of AI models but also jeopardize the accuracy and reliability of AI-generated outcomes.
The researchers warned of the risks posed by exposing private AI models and sensitive data, including personally identifiable information and proprietary knowledge, due to the vulnerability in Replicate.
Replicate promptly addressed the vulnerability, disclosed responsibly in January 2024, to prevent any exploitation of customer data. There is no evidence of the vulnerability being exploited in the wild.
This disclosure follows previous reports by Wiz detailing patched risks in AI platforms like Hugging Face that could enable privilege escalation, cross-tenant access, and compromise of CI/CD pipelines.
The researchers emphasized the significant risks posed by malicious models in AI systems, especially for AI-as-a-service providers, urging enhanced security measures to protect private AI models and data.
The potential impact of such attacks is severe, as attackers could gain access to millions of private AI models and applications stored by AI-as-a-service providers.
