Fake OpenAI Repo Hit #1 on Hugging Face, Stole Windows Credentials

A malicious Hugging Face repository impersonating OpenAI's "Privacy Filter" project briefly reached the #1 trending spot on the platform and accumulated 244,000 downloads before being removed. The repository delivered an infostealer targeting Windows users, capitalizing on the AI research community's trust in the platform.

The attack exploited Hugging Face's role as the de facto repository for AI models and datasets. Researchers and developers regularly download model weights and training code from the platform, often executing code without the scrutiny applied to traditional software packages.

How the Attack Worked

The attackers created a repository that mimicked OpenAI's branding and naming conventions, claiming to offer a privacy-preserving filter for sensitive data processing. The description and documentation appeared legitimate, referencing genuine AI safety concepts and citing real research papers.

When users cloned the repository and ran the setup scripts, the infostealer payload executed alongside any legitimate functionality. The malware targeted:

• Browser credentials and session cookies
• Cryptocurrency wallet data
• SSH keys and authentication tokens
• Configuration files containing API keys

The repository's rapid climb to #1 trending amplified its reach. Hugging Face's trending algorithm favors repositories with high download velocity, creating a feedback loop where the malware's initial traction drove additional visibility.

Targeting AI Developers

This campaign specifically targeted AI practitioners—a demographic likely to have valuable credentials. AI researchers often have access to cloud compute resources, enterprise API keys, and proprietary datasets. Developers working with models frequently handle credentials for GPU clusters, cloud storage buckets, and inference endpoints.

The attack echoes similar supply chain compromises we've covered, including the npm package targeting Claude AI users discovered this week. AI tooling has become a primary attack vector for credential theft.

Platform Response

Hugging Face removed the repository after receiving reports, though the 244,000 download count suggests significant exposure before takedown. The platform has faced criticism for not implementing stronger verification of repository ownership, particularly for profiles claiming affiliation with major AI companies.

Unlike npm or PyPI, which have had years to develop security scanning for malicious packages, AI model repositories present unique challenges. Model weights can be gigabytes in size, making automated analysis expensive. And the boundary between "model file" and "executable code" is often blurry—many model formats include embedded Python code for preprocessing and inference.

Protecting Yourself

1. Verify repository ownership - Check that accounts claiming affiliation with major companies have verification badges or external confirmation
2. Inspect before executing - Review setup scripts and training code before running, especially from unfamiliar repositories
3. Use isolated environments - Run unfamiliar models in sandboxed containers or VMs without access to sensitive credentials
4. Monitor for trending anomalies - Legitimate repositories rarely appear on trending without prior community discussion

Why This Matters

Hugging Face has become infrastructure for the AI industry. Researchers share models there. Companies distribute their pre-trained weights there. The platform's implicit trust makes it an attractive target for supply chain attacks.

The 244,000 downloads represent potential compromises across academia, enterprise AI teams, and independent researchers. Many of those downloads may have been automated—pipelines that pull popular models without human review.

This attack demonstrates that AI infrastructure is now subject to the same supply chain threats as traditional software development. The malware techniques targeting developers have simply adapted to where developers are working. As AI becomes more central to software development, securing AI toolchains becomes essential.

For organizations using Hugging Face in their workflows, this is a reminder that "download from a popular platform" isn't a security control. Model provenance, code review, and isolated execution remain necessary even when the source appears reputable.