Software supply chain security became a mainstream boardroom concern after incidents like SolarWinds showed that compromising one upstream dependency can compromise thousands of downstream victims at once. The AI stack has quietly grown its own version of the same problem — except now the "dependency" can be a 4GB file of model weights, or a package name an AI assistant invented on the spot. Both are already being exploited or actively researched, not theorized about.
The Model File Itself Can Be Malware
Most PyTorch models are distributed using Python's pickle module — a serialization format designed to save and restore Python objects. The problem is that pickle wasn't built to be safe against untrusted input: it can encode instructions that execute arbitrary code the moment the file is loaded, not just data. Loading a model, in other words, can mean running someone else's code with your own privileges.
This isn't a hypothetical risk. In February 2024, JFrog's security research team published findings from ongoing scans of models uploaded to Hugging Face and identified around 100 models carrying genuinely malicious payloads — not proof-of-concept demonstrations, but working exploits. One example they documented, uploaded under the account baller423, contained a pickle payload that opened a reverse shell to an external IP address the moment the model was loaded, handing an attacker a live connection into whatever machine had just tried to use the model. When that model was removed, near-identical payloads reappeared under different account names pointing at different IP addresses. Separately, CVE-2023-6730 documented a related risk in the Hugging Face transformers library, where loading a seemingly safe model could transitively trigger code execution from another compromised model referenced underneath it.
Hugging Face has real mitigations in place: automated pickle-import scanning that flags suspicious opcodes, malware scanning via ClamAV, secrets scanning, and — most importantly — a purpose-built alternative format called safetensors, which stores only tensor data with no capacity to encode executable instructions at all. But it's worth being precise about what the scanning actually does today: models flagged by the pickle scanner are marked "unsafe" on the Hub, not blocked outright. Downloading and loading them remains possible, and the decision — and the risk — still sits with whoever pulls the file.
The Second Vector: AI Assistants Hallucinate Their Own Dependencies
The second emerging threat starts from the opposite direction: instead of a malicious file waiting to be downloaded, it's a non-existent package name that an AI coding assistant confidently recommends, which an attacker then registers and fills with malware before anyone notices it was never real. Security researcher Seth Larson, Developer-in-Residence at the Python Software Foundation, coined the term "slopsquatting" for this pattern in April 2025 — a portmanteau of "AI slop" and "typosquatting."
The scale of the underlying problem is not a fringe finding: an academic study, "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs" (Spracklen et al.), tested a range of code-generating models and found that 19.7% of recommended packages didn't exist at all. Open-source models hallucinated far more often — 21.7% on average — than proprietary, commercially hosted models, at 5.2%. Across the full study, researchers catalogued more than 205,000 unique hallucinated package names, a number large enough to make squatting on the popular ones a low-effort, high-yield strategy for an attacker.
This isn't purely academic risk either. Back in 2023, well before the term existed, researcher Bar Lanyado demonstrated the mechanics directly: he noticed ChatGPT hallucinating a package name resembling huggingface-cli (the real install command is pip install -U "huggingface_hub[cli]"), uploaded an empty package under that exact hallucinated name as a test, and it picked up more than 30,000 downloads in three months — proof that developers really do copy-paste and install whatever a model suggests, without checking whether the package is real.
A caveat worth stating plainly: as of mid-2026, there is no confirmed, publicly documented case of slopsquatting being used as a complete real-world attack chain — i.e., an attacker registering a hallucinated name with malicious code that then compromised someone through it. The mechanics are proven, the exposure is measurable and growing, but the fully realized attack remains, for now, closer to a well-evidenced risk than a catalogued incident. Treat that distinction as current fact, not a reason for complacency.
Same Root Cause, Two Different Doors
Malicious model weights and hallucinated packages exploit the same underlying trust assumption from two different directions: "if it's on the platform, or if the model suggested it, it must be legitimate." AI accelerates both sides of that failure simultaneously — it accelerates the production side, by generating plausible-but-fictional dependency names at a scale no human typosquatter could match, and it accelerates the consumption side, by making it easier than ever to pip-install a suggested package or download a leaderboard-topping model without a human ever pausing to verify the source. The common failure mode isn't a specific tool or format — it's skipping the verification step because the output looked confident and the workflow was moving fast.
Practical Defenses That Actually Reduce Risk
None of this requires abandoning open model or package ecosystems — it requires treating them the way mature engineering teams already treat any other third-party dependency:
Prefer safetensors and refuse untrusted pickle files. If a model is only available in pickle format from an unfamiliar or unverified source, that alone is a reason to slow down, not a reason to trust it more because it's "just weights."
Load models from accounts and organizations you actually trust, and check for signed commits. Provenance matters as much for a model as it does for a code dependency — arguably more, since a model's internals are far harder to audit by inspection than a diff.
Pin and verify package hashes; don't let an AI assistant pip install blind. Lock files and hash verification against known-good versions catch a hallucinated or squatted package before it ever executes, regardless of whether a human or an AI assistant typed the install command.
Treat AI-suggested dependencies and models as unverified input, not settled fact. The review discipline that applies to any AI-generated code should apply doubly to anything that pulls in external code or weights — a wrong sentence in a chat response is embarrassing; a malicious dependency executing in your build pipeline is a breach.
Run dependency and model scanning in CI, not just at commit time. Automated scanners for known malicious packages and models catch the cases a rushed human reviewer won't, and they scale with how fast AI-assisted development actually moves.
Why This Matters Right Now
Both of these attack vectors are growing precisely because AI-assisted development is growing. More teams are pulling pretrained models off public hubs instead of training from scratch, and more code is being written with an AI assistant suggesting the dependencies to install. That's a legitimate and often correct way to build faster — but it also means the traditional "a human typed this, so someone looked at it" checkpoint is disappearing from more of the pipeline than most teams realize.
Key takeaways:
- Malicious models are a documented, active threat, not a theoretical one — JFrog identified around 100 genuinely malicious models on Hugging Face carrying real payloads (including reverse shells), exploiting the fact that PyTorch's default pickle format can execute code on load; safetensors and platform scanning reduce but don't eliminate the risk, since flagged pickle files are marked, not blocked.
- Slopsquatting is real and measurable — studies found nearly 20% of LLM-suggested packages don't exist, with over 205,000 unique hallucinated names catalogued — but as of mid-2026, no fully realized real-world attack has been publicly confirmed; treat the exposure as serious and growing, not as an already-materialized incident.
- Both vectors exploit the same trust gap: AI accelerates the production of plausible-looking malicious content and the speed at which humans skip verifying it. Provenance checks, hash pinning, safetensors adoption, and CI-stage scanning are the concrete, low-cost controls that close that gap today.
If you're building AI-integrated pipelines and want a second opinion on how your model and dependency supply chain is actually secured, get in touch.