ACVE-2026-0202
Llama 3 70B reaches 25.60% targeted AgentDojo ASR without a tool defence.
The configuration
- Configuration label
- model.family llama-3 | family llama-3 + untrustedInput=true + sideEffect=true + approval never|allowlist
- Affected entries
Model: llama-3 — ECOSYSTEM (introduced 0)
- Matcher
- all
- any
- all
model.idexistsmodel.familyin[ "llama-3" ]
- some models
familyin[ "llama-3" ]
- all
- all
- some tools
untrustedInputeqtrue
- some tools
sideEffecteqtrue
- some tools
approvalin[ "never", "allowlist" ]
- any
- all
Threat
- Attacker
- content-author
- Vector
- prompt-injection
- Outcome
- arbitrary-command
Evidence
| Benchmark | Metric | Value | n | Setup | Link |
|---|---|---|---|---|---|
| AgentDojo | targeted-asr | 0.256 | — | auto; simulated stateful workspace tools | https://agentdojo.spylab.ai/results/ |
| AgentDojo comparison | targeted-asr | 0.147 | — | auto; simulated stateful workspace tools | https://arxiv.org/html/2507.02735 |
| AgentDojo comparison | targeted-asr | 0.002 | — | auto; simulated stateful workspace tools | https://arxiv.org/html/2507.02735 |
Why it is exploitable
What the vulnerability is
The Llama 3 70B Chat configuration has a 25.60% targeted ASR on AgentDojo important_instructions with no defence. R7 identifies the model as meta-llama/Llama-3-70b-chat-hf served through Together, with quantisation unknown. The benchmark uses four simulated, stateful suites whose tools can alter the environment; it is not a claim that every Llama 3 response is unsafe. The useful comparison is Llama 3.3 70B at 14.7% on the same family of AgentDojo-style measurement and GPT-5 at 0.2% in the SecAlign table.
Which configuration is exposed
A tool-calling agent is exposed when this Llama 3 70B family reads attacker-controlled email, documents, web pages, repository content, or MCP results and can then call a side-effecting tool without a human approval step. In practical terms, that is a hosted Llama 3 70B endpoint wired to a shell, ticketing, filesystem, or browser action with auto approval. The rule is about the conjunction, not the model name alone; a text-only chat deployment does not satisfy the tool condition.
How ACVE detects it
ACVE first matches model.family llama-3 through model.id or a reachable models[] entry. It then requires both an untrustedInput tool and a sideEffect tool and checks approval for never or allowlist. A resolved model id and enumerated tools support a confirmed result. An undated or floating service alias, or an agent server that does not disclose tools, is deliberately only possible. Runtime prompt filters, planner isolation, and vendor safeguards count only if discovery records them under safeguards; the lock cannot infer them from the endpoint.
Fix
- Set agent.approval to ask before every side-effecting call. 2. Turn on tool_filter or equivalent plan-then-execute isolation so untrusted tool output cannot directly select a write or execution operation. 3. If a model change is feasible, use Llama 3.3 70B Instruct and measure it again; its 14.7% AgentDojo result is lower than 25.60%, and a later defence must still be recorded. Pin the chosen model rather than relying on a moving family tag.
Grounding needed
The reproduction must identify the exact Llama 3 70B revision, provider and quantisation, AgentDojo commit and important_instructions attack, all four suites, attack strings, seeds, auto-approval setting, and any tool filter. R7 gives H confidence: the 25.60% result is a verified primary leaderboard value, but its quantisation is unknown. The comparison values are not a substitute for a paired run, and the simulated environment should be named in any report.
Underlying CVEs / aliases
No CVE assigned. behavioural — no code defect; evidence is an evaluation
Detect
npx @pickbitsai/acve audit
For CI, emit SARIF with npx @pickbitsai/acve audit --format sarif --out acve.sarif --apply and upload it with GitHub code scanning.
Fix
Require approval, filter tool output, or move off Llama 3 70B.
| Action | Target | To | Why |
|---|---|---|---|
| reconfigure | agent.approval | ask | Prevent the 25.60% AgentDojo path from directly authorising a side effect. |
| reconfigure | defence:tool_filter | on | AgentDojo reports 6.84% for GPT-4o with tool_filter versus 47.69% without it, establishing the relevant defence pattern. |
| reconfigure | agent.model | meta-llama/Llama-3.3-70B-Instruct | The cited AgentDojo comparison is 14.7%, below Llama 3 70B at 25.60%. |
Validation
No validation entries recorded.
Grounding
No grounding checks recorded.