ACVE

ACVE advisory

ACVE-2026-0202

Llama 3 70B reaches 25.60% targeted AgentDojo ASR without a tool defence.

HIGH Behavioural Unverified advisory — awaiting validation

Status meaning: Unverified advisory — awaiting validation

The configuration

Configuration label
model.family llama-3 | family llama-3 + untrustedInput=true + sideEffect=true + approval never|allowlist
Affected entries
  • Model: llama-3 — ECOSYSTEM (introduced 0)
Matcher
  • all
    • any
      • all
        • model.id exists
        • model.family in [ "llama-3" ]
      • some models
        • family in [ "llama-3" ]
    • all
      • some tools
        • untrustedInput eq true
      • some tools
        • sideEffect eq true
    • approval in [ "never", "allowlist" ]

Threat

Attacker
content-author
Vector
prompt-injection
Outcome
arbitrary-command

Evidence

BenchmarkMetricValuenSetupLink
AgentDojotargeted-asr0.256auto; simulated stateful workspace toolshttps://agentdojo.spylab.ai/results/
AgentDojo comparisontargeted-asr0.147auto; simulated stateful workspace toolshttps://arxiv.org/html/2507.02735
AgentDojo comparisontargeted-asr0.002auto; simulated stateful workspace toolshttps://arxiv.org/html/2507.02735

Why it is exploitable

What the vulnerability is

The Llama 3 70B Chat configuration has a 25.60% targeted ASR on AgentDojo important_instructions with no defence. R7 identifies the model as meta-llama/Llama-3-70b-chat-hf served through Together, with quantisation unknown. The benchmark uses four simulated, stateful suites whose tools can alter the environment; it is not a claim that every Llama 3 response is unsafe. The useful comparison is Llama 3.3 70B at 14.7% on the same family of AgentDojo-style measurement and GPT-5 at 0.2% in the SecAlign table.

Which configuration is exposed

A tool-calling agent is exposed when this Llama 3 70B family reads attacker-controlled email, documents, web pages, repository content, or MCP results and can then call a side-effecting tool without a human approval step. In practical terms, that is a hosted Llama 3 70B endpoint wired to a shell, ticketing, filesystem, or browser action with auto approval. The rule is about the conjunction, not the model name alone; a text-only chat deployment does not satisfy the tool condition.

How ACVE detects it

ACVE first matches model.family llama-3 through model.id or a reachable models[] entry. It then requires both an untrustedInput tool and a sideEffect tool and checks approval for never or allowlist. A resolved model id and enumerated tools support a confirmed result. An undated or floating service alias, or an agent server that does not disclose tools, is deliberately only possible. Runtime prompt filters, planner isolation, and vendor safeguards count only if discovery records them under safeguards; the lock cannot infer them from the endpoint.

Fix

  1. Set agent.approval to ask before every side-effecting call. 2. Turn on tool_filter or equivalent plan-then-execute isolation so untrusted tool output cannot directly select a write or execution operation. 3. If a model change is feasible, use Llama 3.3 70B Instruct and measure it again; its 14.7% AgentDojo result is lower than 25.60%, and a later defence must still be recorded. Pin the chosen model rather than relying on a moving family tag.

Grounding needed

The reproduction must identify the exact Llama 3 70B revision, provider and quantisation, AgentDojo commit and important_instructions attack, all four suites, attack strings, seeds, auto-approval setting, and any tool filter. R7 gives H confidence: the 25.60% result is a verified primary leaderboard value, but its quantisation is unknown. The comparison values are not a substitute for a paired run, and the simulated environment should be named in any report.

Underlying CVEs / aliases

No CVE assigned. behavioural — no code defect; evidence is an evaluation

Detect

npx @pickbitsai/acve audit

For CI, emit SARIF with npx @pickbitsai/acve audit --format sarif --out acve.sarif --apply and upload it with GitHub code scanning.

Fix

Require approval, filter tool output, or move off Llama 3 70B.

ActionTargetToWhy
reconfigureagent.approvalaskPrevent the 25.60% AgentDojo path from directly authorising a side effect.
reconfiguredefence:tool_filteronAgentDojo reports 6.84% for GPT-4o with tool_filter versus 47.69% without it, establishing the relevant defence pattern.
reconfigureagent.modelmeta-llama/Llama-3.3-70B-InstructThe cited AgentDojo comparison is 14.7%, below Llama 3 70B at 25.60%.

Validation

No validation entries recorded.

Grounding

No grounding checks recorded.

References

EVIDENCE