agent-audit - Static security scanner for LLM agents — prompt in

项目说明

Agent Audit

Find security in your AI agent code before they reach production.

![PyPI ]()
![Python]()
![License: MIT]()
![CI]()
![codecov]()
![Tests]()
![Docs]()

---

Why Agent Security Fails in Production

AI agents are not just chatbots. They execute code, call tools, and touch real systems, so one unsafe input path can become a production incident.

If your team ships agent features, owns CI security gates, or operates MCP servers and tool integrations, this is a high-probability risk surface rather than an edge case.
You likely need this before every merge if agent code can trigger tools, commands, or external systems.

Agent Audit catches these issues before deployment with an analysis core designed for agent workflows today: tool-boundary taint tracking, MCP configuration auditing, and semantic secret detection, with room to extend into learning-assisted detection over time.

Think of it as security linting for AI agents, with 66 rules mapped to the OWASP Agentic Top 10 (2026).

---

Quick Start in 6 Lines

1. Install

2. Scan your project

3. Interpret and gate in CI

--se controls what is reported. --fail-on controls when the command exits with code 1.

Sample report output:

---

snapshot (as of 2026-08-11, v0.20.0, ground-truth dataset v2.2, 81 samples / 238 labels):

Reproducible from a clean clone: pip install -e packages/audit/ && python tests/benchmark/precision_recall.py --output-json results/layer1.json. Result file checked in at results/layer1_v0.19.0.json. Ground-truth label refresh for the 18 new rules (AGENT-053+) lands June 2026.

Details: Benchmark Results | Competiti

---

What It Detects

| Category | What goes wrong | Example rule |
|----------|----------------|--------------|
| Injection attacks | User input flows to exec(), subprocess, SQL | AGENT-001, AGENT-041 |
| Prompt injection | User input concatenated into system prompts | AGENT-010 |
| Leaked secrets | API keys hardcoded in source or MCP config | AGENT-004, AGENT-031 |
| Missing input | functions accept raw strings without checks | AGENT-034 |
| Unsafe MCP servers | No auth, no pinning, overly broad permissions | AGENT-005, AGENT-029, AGENT-030, AGENT-033 |
| MCP tool poisoning | Hidden instructions or data exfiltration in tool descriptions | AGENT-056, AGENT-057 |
| MCP tool shadowing | Multiple servers register identical tool names to o behavior | AGENT-055 |
| MCP rug pull / drift | Ser change after initial security audit | AGENT-054 |
| No guardrails | Agent runs without iteration limits or human approval | AGENT-028, AGENT-037 |
| Unrestricted code execution | Tools run eval() or shell=True without sandboxing | AGENT-035 |
| Source map leakage | Debug artifacts (.map, .pdb) included in published agent packages | AGENT-110 |
| Sub-agent pri escalation | Child agents inherit parent's full tool set without restriction | AGENT-112 |
| Delegation without auth | Cross-agent delegation without identity | AGENT-113 |
| Auto-approve all tools | Agent auto-approves tool execution without safety classification | AGENT-117 |
| HITL bypass | Human-in-the-loop appro or self-modification | AGENT-118 |
| Trace suppression | AI attribution removed from git commits, logs, or outputs | AGENT-119 |
| Config hooks poisoning | Malicious hooks in .claude/settings.json, .cursor/, .mcp.json (C) | AGENT-120 |

Full co of all 10 OWASP Agentic Security categories. Framework-specific detection for LangChain, CrewAI, AutoGen, and AgentScope. See all rules ->

---

OpenClaw Support

Agent Audit is a as an OpenClaw skill on ClawHub:

Once installed, ask your OpenClaw agent:

The scanner covers all 10 OWASP Agentic AI threat categories, including se written specifically for OpenClaw skills (AGENT-058 to AGENT-064).

---

Who Is This For

---

Usage

Baseline Scanning

Track only *new* findings across commits:

GitHub Actions

<details>
<summary><b>Show GitHub Action Example and Inputs</b></summary>
<br/>

| Input | Description | Default |
|-------|-------------|---------|
| path | Path to scan | . |
| format | Output format: terminal, json, sarif, markdown | sarif |
| se | Minimum se to report | low |
| fail-on | Exit with error at this se | high |
| baseline | Baseline file for incremental scanning | - |
| upload-sarif | Upload SARIF to GitHub Security tab | true |

</details>

---

E Results

<details>
<summary><b>Show E Details</b></summary>
<br/>

E on the labeled benchmark in tests/ground_truth/labeled_samples.yaml (81 samples, 236 positive + 2 negati, GT v2.2) — reproducible via python tests/benchmark/precision_recall.py. Bandit and Semgrep numbers below were measured on the equi injection/RCE/credential subset that those tools can express; see BENCHMARK-RESULTS.md for methodology.

| Tool | Recall | Precision | F1 |
|------|-------:|----------:|---:|
| agent-audit (v0.20.0, raw, reproducible) | 84.32% | 68.86% | 0.7581 |
| Bandit 1.8 | 29.7% | 100% | 0.46 |
| Semgrep 1.x | 27.0% | 100% | 0.43 |

| Category | agent-audit | Bandit | Semgrep |
|----------|:-----------:|:-----:|:-------:|
| Set A -- Injection / RCE | 100% | 68.8% | 56.2% |
| Set B -- MCP Configuration | 100% | 0% | 0% |
| Set C -- Data / Auth | 84.6% | 0% | 7.7% |

> Neither Bandit nor Semgrep can parse MCP configuration files -- they achieve 0% recall on agent-specific configuration (Set B).

Full e details: Benchmark Results | Competiti

</details>

How It Works

<details>
<summary><b>Show Architecture and Technical Notes</b></summary>
<br/>

Key technical contributions:

</details>

Threat Co

66 detection rules co all 10 categories of the OWASP Agentic Top 10 (2026):

| OWASP Category | Rules | Example Detections |
|----------------|------:|-------------------|
| ASI-01 Agent Goal Hijack | 7 | Prompt injection, tool description poisoning, argument poisoning |
| ASI-02 Tool Misuse | 11 | input to subprocess without , browser/subprocess sandbox |
| ASI-03 Identity & Pri | 12 | Daemon pri escalation, sudoers NOPASSWD, sub-agent boundary, >10 MCP servers |
| ASI-04 Supply Chain | 17 | Un MCP source, tool shadowing, baseline drift (rug pull), extension boundary, deserialization, OpenClaw skill obfuscation |
| ASI-05 Code Execution | 5 | eval/exec in tool without sandbox, credential store access, skill sandbox o |
| ASI-06 Memory Poisoning | 3 | Unsanitized input to store upsert, persistent session memory |
| ASI-07 Inter-Agent Comm | 1 | Multi-agent over HTTP without TLS |
| ASI-08 Cascading Failures | 3 | AgentExecutor without max_iterations |
| ASI-09 Trust Exploitation | 10 | Critical ops without human_in_the_loop, HITL bypass, trace suppression |
| ASI-10 Rogue Agents | 3 | No kill switch, no behavior monitoring, self-modification |

Real-World

<details>
<summary><b>Show Real-World Target Results</b></summary>
<br/>

Scanned 9 open-source projects to detection quality:

| Target | Project | Findings | OWASP Categories |
|--------|---------|----------|------------------|
| T1 | damn- | 4 | ASI-01, ASI-02, ASI-06 |
| T2 | Damn | 41 | ASI-01, ASI-02, ASI-04 |
| T3 | langchain-core | 3 | ASI-01, ASI-02 |
| T6 | openai-agents-python | 25 | ASI-01, ASI-02 |
| T7 | adk-python | 40 | ASI-02, ASI-04, ASI-10 |
| T8 | agentscope | 10 | ASI-02 |
| T9 | crewAI | 155 | ASI-01, ASI-02, ASI-04, ASI-07, ASI-08, ASI-10 |
| T10 | MCP Config (100-tool server) | 8 | ASI-02, ASI-03, ASI-04, ASI-05, ASI-09 |
| T11 | streamlit-agent | 6 | ASI-01, ASI-04, ASI-08 |

10/10 OWASP Agentic Top 10 categories detected across targets. Quality gate: PASS.

</details>

Comparison with Existing Tools

| Capability | agent-audit | Bandit | Semgrep |
|-----------|:-----------:|:-----:|:-------:|
| Agent-specific threat model (OWASP Agentic Top 10) | Yes | No | No |
| MCP configuration auditing | Yes | No | No |
| Tool-boundary taint analysis | Yes | No | No |
| decorator awareness | Yes | No | No |
| Semantic credential detection | Yes | Basic | Basic |
| General Python security | Partial | Yes | Yes |
| Multi-language support | Python-focused | Python | Multi |

agent-audit is complementary to general-purpose SAST tools. It targets the security gap specific to AI agent applications that existing tools cannot address.

Configuration

Current Scope

<details>
<summary><b>Show Current Limitations and Scope</b></summary>
<br/>

</details>

Documentation

De

See CONTRIBUTING.md for full de setup and PR guidelines.

Citation

If you use agent-audit in your research, please cite:

Acknowledgments

License

MIT -- see LICENSE.

项目信息

开发语言：Python

Star 数：237

技术要点

该项目属于安全软件方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。