AI-Agent-Security - Educational security research on AI coding agent a

项目说明

🛡️ AI Agent Security Research

> *Documenting how AI coding agents get exploited -- so we can build better defenses.*

![Educational]()
![AI Agents]()
![No Li]()

---

🔍 What This Is

A growing collection of research, annotated attack examples, and defense strategies targeting the security of AI coding agents -- Claude Code, Cursor, Copilot, Windsurf, and the broader ecosystem of LLM-powered de tools.

Every attack has a defense. Every payload is annotated, defanged, and educational.

> Note: This project is actively maintained and frequently updated as new findings emerge, attack surfaces evolve, and AI-assisted research uncovers new patterns. Expect content to change regularly.

---

📝 Research Notes

| | Topic | What You'll Learn |
|---|-------|-------------------|
| 🗂️ | Tools & Frameworks Index | Quick reference for every tool, framework, benchmark, and standard mentioned across all notes |
| 💉 | Prompt Injection & Skill Injection | Foundational injection concepts, agent attack surface, trojanized skill teardown, supply chain comparison |
| 🧱 | Defense Patterns | Sanitization, sandboxing, and mitigation strategies with working code |
| ⚙️ | Claude Code Skill Architecture | How Claude Code's extensibility (skills, hooks, MCP) creates attack surface |
| 👻 | LLM Hallucination Pre | Why models invent things, how to detect it, and how to stop it |
| 🌐 | AI Coding Language Performance | Multilingual benchmarks, token efficiency, and language-steering attacks |
| 🔓 | LLM Jailbreaking Deep Dive | Full taxonomy: DAN to GCG to Crescendo, defenses, benchmarks, agent implications |
| 🔍 | Skill Scanning & Detection Landscape | Cisco Skill Scanner, , ToxicSkills audit, gap analysis, what to build next |
| 📋 | AI GRC & Policy Landscape | NIST AI RMF, EU AI Act, ISO 42001, state laws, agentic go, OWASP Agentic Top 10 |
| 🧠 | AI Memory & Corruption | Memory architectures, RAG poisoning, MINJA, persistence risks, real-world case studies, defenses |
| 📄 | Agent Configuration Files | Cross-tool instruction file attack surface: CLAUDE.md, AGENTS.md, Copilot, Cursor, Unicode obfuscation, hardening recommendations |
| 🧠 | Chatbot & AI Psychosis | AI-induced psychosis, sycophancy mechanisms, documented deaths, folie a deux, weaponization, RAND national security analysis |
| 🦞 | OpenClaw & ClawHub Security | OpenClaw architecture, ClawHub supply chain, C, ClawHa, AMOS stealer, memory poisoning, 42K exposed instances |
| 🏪 | AI Application Ecosystem Security | GPT Store, MCP tool poisoning, LangChain, HuggingFace, AutoGPT, CrewAI, Devin, IDEsaster, GlassWorm, OWASP Agentic Top 10, MITRE ATLAS |
| ⚔️ | AI Hacking Frameworks | XBOW, Shannon, Strix, PentAGI, CAI, Reaper, Nebula, CHECKMATE, Garak, Promptfoo, PyRIT, benchmarks, architecture patterns |
| 💩 | Bullshit Benchmark & LLM Honesty | BullshitBench, TruthfulQA, SimpleQA, sycophancy benchmarks, Bullshit Index, abstention, slopsquatting, RLHF-security tension |
| 🛡️ | AI Blue Teaming & Defensive AI | AI SOC agents, CrowdStrike Charlotte, Microsoft Security Copilot, malware RE, DARPA AIxCC, NIST AI 100-2, defender's ad analysis |
| 🔤 | Unicode Selector Attacks | In jailbreaking, guardrail e, Sneaky Bits encoding, GlassWorm malware, token expansion DoS, defense: explicit stripping required |
| 🪙 | Token Optimization & LLM Efficiency | Context engineering, prompt structure, model routing, caching, batching, agent loop optimization, Claude Code cost management |
| 💸 | Token-Based Attacks & Resource Exploitation | Denial of wallet, sponge examples, reasoning exhaustion (ThinkTrap/ReasoningBomb), context window poisoning, token smuggling, tokenizer security, model extraction, LLMjacking |
| 📊 | LLM Landscape & Token Economics | Security-framed model reference: tokenization attack surface, cost economics for threat modeling, model selection for security work, open model supply chain risks, Chinese censorship implications |
| 🤝 | Multi-Agent Security | Agent-to-agent attacks, A2A protocol spoofing, 82.4% peer compliance bypass, delegation chain injection, memory contagion, cross-agent config attacks, offensi, defense patterns |

---

🧪 Attack / Defense Examples

Hands-on annotated scenarios -- each one shows the attack and the fix.

| | Technique | TL;DR |
|---|-----------|-------|
| 🕵️ | Hidden Comment Injection | HTML comments are in in markdown previews but the LLM reads every word |
| 🌊 | Indirect Prompt Injection | Poison the web page, API response, or file the agent fetches -- it obeys |
| 📤 | Data Exfiltration | The agent becomes an unwitting mule for your secrets, keys, and credentials |
| 📦 | Hallucinated Package Injection | LLM invents a package name, attacker registers it -- instant supply chain attack |
| 🔧 | MCP Tool Poisoning | Malicious instructions hidden in tool descriptions hijack agent behavior silently |
| 💸 | Resource Exhaustion & Denial of Wallet | Reasoning bombs, context saturation, and cost math -- drain API budgets without crashing the service |
| 🔤 | Unicode In Injection | selectors and Tags block encode hidden instructions that survive diff review and Unicode normalization |

---

🗂️ Attack Taxonomy

---

🔧 Security Skill Suite

Working defensi built on Claude Code's skill + hook architecture. These turn the research above into practical detection.

Install from ClawHub

The fastest way to install -- each link goes to the ClawHub listing:

| Skill | ClawHub | What It Does |
|---|---|---|
| | clawhub.ai/ItsNishi/ | Scans .claude/, .mcp.json, CLAUDE.md, VS Code/Cursor configs for hook abuse, injection, MCP poisoning |
| scan-skill | clawhub.ai/ItsNishi/scan-skill | Deep analysis of a single skill before installation -- frontmatter, HTML comments, persistence triggers, supporting scripts |
| audit-code | clawhub.ai/ItsNishi/audit-code | Code security review -- hardcoded secrets, dangerous calls, SQL injection, .env files, file permissions |

Install from Source

If you prefer to install manually from this repo:

Usage

Once installed, invoke in any Claude Code session:

Prerequisites

Hooks

Ad PreToolUse guards in .claude/settings.json that warn (not block) on:

To install the hooks, copy .claude/settings.json into your project's .claude/ directory.

Shared Pattern Database

151 detection patterns across 15 categories. Each skill bundles its own copy of patterns.py so it works standalone:

All patterns derived from the research notes and examples in this repo.

---

📁 Project Structure

---

⚠️ Disclaimer

This research is for educational and defensi only. All examples use defanged URLs (hxxps://, [.]), annotated payloads marked [MALICIOUS], and non-executable demonstrations. Every attack technique includes corresponding defenses.

---

📜 License

MIT

项目信息

开发语言：Python

Star 数：4

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Educational security research on AI coding agent attack surfaces -- prompt injection, skill poisoning, defense patterns展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。