Awesome-Hacking-with-AI - A curated, evidence-led guide to AI for security a

项目说明

<p align="center">
<img src="assets/images/logo1.png" alt="Awesome Hacking with AI Logo" width="700">
</p>

---

<di="center">

![PRs Welcome]()
![Awesome]()
![Stars]()
![Last Updated]()

</div>

---

Awesome Hacking with AI

> A curated, e guide to AI for security and security for AI systems. It brings together authorized security testing, application security, AI agents, model adaptation, datasets, benchmarks, MCP security, defensi, and research.

This repository is for security practitioners, AI engineers, researchers, educators, and authorized red teams. It is deliberately broad: the aim is to explain what a resource does, why it matters, what e supports it, and where it belongs in a responsible workflow.

Responsible Use

All practical security work must be explicitly authorized, legally compliant, and proportionate to the agreed scope. This repository supports research, education, hardening, , detection, remediation, disclosure, and authorized testing. It does not provide instructions for unauthorized access, e protections, deploying malware, operating command-and-control, impersonating people, or targeting third-party systems.

> If you cannot demonstrate authorization for a target, do not test it. Use a lab, a benchmark, a capture-the-flag en, or an intentionally training application instead.

For coordinated disclosure guidance, consult the OWASP Disclosure Cheat Sheet and CERT/CC.

Contents

How to Na

The collection follows two complementary paths. AI for security concerns how AI can improve code review, management, detection engineering, threat intelligence, incident response, and authorized assessments. Security for AI systems concerns models, data, RAG, agents, memory, tools, MCP servers, and their software supply chain.

| Reader goal | Start with | Then continue with |
|---|---|---|
| Secure an LLM, RAG, or agentic application | LLM, RAG, and Agent Security | MCP Security and AI Security Guide |
| E security agents responsibly | Authorized Security Agents | Benchmarks and Agentic Security Guide |
| Specialize a model for a security task | Models, Data, and Adaptation | Models, Data, and E Guide |
| Build an approved MCP workflow | MCP Security | MCP ser and MCP Security Guide |
| Learn systematically | Learning Path | Foundations, extended guides, benchmarks, and labs |

Foundations, Go, and Threat Taxonomies

Security work in AI benefits from a shared , a lifecycle view, and an e model. The following resources help teams reason about risk before selecting a tool or model.

| Resource | Type | Why it matters |
|---|---|---|
| OWASP GenAI LLM Top 10 | Guidance | Current OWASP guidance for LLM and generative AI application risks; the former site is maintained as a historical entry point.[1] |
| OWASP AI Security and Privacy Guide | Guidance | Broad security and privacy reference for AI-system design, deployment, and operation. |
| NIST AI RMF | Framework | framework organized around Govern, Map, Measure, and Manage.[2] |
| NCSC Secure AI System De | Lifecycle guidance | Secure design, de, deployment, and operation/maintenance for AI systems.[3] |
| OWASP AI Security Standard (AISVS) | standard | , testable security requirements across data, models, infrastructure, identities, agents, MCP, ad robustness, and monitoring.[20] |
| MITRE ATLAS | Threat knowledge base | Living knowledge base of threats to AI-enabled systems, with techniques, mitigations, and case studies.[4] |
| NIST Ad ML Taxonomy | Taxonomy | Shared terminology for ad ML threats and mitigations. |
| AI Incident Database | Incident resource | Documented AI incidents to inform risk discovery and go exercises. |
| A | Risk taxonomy | Structured for AI risk identification and documentation. |
| MLSecOps Top 10 | Practice guide | Security perspective on the ML lifecycle and operational controls. |

Security for AI Systems

LLM, RAG, and Agent Security

An AI application is more than a model. Prompts, retrie, stores, identities, tool definitions, code execution paths, logs, memory, deployment infrastructure, pro, and users are all part of the security boundary. Secure design should consider the whole system.[2] [3]

| Resource | Category | Description |
|---|---|---|
| OWASP GenAI Security Project | Community guidance | Active home for LLM, agentic AI, red teaming, incident response, and data-security resources. |
| Promptfoo | Testing framework | Declarati, red teaming, and regression testing for LLM applications, agents, and RAG systems. |
| Garak | LLM probing | Open-source LLM scanner for systematic model-behavior probing. |
| PyRIT | AI risk assessment | Microsoft framework for identifying and mitigating generative-AI risks. |
| Purple Llama | Safety resources | Meta tools and research artifacts for trust, safety, and LLM security e. |
| FuzzyAI | Fuzzing | Automated LLM fuzzing for resilience and jailbreak testing. |
| Open Prompt Injection | Benchmark | Benchmark resources for studying prompt injection and defenses. |
| Agentic Radar | Agent security | CLI scanner for issues in agentic workflows. |
| NVIDIA SkillSpector | Skill and extension security | Scanner for agent skills and related supply-chain, permission, prompt-injection, and data-flow risks. Treat optional model-assisted analysis as an explicit data-handling decision.[21] |
| Agent Threat Rules | Detection-rule format | Machine-readable agent-threat detection rules and reference implementations. The upstream format is a working draft; assess maturity and test co before enforcement.[22] |
| OWASP Agent Memory Guard | Memory security | OWASP project for detecting and pre agent memory poisoning. |
| Semgrep AI Best Practices | Secure de | Security rules for code integrating LLM pro, MCP, assistants, and agent frameworks. |
| Giskard | E | Open-source e and testing for AI and LLM systems. |
| DeepTeam | Red teaming | LLM red-team testing for engineering workflows. |

MCP Security

The Model Context Protocol (MCP) standardizes how hosts, clients, and servers expose resources, prompts, and tools to AI systems. That composability is powerful, but tools are security boundaries: they can access data or cause actions, and untrusted tool descriptions must be handled with caution.[5]

| Resource | Type | Description |
|---|---|---|
| MCP Specification | Protocol | Canonical concepts and trust-and-safety principles for resources, prompts, tools, authorization, and user interaction. Pin review to the deployed specification .[5] |
| MCP Security Best Practices | Technical guidance | Covers authorization, consent, token passthrough, SSRF, state handling, and local-ser.[6] |
| MCP Security Bench (MSB) | Research benchmark | End-to-end e of MCP-specific attacks across planning, in, and response handling. Use only in controlled e and report both security and task-performance conditions.[23] |
| OWASP MCP Top 10 | Threat model | OWASP project in beta co token exposure, scope creep, tool poisoning, supply chain, execution, and telemetry.[7] |
| OWASP Third-Party MCP Guide | Consumer guidance | Secure discovery, authentication, authorization, sandboxing, least pri, and human o.[8] |
| MCP Security Checklist | Checklist | Community review aid; its advice against protocol guidance and the local en. |
| Awesome MCP Security | Curated list | Complementary directory of MCP-security resources, tools, research, and guides. |
| Agent Scan (formerly mcp-scan) | Security scanner | Tooling for inspecting MCP ser before adoption. |
| Cisco MCP Scanner | Security scanner | Multi-engine scanner for MCP servers and tools. |

Before connecting a server, establish source pro, transport, identities, requested scopes, filesystem and network access, whether it can read/write/execute, logging behavior, dependency posture, and appro for consequential actions. See the MCP Security Guide.

AI Red Teaming and E

AI red teaming is disciplined testing of models and AI applications under defined rules, controlled data, and measurable outcomes. Its purpose is remediation—not bypassing safeguards in deployed third-party systems.

| Resource | Focus | Description |
|---|---|---|
| Ad Robustness Toolbox | Classical ML | IBM library for ad ML attacks, defenses, and robustness e. |
| Counterfit | Classical ML | Automation layer for assessing security risks in ML systems. |
| Cle | Classical ML | Library for ad examples and defense benchmarking. |
| Foolbox | Classical ML | Robustness-e toolbox across common ML frameworks. |
| TextAttack | NLP robustness | Framework for ad NLP testing, augmentation, and training. |
| AI Red Teaming Playground Labs | Training | Learning materials and labs for AI red teaming. |
| RAMPART | Agentic AI testing | Pytest-nati for safety and security testing of agentic applications. |
| AgentDojo | Agent-security benchmark | Dynamic en for prompt-injection attacks and defenses in tool-using agents. Use in an isolated test en; its API is under acti.[24] |
| AI-Infra-Guard | Platform | AI-security platform with infrastructure, MCP, and jailbreak e components. |

AI for Security

Application Security and Research

The high-value role of AI in AppSec is improving discovery, triage, , reporting, remediation, and learning while keeping people accountable for security decisions. Strong workflows combine model reasoning with deterministic checks, reproducible e, source-code context, and independent .[9] [10]

| Resource | Area | Description |
|---|---|---|
| DARPA AI Cyber Challenge | research | Public effort around cyber reasoning systems for disco and patching in critical open-source software. |
| AIxCC Open Source Archive | Open systems | Competition systems, artifacts, and resources released for study and defensi. |
| AutoPatchBench | Secure repair | Benchmarking for automated repair of fuzzing-detected . |
| VLoc Bench | Localization | Benchmark for repository na, localization, and patch . |
| | CodeQL triage | LLM-assisted triage for workflows. |
| | Code review | Open-source assistance for codebases. |
| Semgrep | Static analysis | Deterministic companion to AI-assisted code review. |
| CodeQL | Code analysis | Semantic code analysis and query ecosystem. |
| OSS-Fuzz | Fuzzing | Continuous fuzzing for critical open-source projects. |
| SARIF | Interoperability | Standard format for static-analysis results and review. |

The AIxCC final reported 54 synthetic found out of 63 and 43 patched, alongside responsibly disclosed non-synthetic findings. The useful lesson is not a headline number: it is the importance of end-to-end , patch quality, and transparent scoring.[11]

Authorized Security Agents

There is no uni “best” security agent. Capability depends on task, target access, permitted tools, scaffold, model, e protocol, and human o. The directory therefore groups resources by role and avoids global performance claims without comparable e.

| Resource | Role | Curation note |
|---|---|---|
| PentAGI | Authorized pentesting agent | Multi-agent system for complex security testing with sandboxing, multiple pro, knowledge-graph integrations, and obser. Review current permissions and deployment boundaries. |
| PentestGPT | Human-guided testing | GPT-empowered assistance for penetration-testing research and authorized workflows. |
| HackingBuddyGPT | Research agent | LLM-assisted security agent and benchmark resources. |
| Shannon | Web and API security | Autonomous application-security project; assess any performance claim in its stated test setting. |
| Strix | Application security | Agents for dynamic code analysis and . |
| Zen-AI-Pentest | Framework | Multi-agent security testing framework; its tool integrations are pri operations. |
| CyberStrikeAI | Platform | AI-native tool orchestration, roles, and lifecycle management. |
| BugTraceAI | Authorized security agent | Acti.0 open-source platform maintained by BugTraceAI for authorized security testing with multi-agent orchestration, independent , e capture, and reporting. Review tool permissions, scope, and isolation before use. |
| OpenHack | Code security | Multi-agent source-code scanner and system. |
| CAI — archived | Historical artifact | The project reports that it is archived and will receive no further fixes or security patches; retain as research context, not an acti. |
| BoxPwnr | Lab agent | Agent resources for controlled security challenges. |
| NYU CTF Agents | Research agents | Agents associated with scalable CTF e research. |
| Cyber Security LLM Agents | Automation examples | LLM-agent examples for common security workflows. |

Strong agent designs separate architecture mapping, hypothesis generation, deterministic scanning, targeted testing, independent , e capture, deduplication, reporting, and remediation. A single agent that discovers and grades its own finding is a weak design.[9] [10]

Threat Intelligence and Security Operations

AI can reduce toil in intelligence gathering, alert triage, detection engineering, and incident response. These workflows should default to read-only access, source citation, data minimization, analyst review, and explicit escalation paths.

| Resource | Area | Description |
|---|---|---|
| MITRE ATT&CK | Threat knowledge | General ad knowledge base that complements ATLAS. |
| OpenCTI | CTI platform | Open platform for structuring and sharing threat knowledge. |
| MISP | Threat sharing | Open-source threat-intelligence sharing platform. |
| Sigma | Detection engineering | Open generic signature format for shared detections. |
| YARA | Pattern matching | Rule-based pattern matching for research and classification. |
| Wazuh | Security operations | Open-source XDR/SIEM platform. |
| Elastic Detection Rules | Detection engineering | Public detection rules and rule-de resources. |
| Splunk Boss of the SOC Dataset | Training | Dataset and en for SOC-in practice. |
| CTI-Bench | Benchmark | LLM benchmark focused on cyber-threat-intelligence tasks. |
| SECURE | Benchmark | Cybersecurity scenario dataset for extraction, understanding, and reasoning. |

Models, Data, and Adaptation

Model specialization is not synonymous with training a model from scratch. RAG adds context at inference time without changing weights. Instruction tuning adapts behavior to instruction-response examples. PEFT/LoRA updates a small parameter subset. Domain-adapti pretraining adds domain knowledge through further pretraining. Distillation transfers behavior from a larger model to a smaller one. The right approach depends on objectives, data, permissions, risk tolerance, and e.[12]

| Resource | Type | Why it belongs here |
|---|---|---|
| Primus | Paper, datasets, models | Cybersecurity resources spanning pretraining, instruction tuning, and reasoning distillation, with ablations and benchmark e.[13] |
| Trend Micro Primus Collection | Models and data | Canonical collection referenced by the Primus paper. |
| Foundation-Sec | Specialized model | Cybersecurity-focused foundation model family; inspect model card, terms, data, and e. |
| Antares Collection | Specialized models | Security-oriented models for on workflows. |
| SecureBERT | Specialized model | BERT-family model for cybersecurity text tasks. |
| Llama-Primus | Specialized model | Cybersecurity model family associated with the Primus work. |
| Less Data, More Security | Research | Empirical study of domain-adapti pretraining with a curated corpus and multiple benchmarks.[12] |
| HackMentor | Historical project | Repository separating data construction, training, and e for a cybersecurity LLM. |
| CyberLLMInstruct safety study | Safety research | Why cybersecurity-oriented fine-tuning needs safety e before and after adaptation.[14] |
| CyberLLMInstruct (ACM AISec 2025) | Dataset and paper | Pseudo-malicious cybersecurity instruction data and an empirical safety-performance trade-off study. Treat it as research material requiring local rights, risk, and pre/post safety review—not as a default training corpus.[25] |
| Model Cards | Documentation | Document intended use, pro, limitations, e, and ethical considerations. |
| Dataset Cards | Documentation | Document source, composition, license, processing, and limitations. |

Any model, dataset, or adapter entry should state its license, source, intended use, limitations, data pro, safety e, benchmark protocol, and review date. The safety study found reduced safety resilience after fine-tuning in its e settings; utility gains must not substitute for safety testing.[14]

Benchmarks, Cyber Ranges, and E

A benchmark score is meaningful only when task, model, agent scaffold, tools, en, trials, feedback channels, budget, and metric are known. Do not compare white-box and black-box results, different numbers of attempts, hidden grader feedback, or contaminated tasks as though they were equi.

| Resource | Focus | Description |
|---|---|---|
| CyBench | Cyber capability | Professional CTF tasks with subtask measurement, metric definitions, logs, and cautions about comparability.[15] |
| CyberGym | Real-world | Large-scale benchmark across real and projects; its abstract reports roughly 20% success even for the best e combinations.[16] |
| NYU CTF Bench | Agent e | Dockerized CTF challenges for controlled LLM-agent e. |
| CyberSecEval | Model e | Cybersecurity capability and safety e resources. |
| SecBench | Security knowledge | Multi-dimensional cybersecurity benchmark dataset. |
| C | Web security | web applications and CVEs for controlled agent e. |
| BountyBench | Research impact | Benchmark for detection, , and patching. |
| AgentCyberRange | Cyber-range research | Open research infrastructure for e agents in isolated multi-host web-exploitation and post-exploitation scenarios. Interpret reported scores only with stated prompts, budgets, and methods.[26] |
| AISI multi-step cyber-range study | Capability research | Controlled study of multi-step cyber-range performance. Its findings show why budgets, repeated runs, containment, and task design need explicit reporting.[27] |
| AI Cyber Model Arena | Agent/model matrix | Wiz Research e across real-world domains in isolated containers; interpret it as a specific methodology.[17] |
| BotsBench | SOC in | E of agents on security-operations in tasks. |
| DefenseBench | Defensi | E of agents on defensi operations. |

MCP Servers for Security Workflows

These are not plug-and-play endorsements. Prefer read-only access, minimal scopes, isolated en, tool calls, and explicit human appro any action that changes a system or sends traffic outside a lab.

| Server or collection | Area | Notes |
|---|---|---|
| PortSwigger Burp Suite MCP Server | Application security | Official integration between Burp Suite and MCP clients; constrain any configuration-changing capabilities. |
| MCP Security Hub | Tool collection | Dockerized collection; its documentation describes non-root containers, dropped capabilities, limits, health checks, and CI scanning. Inspect each ser.[18] |
| mcp-for-security | Community collection | Independently each upstream, package, and permission boundary. |
| OpenCTI MCP Server | Threat intelligence | Begin with a read-only service identity and audit access to intelligence data. |
| Wazuh MCP Server | SOC operations | Separate telemetry queries from response or configuration capabilities. |
| Semgrep MCP | Code security | Integration for static-analysis workflows. |
| radare2 MCP | Binary analysis | Use isolated sample handling and read-only mounts. |
| MCP | Threat intelligence | Protect API credentials and comply with data-sharing terms. |
| MCP Servers | Discovery | Official project collection and references; apply the same review to every server. |

Research and Architecture Case Studies

Case studies are useful when read for design choices and stated limitations, not as independent rankings. The transferable lessons are task decomposition, independent , durable e, model di, controlled execution, and human decision ownership.

| Source | Focus | Lesson |
|---|---|---|
| AWS Security Agent | Multi-agent testing | Baseline co, specialized workers, , structured e, and explicit recognition that plausible findings require rigorous .[9] |
| Cloudflare Harness | Fleet-scale code security | External state, narrow agent context, separate hunting and , deduplication, and reproducible e before triage.[10] |
| Wiz AI Cyber Model Arena | E design | Separate model effects from agent effects, use ground truth, repeat trials, and isolate en.[17] |
| Praetorian C | C | Research, technology correlation, detection design, critique, , and human review as distinct phases.[19] |
| Team Atlanta | AIxCC | Finalist retrospective and implementation-le. |
| Trail of Bits Buttercup | AIxCC | Defensi and remediation case study. |
| Theori RoboDuck | AIxCC | Finalist retrospective and resources. |
| Agent security stack design | Practitioner guidance | Separate beha guidance from the runtime control plane: the agent proposes actions, while identity, policy, egress, and audit controls enforce allowed effects.[28] |
| AISI cyber-e incident report | Safety case study | A controlled-e incident that reinforces explicit egress decisions, real-time monitoring, solvable task scope, and containment that does not depend on model behavior.[29] |

Learning Path

| Stage | Focus | Suggested starting resources | Outcome |
|---|---|---|---|
| 1. Foundations | Application security, ML concepts, threat modeling | OWASP, NIST, NCSC, ATT&CK/ATLAS | for risks, assets, controls, and e. |
| 2. Secure AI applications | LLM, RAG, prompts, tools, and data flows | OWASP GenAI, Promptfoo, Garak, PyRIT | A test plan for an AI application and integrations. |
| 3. Agents and MCP | Permissions, memory, tools, isolation, logs | MCP specification, MCP guidance, OWASP MCP | A least-pri architecture and review checklist. |
| 4. AI-assisted AppSec | Code analysis, fuzzing, , remediation | Semgrep, CodeQL, OSS-Fuzz, AIxCC | Reproducible, re findings. |
| 5. Adaptation | RAG, SFT, PEFT, domain adaptation, safety | Primus, Foundation-Sec, DAP research | A documented retrie decision. |
| 6. E | Benchmarks, ranges, cost, failure analysis | CyBench, CyberGym, VLoc Bench | An e plan with limitations. |
| 7. Operations | Monitoring, go, disclosure, maintenance | NIST, NCSC, OWASP | An auditable program rather than a demo. |

Extended Guides

| Guide | Focus |
|---|---|
| AI Security Guide | Securing LLM, RAG, and agentic applications across the lifecycle. |
| MCP Security Guide | E and adopting MCP servers with secure permissions, identity, isolation, logs, and supply chain. |
| Models, Data, and E Guide | Responsible cybersecurity-model specialization, dataset review, tuning, RAG, benchmarks, and release decisions. |
| Agentic Security Guide | Authorized AI-assisted security workflows with , e, autonomy boundaries, and human review. |
| Benchmarking and E Guide | E cards, comparable results, model-adaptation release gates, MCP-loop testing, cyber-range containment, and e reporting. |
| Curation Policy | Inclusion rules, e standards, review cadence, deprecation, and contributor expectations. |
| Resource Catalog | Normalized metadata, status, e, effects, and adoption boundaries for high-impact resources. |
| Resource Catalog Supplement | Metadata and authorized-use boundaries for the remaining discovery resources promoted in this repository. |
| Catalog Maintenance Guide | Reproducible catalog regeneration, integrity checks, review authority, e requirements, and link-triage procedure. |
| Maintenance Register | review cadence, e to record, current baseline, and retirement rules. |
| Pre-Adoption Review | Required local before operational use of an un, experimental, historical, or high-impact resource. |

Contributing and Curation

Contributions are welcome. A high-quality contribution gi enough information to assess relevance, activity, e, and safe use. Read CONTRIBUTING.md and the Curation Policy before opening an issue or pull request.

Every proposed entry should include a canonical URL, resource type, neutral description, maintainer, license, current status, relevant task or security domain, date, and e for quantitati. Entries in tools, automation, or MCP must explain permissions, data exposure, actions, isolation, and authorized use. Archived, unmaintained, or experimental projects can be historically , but must be labeled prominently.

References

[1]: "OWASP Top 10 for Large Language Model Applications"
[2]: "NIST AI Risk Management Framework"
[3]: "NCSC Secure AI System De"
[4]: "MITRE ATLAS"
[5]: "MCP Specification"
[6]: "MCP Security Best Practices"
[7]: "OWASP MCP Top 10"
[8]: "OWASP Third-Party MCP Guide"
[9]: "AWS Security Agent"
[10]: "Cloudflare Harness"
[11]: "AI Cyber Challenge Final Competition Winners Announcement"
[12]: "Less Data, More Security"
[13]: "Primus"
[14]: "CyberLLMInstruct Safety Study"
[15]: "CyBench"
[16]: "CyberGym"
[17]: "Wiz AI Cyber Model Arena"
[18]: "MCP Security Hub"
[19]: "Praetorian C"
[20]: "OWASP AI Security Standard"
[21]: "NVIDIA SkillSpector"
[22]: "Agent Threat Rules"
[23]: "MCP Security Bench"
[24]: "AgentDojo"
[25]: "CyberLLMInstruct"
[26]: "AgentCyberRange"
[27]: "Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios"
[28]: "Where Security Fits in an AI Agent Stack"
[29]: "Incident Report: unsanctioned agent behaviour during cyber testing"

Content review policy

See docs/content-re.md.

项目信息

开发语言：Python

Star 数：28

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕A curated, evidence-led guide to AI for security and security for AI. Defensive research, agent safety, MCP security, AI red-teaming and evaluation, authorized testing workflows, and detection engineering.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。