Awesome-LLM-Red-Teaming - A curated list of awesome LLM Red Teaming training

项目说明

Awesome LLM Red Teaming

A comprehensive list of tools and resources for red-teaming large language models (LLMs), research, and reverse engineering using techniques like prompt manipulation, ad testing, and model interpretation.

<di="center">

![Awesome]()
!GitHub contributors
!GitHub Repo stars
!GitHub watchers
!GitHub last commit

</div>

LLM Red Teaming Resources by Technical Function

Playgrounds and practice targets

Folly - Open-source LLM prompt injection and jailbreak playground for hands-on experimentation.

Damn - Intentionally maths LLM app for practising prompt injection and logic exploits.

Damn - Intentionally shopping assistant scenario for testing data leakage and unsafe tool use.

RedTeam Arena - Competiti challenges that train you to bypass model guardrails under time pressure.

MyLLM Bank - WithSecure CTF where you exploit a multi-LLM banking app to learn real-world prompt injection chains.

MyLLM Doctor - WithSecure medical chatbot challenge showing multi-stage prompt injection and policy e.

Gandalf by Lakera - Public challenge where you try to extract a secret from an aligned model despite protections.

Red teaming frameworks and agent harnesses

PyRIT - Microsoft framework for systematic, repeatable red teaming of generative AI systems with automation support.

Rigging - Lightweight agent and tool-calling harness for orchestrating offensive LLM experiments in controlled en.

Attack generation and jailbreak toolkits

BrokenHill - Implementation of the Greedy Coordinate Gradient attack that automatically crafts ad prompts.

llm-attacks - Research code for uni ad suffixes that transfer across aligned LLMs.

Spikee - Simple prompt injection toolkit for probing and exploiting LLM applications.

Obfuscation and payload encoding utilities

In Unicode Tags Playground - Converts text to in Unicode tags to smuggle hidden instructions past filters.

Emoji Selector Playground - Hides data in emoji selectors to evade naï checks.

ASCII to Unicode Character Reducer - Re-encodes ASCII as compact Unicode lookalikes to defeat brittle filters.

Traffic interception and support tooling

LLM-itM - Man-in-the-middle proxy for OpenAI-compatible APIs that lets you inspect and modify prompts and responses.

Defences, standards and guardrails to test against

Anthropic Constitutional Classifiers - Rule-based alignment classifiers that red teamers can e and attempt to bypass.

Prompt Guard 86M - Small model for detecting unsafe prompts that can be assessed and circum.

OWASP Top 10 for LLMs - Community guidance on common LLM application risks and mitigations.

Research papers and technical articles

Best-of-N Jailbreaks - Black-box strategy that samples many prompts and selects those most likely to bypass guardrails.

Uni and Transferable Attacks - Demonstrates uni ad suffixes that elicit unsafe outputs across models.

AutoDAN - Dual-LLM method that crafts gradient-inspired ad prompts to defeat filters.

Faster-GCG - Efficiency impro that speed up discrete optimisation for jailbreak prompts.

GenBFA - Shows bit-flip weight corruption can induce malicious behaviour in generati.

Manipulating LLM Recommendations - Examines how ad prompts can game LLM-dri.

GCG on OpenReview - Peer-review discussion of coordinate gradient jailbreak optimisation.

GCG explainer by Nightfall - Plain-language o of GCG and why it works.

BrokenHill o by Bishop Fox - Practitioner write-up on applying GCG in the wild.

ACG by Haize Labs - Introduces an accelerated that dramatically reduces attack time.

Case studies and exploitation write-ups

Microsoft on e guardrail attacks - Details jailbreak patterns observed in the wild and defensi.

Lessons from red teaming 100 AI products - Summarises lessons from large-scale AI product red teaming.

Multi-chain prompt injection attacks - Explains chaining attacks across stages in complex LLM systems.

Context pollution and delayed tool use - Shows how polluting context can trigger dangerous automated tool use later.

Terminal ANSI sequence abuse - Demonstrates how hidden escape codes in outputs can lead to local code execution.

Air Canada chatbot case - Real-world legal impact when an AI system produces binding misinformation.

Bug bounties and programmes

Microsoft AI Bounty - Official programme that accepts AI reports with rewards.

0DIN AI - Platform focused on GenAI bug bounties and e.

Contributing

Your contributions are always welcome! Please read the contribution guidelines first. We follow the Contributor Co Code of Conduct. Please make sure to review and adhere to this code of conduct when contributing.

License <a href="LICENSE">!GitHub</a>

This project is licensed under the MIT License - see the LICENSE.md file for details.

项目信息

开发语言：未知

Star 数：136

技术要点

该项目属于红蓝攻防方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕A curated list of awesome LLM Red Teaming training, resources, and tools.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。

登录后才能参与讨论

下载链接

会员才能下载

登录后开通会员

同分类推荐

暂无同类资源

资源标题	Awesome-LLM-Red-Teaming - A curated list of awesome LLM Red Teaming training
所属分类	红蓝攻防
发布作者	admin
发布时间	2026-10-06 00:50:27
下载权限	免费下载
附件数量	2 个下载链接
浏览次数	361
讨论数量	0
主题状态	正常讨论中

Awesome-LLM-Red-Teaming - A curated list of awesome LLM Red Teaming training

项目说明