OpenRT - Open-source red teaming framework for MLLMs with 4

项目说明

<di="center">

<img src="docs/materials/icon_opnert.png" alt="OpenRT Logo" width="200">

An Open-Source Red Teaming Framework for Multimodal LLMs

![arXiv]()
![Project Page]()
![GitHub Stars]()

</div>

🚀 What's New <a><img width="35" height="20" src="

[2026.07.17] We released OpenART, an open-source framework for agent red teaming under ad e en! 🔥🔥🔥

\[2026.03.16\] We added 5 new attack methods (2 white-box + 3 black-box): ASA, Imperceptible Jailbreak, Past Tense, LAA, and ArtPrompt. Check out the examples under example/ for quick start!

\[2026.01.05\] OpenRT is released! An open-source red teaming framework for MLLMs with 37+ attack methods. Please check our paper and project page for more details! 🎉🎉🎉

Features

🔧 Modular Architecture: Plugin-based component registry with flexible composition

🎯 42+ Attack Methods: Co both black-box and white-box attacks

🖼️ Multi-modal Support: Text and image attack

📊 Comprehensi: Keyword matching and LLM Judge e

⚙️ Configuration-Driven: YAML config files for experiment definition

Quick Start

Installation

Configure API

Run Examples

Project Structure

Attack Methods

White-Box
> GCG · Jailbreak · ASA · Imperceptible Jailbreak

Black-Box: Optimization & Fuzzing
> AutoDAN · GPTFuzzer · TreeAttack · SeqAR · RACE · AutoDAN-R · LAA

Black-Box: LLM-dri
> PAIR · ReNeLLM · DrAttack · AutoDAN-Turbo

Black-Box: Linguistic & Encoding
> CipherChat · CodeAttack · Multilingual · Jailbroken · ICA · FlipAttack · Mousetrap · Prefill · Past Tense · ArtPrompt

Black-Box: Contextual Deception
> DeepInception · Crescendo · RedQueen · CoA

Black-Box: Multimodal
> FigStep · Query-Relevant · IDEATOR · MML · HADES · HIMRD · JOOD · SI · CS-DJ

Black-Box: Multi-Agent
> ActorAttack · Rainbow Teaming · X-Teaming · E

Configuration Example

Adding New Attacks

E

Standard E with YAML Configs

Ad Batch E (eval.py)

The eval.py script provides a powerful command-line interface for running batch e across multiple models and attack methods.

#### Basic Usage

#### Command-Line Arguments

Model Configuration:

--attacker-model (str, default: "deepseek-v3.2"): Model used for generating attack prompts

--judge-model (str, default: "gpt-4o-mini"): Model used for e attack success

--embedding-model (str, default: "text-embedding-3-large"): Model for generating embeddings

--target-models (list, default: ["baidu/ERNIE-4.5-300B-A47B", "MiniMax-M2", "Qwen/Qwen3-235B-A22B-Thinking-2507"]): Target models to attack

API Configuration:

--api-key (str, en): OpenAI API key

--base-url (str, en): Custom OpenAI-compatible API base URL

Model Parameters:

--attacker-temperature (float, default: 1.0): Temperature for attacker model

--judge-temperature (float, default: 0.0): Temperature for judge model (0.0 for deterministic e)

Execution Options:

--max-workers (int, default: 50): Maximum parallel workers for attack execution

--e (int, default: 32): Maximum workers for e

Attack Methods:

--attacks (list, default: ["AutoDANTurboR", "HIMRD", "JOOD"]): Attack methods to run. A options:

- ActorAttack: Multi-agent coordination attack
- AutoDAN: Hierarchical genetic algorithm attack
- AutoDANTurbo: Enhanced AutoDAN with turbo optimization
- AutoDANTurboR: Hierarchical genetic algorithm with turbo optimization
- CipherChat: Cipher-based obfuscation attack
- CoA: Chain-of-action attack
- CodeAttack: Code-style transformation attack
- Crescendo: Progressi attack
- CSDJ: Composite semantic decomposition jailbreak
- DeepInception: Multi-layered role-playing attack
- DrAttack: Automated prompt engineering attack
- E: Code-le synthesis attack
- FigStep: Figure-based stepping stone attack
- FlipAttack: Polarity flipping attack
- GPTFuzzer: Mutation-based fuzzing attack
- HADES: amplification attack
- HIMRD: Hierarchical multi-turn red-teaming with image generation
- ICA: In-context attack
- Ideator: Iterati thinking attack with image generation
- JailBroken: Template-based jailbreak
- JOOD: Just-in-time ad prompts with image mixing
- MML: Cross-modal encryption attack
- Mousetrap: Prompt injection attack
- Multilingual: Cross-language attack
- PAIR: Prompt automatic iterati
- Prefill: Pre-filled context attack
- QueryRelevant: Query-relevant attack with diffusion models
- RACE: Multi-round ad refinement
- RainbowTeaming: Diverse agent strategy attack
- RedQueen: Adapti transformation attack
- ReNeLLM: Neural-guided prompt optimization
- SeqAR: Sequential ad refinement
- SI: Shuffle inconsistency optimization attack
- TreeAttack: Tree-structured prompt e
- XTeaming: Multi-agent coordination attack

Output & Control:

--results-dir (str, default: "results/baseline_vlm"): Base directory for storing results

--dataset (str, default: "harmbench"): Dataset name (loads from data/{dataset}.csv)

--reload-existing (default: True): Reload existing results instead of skipping

#### Examples

Example 1: Custom Model Configuration

Example 2: Run Only Specific Attacks

Example 3: Custom API Endpoint

Example 4: Disable Result Reloading

Example 5: Complete Custom Setup

#### Result Structure

Results are organized as follows:

Each run creates timestamped directories for reproducibility and includes:

Metrics: Attack success rates and e scores

Logs: Complete attack execution logs with con history

Errors: Error details for failed attacks

#### Datasets

Place CS in the data/ directory with a 'Goal' column containing test prompts:

If the specified dataset is not found, the script will fall back to sample queries.

Acknowledgements

This project integrates a collection of attack methods proposed by prior research and the open-source community. We sincerely thank the original authors and maintainers for making their work publicly a and for their contributions. Please refer to the cited papers and respecti for full details and credits.

Citation

项目信息

开发语言：Python

Star 数：276

技术要点

该项目属于红蓝攻防方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Open-source red teaming framework for MLLMs with 42+ attack methods展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。