basilisk - Basilisk — Open-source AI red teaming framework wi

项目说明

Basilisk - Open-Source AI Red Teaming Framework

> Basilisk is an open-source AI red teaming and LLM security testing framework. It automates ad prompt testing against Claude, Gemini, Grok, GPT-family models, local model runtimes, and custom LLM APIs using e prompt search. Built for security researchers, penetration testers, and defensi that need repeatable ad testing against LLM applications.

> 📄 Research Paper:
> 📌 DOI:

Basilisk combines structured attack modules, e prompt search, typed e bundles, signed audit logs, and exportable reports for repeatable LLM security testing workflows.

---

<di="center">
<img src="assets/demo.gif" alt="Basilisk AI Red Teaming Demo - Genetic Prompt E Dashboard" style="border-radius: 12px; margin: 20px 0; max-width: 100%; border: 1px solid #1f1f27;" />
Basilisk v2.0.2 -- Automated LLM Security Testing with Genetic Prompt E
<a href="
<img src=" alt="Basilisk YouTube Demo" />
</a>
</div>

What the Demo Shows

* Genetic Prompt E: Watch the mutation engine iterate through payload families across generations.
* Differential Mode: Compare behavior across multiple LLM pro and model targets.
* Guardrail Posture Scan: Run recon-oriented posture grading without a full exploit pass.
* Real-Time Scan Dashboard: Review li, e stats, and operator telemetry.
* Report Export: Export results as HTML, JSON, SARIF, Markdown, and PDF.

<a href="#what-is-basilisk">What is Basilisk?</a> •
<a href="#research-context">Research Context</a> •
<a href="#quick-start">Quick Start</a> •
<a href="#features">Features</a> •
<a href="#whats-new-in-v200">What's New</a> •
<a href="#attack-modules">Attack Modules</a> •
<a href="#desktop-app">Desktop App</a> •
<a href="#ci-cd-integration">CI/CD</a> •
<a href="#docker">Docker</a> •
<a href="

---

What is Basilisk?

Basilisk is an open-source offensi framework purpose-built for AI red teaming and LLM penetration testing. It maps its attack surface to the OWASP LLM Top 10 threat model and combines that co with a genetic algorithm engine called Smart Prompt E (SPE-NL) that e ad prompt payloads across generations.

Whether you are testing OpenAI GPT-4o, Anthropic Claude, Google Gemini, NVIDIA API Catalog, xAI Grok, Meta Llama, or a custom LLM endpoint, Basilisk provides 33 attack modules, 5 recon modules, differential multi-model scanning, guardrail posture grading, and signed audit logging out of the box.

v2.0.2 adds deterministic CLI/desktop parity testing, fixes Linux Electron backend startup, and keeps the v2 runtime, e, and secret-handling hardening.

Why Basilisk?

Automated AI Red Teaming: Start from probes and attack modules, then let Basilisk iterate through payload automatically.

Genetic Prompt E: The SPE-NL engine mutates, crosses over, and scores prompts across generations. When a static payload gets refused, e searches for nearby that preser while changing delivery.

OWASP-Aligned Attack Co: 33 modules co prompt injection, system prompt extraction, data exfiltration, tool abuse, guardrail bypass, denial of service, multi-turn manipulation, RAG attacks, and multimodal injection.

E Findings: Every finding carries structured e signals, replay steps, and calibrated . Production-tier findings get downgraded if the proof is weak. No more "the model said something bad" as your entire e chain.

Broad Provider Support: OpenAI, Anthropic, Google, NVIDIA API Catalog, xAI (Grok), Groq, Azure, AWS Bedrock, GitHub Models, Ollama, vLLM, and custom HTTP/WebSocket endpoints.

CI/CD Ready: Nati Action with SARIF output and baseline regression detection for automated AI security testing in your pipeline.

Desktop App: Electron GUI for red teaming with live scan dashboards, campaign controls, and report export.

Built by Regaan, Lead Researcher at ROT Independent Security Research Lab, and creator of WSHawk.

Website: basilisk.rothackers.com

---

Research Context

Basilisk is described in the following research paper:

Regaan R. *Basilisk: An E AI Red-Teaming Framework for Systematic Security E of Large Language Models*. SSRN Electronic Journal, 2026.

The paper formalizes:

E prompt search (SPE-NL)

Multi-objecti scoring

Beha space exploration

Differential model e

This repository contains the full implementation used for experimental e.

---

Key Contributions

Introduces e prompt search (SPE-NL) for LLM security testing

Demonstrates multi-objecti scoring for ad prompts

Enables beha space exploration beyond static jailbreaks

Provides reproducible e framework across multiple LLM pro

Quick Start

Zero-Setup Live Demo

Want to see Basilisk in action without configuring API keys? We maintain an intentionally LLM target for testing:

Target URL:

Or use the Desktop App: open New Scan, set the endpoint URL above, pick Custom HTTP as the provider, and hit start.

Docker

---

Reproduce Results

Run a minimal experiment from the paper:

This reproduces:

E prompt search

Guardrail bypass discovery

Beha di patterns

---

Experimental Setup

Models tested: GPT-4o, Claude 3.5, Gemini 2.0

Attack modules: 33 (OWASP LLM Top 10 aligned)

E metrics:

- Refusal bypass rate
- Leakage success rate
- Beha di score
- E con rate

---

Features

Smart Prompt E (SPE-NL)

The core differentiator. Genetic algorithms adapted for natural language attack payloads:

10 mutation operators -- synonym swap, encoding wrap, role injection, language shift, structure o, fragment split, homoglyph substitution, context padding, token smuggling, LLM-dri

5 crosso -- single-point, uniform, prefix-suffix, semantic blend, best-of-both

Multi-objecti -- refusal a, information leakage, compliance scoring, target pattern matching, cost efficiency, intent preser, reproducibility, and novelty. NSGA-II Pareto ranking when multiple objectives are active.

Curiosity-dri -- beha space partitioning with TF-IDF clustering and adaptive bin splitting. Rewards mutations that land in under- response regions instead of repeating the same refusal pattern.

Intent preser -- tracks semantic drift from original seed payloads across generations. Mutations that stray too far from the attack goal get penalized.

Operator learning -- multi-armed bandit selects which mutation operators work best against the current target

Stagnation detection with adapti rate, early breakthrough exit, and population deduplication

Response cache -- SHA-256 keyed LRU cache avoids burning API tokens on duplicate payloads

Payloads that fail get mutated, crossed, and re-e. The search loop keeps useful , remo, and shifts pressure when the target response distribution stagnates.

33 Attack Modules

33 modules across 9 attack categories with 3 trust tiers, mapped to the OWASP LLM Top 10 threat model. See Attack Modules below.

Trust Tiers and E Policy

Not every module is treated the same. Basilisk assigns each module a trust tier:

production (11 modules) -- strictest e requirements, included by default

beta (18 modules) -- included by default but held to a lower proof standard

research (4 modules) -- excluded unless you explicitly opt in

High and critical findings from production modules get checked against module-specific proof requirements. If the e is weak, the finding is downgraded and the downgrade reason is preserved in session data and reports. This is how Basilisk separates "interesting behavior" from "defensible e."

5 Reconnaissance Modules

Model Fingerprinting -- identifies GPT-4, Claude, Gemini, Llama, Mistral patterns and timing

Guardrail Profiling -- systematic probing across 8 content categories

Tool/Function Discovery -- enumerates a tools and API schemas

Context Window Measurement -- determines token limits

RAG Pipeline Detection -- identifies retrie generation setups

Differential Testing

Run identical probes against multiple pro or model targets and compare where one path refuses while another complies. Basilisk records per-model outcomes and per-probe breakdowns for side-by-side review.

Guardrail Posture Assessment

Recon-only guardrail grading:

A+ through F posture grades

8 categories with 3-tier probing (benign/moderate/ad)

Strength classification: None, Weak, Moderate, Strong, Aggressive

Actionable recommendations for weak spots and o

E

When you already know the exact prompts and assertions you need, skip the exploratory scan and run a deterministic e:

Assertion types: must_refuse, must_not_refuse, must_contain, must_not_contain, max_compliance, max_tokens, regex_match, regex_no_match, similarity, llm_grade

Output in console, JSON, JUnit XML, or Markdown

Diff against previous results for regression detection

Tag-based filtering and parallel execution

Probe Corpus

263 probes in YAML across 9 categories. Single prompts, ordered multi-turn sequences, and fabricated role/content histories use the same loader. Every probe carries an OWASP mapping; most include expected signals, and many include tags. The loader also supports richer optional metadata such as objectives, preconditions, failure modes, target archetypes, tool requirements, and follow-up probe IDs as the corpus e. The corpus seeds e runs and feeds the effecti tracker.

Audit Logging

On by default. Ed25519-signed JSONL with SHA-256 chain integrity:

Scan configuration, campaign policy, findings, errors, e, recon, and report events are logged

Prompt and response event methods are a for integrations that explicitly enable those hooks

API keys automatically redacted before writing

Desktop master keys protected with Electron safeStorage when a secure operating-system backend is a

CLI scans executed in restricted child workers with mode-specific OS resource ceilings

Audit key resolution: key file, encrypted secret store, or legacy en

Export the signing trust anchor once with basilisk audit-trust-export audit-public.key, store it independently, then files with basilisk audit- PATH --public-key-file audit-public.key

basilisk audit- PATH --allow-embedded-key checks integrity only and does not establish external authenticity

5 Report Formats

| Format | Use Case |
|--------|----------|
| HTML | Dark-themed report with expandable findings, con replay, se charts |
| SARIF 2.1.0 | CI/CD integration -- GitHub Code Scanning, DefectDojo, Azure DevOps |
| JSON | Machine-readable with full metadata |
| Markdown | Documentation-ready, commit-friendly |
| PDF | Offline sharing using weasyprint, reportlab, or the built-in basic PDF renderer |

Campaign and Policy Controls

Scans carry operator context and execution policy:

Campaign metadata: operator name, ticket ID, target owner, objecti

Execution modes: recon, , exploit_chain, research

E thresholds, retention windows, module allow/deny lists

Appro and dry-run planning

All of this lands in session state and reports -- not decorati

Uni Provider Support

Via litellm + custom adapters:

Cloud -- OpenAI, Anthropic, Google, NVIDIA API Catalog, xAI (Grok), Groq, Azure, AWS Bedrock

GitHub Models -- free access to GPT-4o, o1, and more via github.com/marketplace/models

Local -- Ollama, vLLM, llama.cpp

Custom -- any HTTP REST API or WebSocket endpoint

WSHawk -- pairs with WSHawk for WebSocket-based AI testing

Electron Desktop App

Full desktop GUI with:

Live scan via the desktop event bridge

Campaign control plane with operator, ticket, and policy fields

Differential scan tab with multi-model comparison

Guardrail posture tab with li

Audit trail with integrity

Module browser with OWASP mapping and trust tier display

Probe explorer with filtering and stats

E with assertion-dri

Session management with replay and e review

Report export dialog for HTML, JSON, SARIF, and PDF

Cross-platform: Windows (.exe), macOS (.dmg), Linux (.AppImage/.deb/.rpm/.pacman)

Native C/Go Extensions

Performance-critical paths compiled to shared libraries with full Python fallbacks:

C -- BPE token estimation, Shannon entropy, Le distance, confusable detection, payload encoding (base64, hex, ROT13, URL, Unicode escape)

Go -- concurrent mutation operators (including 4 multi-turn aware), crosso, batch mutation, population di scoring, Aho-Corasick multi-pattern matching, refusal/compliance/sensitive data detection

Nati load only after an Ed25519-signed manifest and SHA-256 hashes . Unsigned or mismatched libraries are rejected and Basilisk uses its Python fallback.

---

What's New in v2.0.2

Security Hardening

v2.0.2 includes the v2 runtime hardening plus release, benchmark, and desktop reliability fixes:

Ed25519 Signed Nati -- Nati are loaded only after their SHA-256 hashes against an Ed25519-signed manifest.

Ed25519 Audit Log Signatures -- Every audit log entry is digitally signed. Legacy HMAC-SHA256 is deprecated. Audit key resolution prefers encrypted local storage o .

CLI Secret Rejection -- inline --api-key, --attacker-api-key, and --auth are rejected. Credentials must come from en , encrypted desktop settings, or references, keeping them out of shell history and process listings.

SQLite Worker Architecture -- Replaced ad-hoc locking with a dedicated worker thread per database path. WAL mode, async queue, and single-writer semantics reduce e contention.

Nati Input Guardrails -- FFI calls enforce input size limits. O Le or similarity calls are rejected before reaching native code.

Path Tra Protection -- Config file loading uses Path.resolve() and ancestry checks against a safe root allowlist. String-prefix matching is gone.

LLM Mutation Isolation -- The attacker-model mutation prompt wraps payloads in JSON and explicitly treats embedded payload content as untrusted data.

Go Fuzzer Crypto Seeding -- PRNG seeded from crypto/rand instead of time.Now(). Mutation sequences are no longer predictable.

Retention Timestamp Hardening -- Artifact pruning prefers embedded UTC timestamps o mtime, which is tri spoofable.

Desktop Context Isolation -- contextIsolation: true, nodeIntegration: false, sandbox: true, webview and na blocked, permission requests denied, external URLs allowlisted, backend requests gated through the desktop request bridge.

E Findings

Findings now carry structured E objects with typed signals, replay steps, calibrated , and per-module proof requirements. The e policy engine can downgrade se when proof is insufficient.

Curiosity-Dri

The e engine now partitions the response space into beha clusters and rewards mutations that land in sparse, under- regions. Adaptive bin splitting helps prevent curiosity collapse. The novelty signal combines visit frequency, semantic novelty, and beha novelty.

Multi-Objecti

NSGA-II Pareto ranking with crowding distance. The fitness function e exploit e, target-signal match, refusal a, novelty, intent preser, reproducibility, and cost efficiency as separate objectives when multi-objective mode is active.

Probe Effecti Tracking

SQLite-backed tracker at ~/.basilisk/probe_effecti.db. Records which probes worked against which models, categories, and archetypes. Computes bypass rates, block rates, and historical trends. Feeds back into probe prioritization.

E

Deterministic assertion-driven test harness. YAML config with typed assertions (must_refuse, must_contain, regex_match, similarity, llm_grade, etc.). Console, JSON, JUnit, and Markdown output. Diff against previous results for regression detection. Parallel execution. Tag-based filtering.

Previous Releases

See the GitHub releases for previous release notes and artifacts.

---

Attack Modules

| Category | Modules | OWASP | What They Test |
|----------|---------|-------|----------------|
| Prompt Injection | Direct, Indirect, Multilingual, Encoding, Split | LLM01 | O system instructions via user input |
| System Prompt Extraction | Role Confusion, Translation, Simulation, Gradient Walk | LLM06 | Extract confidential system prompts and policies |
| Data Exfiltration | Training Data, RAG Data, Tool Schema | LLM06 | Extract PII, documents, and API schemas |
| Tool/Function Abuse | SSRF, SQLi, Command Injection, Chained | LLM07/08 | Exploit tool-use capabilities for lateral mo |
| Guardrail Bypass | Roleplay, Encoding, Logic Trap, Systematic | LLM01/09 | Circumvent content safety filters |
| Denial of Service | Token Exhaustion, Context Bomb, Loop Trigger | LLM04 | Resource exhaustion and infinite loops |
| Multi-Turn | Culti, Authority Escalation, Escalation, Persona Lock, Sycophancy, Memory Manipulation | LLM01 | Progressi exploitation and con drift |
| RAG Attacks | Poisoning, Document Injection, Knowledge Enumeration | LLM03/06 | Compromise retrie generation pipelines |
| Multimodal | Image+Text Injection | LLM01 | Combined image and text attack paths |

Trust tier split: 11 production / 18 beta / 4 research. Research modules are excluded by default.

---

Scan Modes

| Mode | Description | E | Speed |
|------|-------------|-----------|-------|
| quick | Top payloads per module, no e | No | Fast |
| standard | Full payloads, 5 generations of e | Yes | Normal |
| deep | Full payloads, 10+ generations, multi-turn chains | Yes | Slow |
| stealth | Rate-limited, human-like timing delays | Yes | Careful |
| chaos | E enabled, maximum e pressure | Yes | Aggressive |

Scan mode and execution mode are separate controls. Scan mode shapes runtime behavior. Execution mode (recon, , exploit_chain, research) shapes the operational policy and e requirements. They are orthogonal.

---

CLI Reference

Full CLI Reference.

---

Configuration

---

CI/CD Integration

GitHub Actions (Nati)

Using GitHub Models (free, no API key purchase needed):

> Tip: Create a personal access token with models:read permission and save it as GH_MODELS_TOKEN.

Required Secrets:

| Secret | Provider | When Needed |
|--------|----------|-------------|
| OPENAI_API_KEY | OpenAI | If scanning with OpenAI |
| ANTHROPIC_API_KEY | Anthropic | If scanning with Anthropic |
| GOOGLE_API_KEY | Google | If scanning with Google |
| N | NVIDIA API Catalog | If scanning an NVIDIA hosted model |
| XAI_API_KEY | xAI | If scanning with Grok |
| GH_MODELS_TOKEN | GitHub Models | Free access to GPT-4o, o1, etc. |

GitHub Actions (Manual)

GitLab CI

---

Desktop App

The Electron desktop app wraps the same scan runtime as the CLI in a full GUI.

For production builds (standalone, no Python install needed -- backend compiled ):

Output in desktop/dist/.

---

Architecture

---

Documentation

| Document | What It Covers |
|----------|----------------|
| CLI Beginner Guide | First scan, basic commands, reading results |
| Desktop Beginner Guide | First desktop scan, UI walkthrough, report export |
| CLI Ad Guide | Campaign controls, module selection, e policy, ad workflows |
| Desktop Ad Guide | Campaign control plane, module catalog, session review, operator workflows |
| Architecture | System design, module breakdown, data flow |
| CLI Reference | All commands and options |
| Attack Modules | Module catalog with trust tiers and e requirements |
| Eval, Probes, and Curiosity | Probe corpus, e, curiosity steering, effecti tracking |
| Reporting | Report formats and CI/CD integration |
| API Reference | Desktop backend API endpoints |
| Security and Release | Roadmap controls, test e, benchmarks, and release gates |
| Contributing | De setup, PR process, coding standards |
| Security Policy | disclosure with SLAs |

---

FAQ

What is Basilisk used for?

Basilisk is used for automated AI red teaming and LLM security testing. It is designed to exercise prompt injection, jailbreak, data leakage, tool-abuse, RAG, and guardrail bypass paths in AI applications built on hosted or local large language models.

How does it compare to manual jailbreak testing?

Manual testing stops at static payloads. Basilisk starts there and then e them. When a payload gets refused, the mutation engine generates by changing structure, encoding, framing, and composition while keeping the same objective. O generations, the search pressure mo payload families that better match the current target.

Does Basilisk work with local models?

Yes. Basilisk supports Ollama, vLLM, llama.cpp, and any custom HTTP or WebSocket endpoint. You can test self-hosted Llama, Mistral, Qwen, or any open-weight model.

Can I use Basilisk in CI/CD?

Yes. Basilisk ships a nati Action (regaan/basilisk) and supports SARIF output for GitHub Code Scanning, DefectDojo, and Azure DevOps. Baseline regression detection is built in -- your pipeline fails when new findings appear that were not in the previous baseline.

What does the trust tier system do?

It controls e expectations. A production-tier finding must carry structured e signals and module-specific proof markers. If those are missing, the finding gets downgraded automatically. Beta and research tiers ha looser requirements. This is how Basilisk separates "the model said something weird" from "here is a reproducible with replay steps."

Is Basilisk free?

Fully open-source under AGPL-3.0. No restrictions on authorized security testing use.

---

About the Creator

Basilisk is built by Regaan, Lead Researcher at the ROT Independent Security Research Lab.

---

Citation

If you reference Basilisk in research or publications:

Archived at:

SSRN:

Zenodo:

Figshare:

OSF:

Legal

Basilisk is designed for authorized security testing only. Obtain proper written authorization before testing AI systems you do not own. Unauthorized use may computer fraud and abuse laws in your jurisdiction.

The authors assume no liability for misuse.

License

AGPL-3.0 -- see LICENSE.

---

Built by <a href=" -- Founder of Rot Hackers | <a href="

项目信息

开发语言：Python

Star 数：31

技术要点

该项目属于安全软件方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Basilisk — Open-source AI red teaming framework with genetic prompt evolution. Automated LLM security testing for GPT-4, Claude, Grok, Gemini. OWASP LLM Top 10 coverage. 32 attack modules.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。

登录后才能参与讨论

下载链接

会员才能下载

登录后开通会员

同分类推荐

暂无同类资源

资源标题	basilisk - Basilisk — Open-source AI red teaming framework wi
所属分类	安全软件
发布作者	admin
发布时间	2026-10-05 22:49:54
下载权限	免费下载
附件数量	2 个下载链接
浏览次数	351
讨论数量	0
主题状态	正常讨论中