VectorGuard - Open-source defensive security testing toolkit for

项目说明

![ CI]()
![Python 3.11+]()
![Linter: ruff]()
![OWASP LLM Top 10]()

is an open-source defensi testing toolkit for LLM, RAG, and AI-agent applications, mapped to the OWASP LLM Top 10. It offers several ways to test, from a fixed regression harness to a fully autonomous attacker:

1. YAML attack suites — repeatable, scripted black-box tests run against OpenAI-compatible chat endpoints or generic HTTP chatbot APIs, e with configurable detectors and turned into JSON/Markdown reports (pass/fail, risk scores, detector e, model responses, latency, transcripts).
2. Autonomous red-team agent — an attacker that pursues OWASP objectives on its own, adapts its tactics based on how the target responds, and escalates until it captures real proof of a . It is *adaptive, not a script replayer*, and *proof-based, not a heuristic alerter*.
3. Web Agent — a bounded, GET-only agent for authorized OWASP-style web application testing.
4. Black-box agent — point-and-shoot testing of a chatbot given *just a URL*: it auto-detects the API shape and runs the autonomous LLM-security battery (prompt injection, unbounded consumption, sensiti, system-prompt leakage) with an optional LLM-dri and single- or multi-turn modes, reporting findings with honest confidence bands.

also includes a local RAG scan mode that loads documents from disk, chunks them, retrie context, builds a RAG-style prompt, and tests whether the target follows malicious retrie.

> Status: adds a point-and-shoot Black-Box Agent (LLM-driven, single- and multi-turn) and a sandbox excessi lab, alongside the Autonomous LLM Red-Team Agent
> is a defensi aid for systems you own or are explicitly authorized to test. Passing tests does not prove that an AI system is secure, and failing tests should be treated as signals for further review.

---

Autonomous LLM Red-Team Agent

includes an autonomous red-team agent that pentests a chatbot on
its own. Unlike the YAML suites (which replay fixed prompts), the agent pursues
objectives and adapts. For each objective it runs a bounded
*observe → act → reflect* loop until it captures proof or exhausts its budget:

1. Propose — an attacker "operator" generates the next attack payload.
2. Execute — deterministic Python sends it and measures the response.
3. Capture — a deterministic oracle decides whether real proof was
obtained (a planted flag, a credential matched by DLP/entropy, or a measured
token/latency budget breach), with an optional LLM-as-judge to catch
paraphrased or encoded wins.
4. Reflect — an analyst scores progress, extracts reusable intel, and
recommends the next tactic, which feeds back into the next payload.

This is what makes it an *attacker, not a scanner*: it keeps maneu against
a single goal — escalating through tactics like persona/role-play, instruction
o, encoding, payload splitting, and context flooding — and every finding
ships with the exact con that produced it as the reproduction
artifact.

Objectives (OWASP LLM Top 10):

| Objective | OWASP | Proof of capture |
|---|---|---|
| system_prompt_leak | LLM07 | a planted system-prompt marker is recovered |
| credential_exfil | LLM02 | a planted secret, or any DLP/entropy-detected credential, is leaked |
| prompt_injection_obey | LLM01 | the model follows an injected instruction (canary or judge-confirmed) |
| unbounded_consumption | LLM10 | output exceeds a token / length / latency budget |

Safety-first by design (same contract as the rest of ): the LLM
only *plans and generates*; deterministic Python *executes* and *judges capture*
— the model ne its own win. This milestone is talk-only (no tool
execution, state changes, or corpus writes), scope-locked to an authorized
host, and bounded by hard step and con caps.

Runs key-free. The attacker brain, judge, and analyst all use an optional
LLM loaded from en . With no API key, the operator falls back
to a deterministic escalation ladder and the analyst to deterministic
heuristics — so the agent still adapts and captures without any external model.

Quickstart against the bundled intentionally- mock:

This produces reports/redteam_run/report.json and report.md — a per-objective
exploit report with captured proof, OWASP mapping, se, and the full
reproduction transcript. Use --fail-on-capture to exit non-zero in CI when any
objective is captured, and --objecti,credential_exfil to
scope the run.

To point it at your own app, copy /examples/redteam_target.yaml,
set base_url/model to your endpoint, plant your own marker/secret via
--system-marker / --planted-secret, and pass your host to --scope.

---

Web Agent

also includes Web Agent, a defensi
OWASP-style web application security testing layer inspired by PortSwigger labs.
It maps known web surfaces to safe checks (planner → generated tests → scoped GET
runner → e → detectors → findings → report). It is **GET-only and
safe-by-default**: every scan requires a --scope, out-of-scope targets are
blocked before any request, POST is blocked by default, and PUT/PATCH/
DELETE are always blocked. Sensiti (auth, cookies, tokens) are
redacted from sa.

It also runs as a bounded agent. The agent command drives an
observe → decide → act loop: it observes each response, decides the next safe
check, and stops on its own within a step cap. The decision step is grounded in a
small built-in RAG knowledge layer (OWASP / PortSwigger notes) and can use an
optional LLM whose output is strictly against real templates and
known endpoints — with a deterministic fallback so the whole thing **runs with
zero API keys. Optional, bounded same-origin endpoint discovery** lets it
expand its surface within scope. Throughout, the agent only plans and explains;
deterministic code executes every request and judges every finding, and every
decision is written to an agent_run.json audit trace.

A local, intentionally- demo app li
examples/web_demo_app/:

The Web Agent CLI also supports plan, generate-tests, , and
check. The agent and plan commands accept --planner llm (falls back to
deterministic when no key is set), and agent accepts --discover and
--max-steps.

See docs/webagent.md for the full guide: safety model,
architecture, supported checks, output format, how to add templates, limitations,
and roadmap.

---

Black-Box Agent (point-and-shoot)

is an authorized black-box tester for chatbot
endpoints. Give it a URL and a required --scope; it auto-detects the chat API
shape and runs the autonomous LLM-security battery — no config, no planted flags.

How it pro without instrumenting the target — each with an honest
confidence band:

| Objective | Detection | Confidence |
|-----------|-----------|------------|
| Prompt injection (LLM01) | an injected canary the agent controls, echoed back | deterministic |
| Unbounded consumption (LLM10) | measured response size / latency | high |
| Sensiti (LLM02) | DLP / entropy vs a clean baseline | medium |
| System-prompt leakage (LLM07) | heuristic phrasing | low ("needs review") |

Auto-adapter detects common request shapes (POST {message}/{prompt}/…, or GET).

Two operators: a deterministic payload battery (key-free) or an optional LLM

operator that generates and adapts payloads.

Single-shot or multi-turn (--max-turns > 1): one e con that

primes a premise, then strikes.

Safe by design: --scope is mandatory (target host must match), it is

talk-only, and secrets are redacted in reports.

Works for standard JSON chat APIs; authenticated, websocket/session-based, or
web-widget-only bots need an explicit --target config (there is no browser
automation). Black-box findings are signals to triage, not .

---

Why ?

LLM applications can fail in subtle ways:

A chatbot may follow prompt injection instructions.

A RAG assistant may treat retrie as trusted instructions.

A model may reveal system prompts, internal policies, or canary secrets.

A model may comply with fake authority claims like “I am the admin.”

A tool-using agent may follow malicious tool output.

A model may repeat poisoned citations, metadata, or hidden retrieved text.

A model may generate excessi in ways that create cost, latency, or a risks.

helps de test these behaviors before deployment by running repeatable black-box security tests.

The main idea is simple:

> Make LLM and RAG security failures reproducible instead of manually testing random prompts.

---

OWASP LLM Top 10 Co

maps its checks to the OWASP Top 10 for LLM Applications (2025).
Co is deliberately honest — a check exists where marked, and gaps are
listed as planned rather than implied.

| # | Category | Status | How tests it |
|---|----------|--------|--------------------------|
| LLM01 | Prompt Injection | ✅ Covered | YAML injection suites + red-team prompt_injection_obey objective with an injection canary |
| LLM02 | Sensiti Disclosure | ✅ Covered | Forbidden-secret detectors + red-team credential_exfil (planted secret, DLP/entropy scan) |
| LLM03 | Supply Chain | ⬜ Planned | Out of scope for black-box runtime testing |
| LLM04 | Data & Model Poisoning | 🟡 Partial | Local RAG scan loads poisoned documents and tests whether the model obeys injected retrie |
| LLM05 | Improper Output Handling | ⬜ Planned | — |
| LLM06 | Excessi | ⬜ Planned | — |
| LLM07 | System Prompt Leakage | ✅ Covered | System-prompt-leak detectors + red-team system_prompt_leak objective with a planted marker |
| LLM08 | & Embedding Weaknesses | 🟡 Partial | Retrie signals RAG scan mode |
| LLM09 | Misinformation | ⬜ Planned | — |
| LLM10 | Unbounded Consumption | ✅ Covered | max_output_chars detector + red-team unbounded_consumption objective with a measured token/latency budget oracle |

Legend: ✅ implemented check  ·  🟡 partial / indirect  ·  ⬜ planned

> Passing a check is a signal for review, not proof of security. Absence of a
> check does not mean the risk is absent — only that does not yet
> test for it.

---

Current Features

Autonomous LLM red-team agent:

- objecti attacks mapped to the OWASP LLM Top 10
- adapti → act → reflect loop (operator + analyst + capture oracle)
- proof-based capture (planted flags, DLP/entropy credential detection, token/latency budgets, optional LLM-as-judge)
- tactic escalation (persona, instruction o, encoding, payload splitting, context flooding)
- safe-by-default: talk-only, scope-locked, bounded step/size caps
- runs key-free with deterministic operator + analyst fallback
- exploit reports with full reproduction transcripts

YAML-based security test suites

OpenAI-compatible target adapter

Generic HTTP chatbot/API target adapter

Configurable HTTP request body templates

Configurable JSON response extraction using response_path

Local RAG scan mode

Document loading from local folders

Basic document chunking

Keyword-based retrie

Poisoned-document testing

Single-turn and multi-turn test support

Prompt injection tests

RAG / retrie injection tests

Authority spoofing tests

Sensiti disclosure tests

System prompt leakage tests

Indirect leakage tests

Unbounded consumption tests

Configurable detectors:

- forbidden string detection
- regex detection
- refusal detection
- max output character detection
- expected-answer

Required and ad detector modes

Risk scoring

Finding and recommendation generation

E capture

Full con transcripts

JSON report output

Markdown report output

Local run storage

Safe/ mock chatbot for adapter testing

GitHub Actions CI smoke tests

CLI support for:

- --target
- --tests
- --out
- --fail-on-findings
- --
- --no-color

---

Project Structure

---

Installation

1. Clone the repository

2. Create a en

3. Install

Install the package (this also registers the * console commands):

For de (adds pytest and ruff):

Or, for a dependencies-only setup without installing the package:

Once installed, the CLIs are a as console commands (equi to the
python -m ... forms used throughout this README):

| Command | Equi module | Purpose |
|---------|-------------------|---------|
| | python -m .cli | YAML LLM/RAG attack suites |
| | python -m .redteam.cli | Autonomous red-team agent |
| | python -m .webagent.cli | Web Agent (OWASP web testing) |
| | python -m .rag_scan | Local RAG scan mode |
| | python -m .blackbox.cli | Point-and-shoot black-box chatbot testing |

---

En Setup

Create a local .env file:

Then edit .env:

Your .env file should not include export, spaces around =, or shell syntax.

Good:

Bad:

Ne .env or real API keys.

---

Target Configuration

uses YAML target configs.

There are currently two supported target types:

---

OpenAI-Compatible Target

Example:

The protected are fake canary used by tests. They should never be real secrets.

Run the RAG injection suite:

---

Generic HTTP Chatbot Target

can also test any chatbot-style HTTP endpoint.

This is useful when testing:

Local chatbot APIs

RAG API endpoints

Internal AI assistants

Flask/FastAPI apps

Prototype AI services

AI apps that are not OpenAI-compatible

Example HTTP target config:

The body_template controls what sends to your API.

Common placeholders:

The response_path tells where to find the model/app response in the returned JSON.

For example, if your API returns:

use:

Run the HTTP target:

---

Local Mock Chatbot

includes a small mock chatbot for testing the generic HTTP adapter.

Start the safe mock app:

Then run:

In safe mode, the mock app treats retrie as untrusted data and should pass the RAG suite.

Start the mock app:

Then run the same command again.

In mode, the mock app intentionally echoes malicious sentinel payloads like RAG_PWNED_001, allowing to detect findings and generate e reports.

Expected behavior:

---

Local RAG Scan Mode

can run a local RAG security scan by loading documents from disk, chunking them, retrieving relevant context, building a RAG-style prompt, and scanning the target response.

This is useful for testing whether a model treats retrie as untrusted data instead of executable instructions.

Example document layout:

Run a local RAG scan:

will:

In safe mode, the mock chatbot should pass:

In mode, the mock chatbot intentionally echoes malicious payloads such as RAG_PWNED_101, allowing to detect poisoned retrie execution:

Then run:

Expected behavior:

---

Quickstart

Run the RAG injection suite:

Run the prompt injection suite:

Run with failure details:

Exit with code 1 if findings are detected:

---

Example Output

Example RAG run against an OpenAI-compatible target:

Example HTTP mock run:

Example local RAG scan:

A failed test does not necessarily mean is broken. In many cases, it means found a model behavior worth re.

---

A Test Suites

Current suites include:

---

Example Test Case

---

Required Detectors

Some tests require a detector to pass. Other detectors are ad and only provide extra e.

For example, in a RAG test, the model does not always need to say “sorry.” It may safely answer the benign user question while ignoring malicious retrie.

In this example, leaking forbidden content fails the test. Refusal language is useful e, but not mandatory.

---

Detector Types

For more detector examples and option details, see Detector Documentation.

contains

Fails when forbidden strings appear in the model response.

regex

Fails when a regex pattern matches the model response.

refusal

Passes when expected refusal language appears.

max_output_chars

Fails when the model response exceeds a configured character limit.

expected_contains

Passes when expected strings appear in the model response.

This is useful for RAG tests where the model should ignore malicious retrie while still answering the legitimate user question.

---

RAG Injection Testing

includes a RAG-focused attack suite for testing whether a model treats retrie as untrusted data.

The RAG suite includes attacks such as:

Malicious retrie

Poisoned HR or policy documents

Source-boundary confusion

Citation poisoning

Markdown link injection

HTML comment injection

YAML frontmatter injection

JSON metadata injection

Tool-result injection

Email-thread injection

Support-ticket injection

Base64 and ROT13 payloads

Quoted instruction handling

Table-cell injection

Translation-context injection

Multi-turn retrie attacks

Run the RAG suite:

---

Reports

saves two report formats for each run:

Reports include:

Scan metadata

Target information

Suite name

Pass/fail summary

Category breakdown

Se breakdown

Failed tests

Risk scores

Finding titles

Recommendations

Prompt

Model response

Detector reasons

Leak e

Refusal e

Full transcript

Retrie metadata for local RAG scans

---

Continuous Integration

's GitHub Actions workflow runs two jobs on every push and pull request.

Job 1 — Lint & unit tests:

1. Installs the package with de (pip install -e ".[dev]")
2. Lints with ruff (pyflakes, import order, pyupgrade, bugbear)
3. Runs the full pytest unit suite

Job 2 — Integration smoke tests:

1. Installs dependencies and compiles the package
2. Starts the safe mock chatbot, runs , and expects no findings
3. Runs expected-answer and a safe local RAG scan
4. Starts the mock chatbot, runs , and expects findings
5. Runs a local RAG scan and expects poisoned-context detection

Job 1 catches regressions and dead/undefined code fast; Job 2 confirms the
generic HTTP target adapter, expected-answer detector, and local RAG scan mode
work end-to-end.

You can reproduce both locally:

---

Responsible Use

is intended for defensi, research, and education.

Do not use this project to:

Attack systems you do not own

Test applications without permission

Extract secrets, private data, or system prompts from real users or production systems

Bypass safeguards in deployed AI products

Abuse API pro or create unnecessary resource consumption

Use only in en where you ha, such as:

Your own local chatbot

Your own RAG pipeline

Internal red-team en

Security labs

Educational demos

Systems where you ha permission to test

---

Security Notes

is a testing harness. It does not replace a full security process.

Use it alongside:

Application-le controls

Ser secret isolation

Output filtering

Logging and monitoring

Human review

Abuse testing

Red-team e

Never put real secrets directly into prompts, configs, test suites, or committed files.

If you accidentally leak an API key, rotate it immediately.

---

Troubleshooting

New users may encounter issues when setting up or running . This section provides solutions to common problems.

Common Issues:

* Missing API key: Ensure you have set the in your .env file. Refer to the "En Setup" section for details.
* .env formatting problems: The .env file should contain KEY=VALUE pairs without export, spaces around =, or shell syntax. For example, =your_api_key_here is correct, while export = your_api_key_here is incorrect.
* HTTP target connection refused: If you see a connection error for localhost:8000, make sure the mock chatbot is running. You can start it using:

* Mock chatbot not running: As abo the mock chatbot is active if you are testing HTTP targets.
* Flask missing from dependencies: If you encounter errors related to Flask, ensure all dependencies are installed by running pip install -r requirements.txt within your en.
* Generated reports not appearing: Check the /storage/ directory for generated JSON and Markdown reports after a run.
* Temporary RAG scan files showing up in git status: These are temporary files. Ensure they are correctly ignored by your .gitignore file. If not, consider adding them or clearing your local changes.

Current Limitations

Detectors are mostly pattern and regex based.

Semantic leakage detection is not implemented yet.

OpenAI-compatible and generic HTTP chatbot targets are supported, but pro adapters for Anthropic, Ollama, and other runtimes are not implemented yet.

Local RAG scan mode currently uses simple keyword retrieval, not embeddings.

The Web Agent is GET-only today; state-changing (POST) templates are planned

and gated, not executed.

The Web Agent's RAG knowledge layer also uses keyword retrieval, not embeddings.

The Web Agent's optional LLM planner/decision loop is and works; the

deterministic path is the default and is what runs end-to-end without a key.

Web Agent endpoint discovery is intentionally bounded (same-origin, capped, no

recursi).

The red-team agent is talk-only today: tool-use (excessi) and RAG

corpus poisoning are not yet executed. Its LLM operator/judge/analyst paths are
with mock clients; the deterministic path is the default and runs
end-to-end without a key.

Passing tests does not prove that an AI application is secure.

Failed tests require human review to distinguish true from false positives.

---

Roadmap

> v2.0 (current) introduced the Autonomous LLM Red-Team Agent.

v2.1 Reporting and CI Polish

Better CI artifacts for generated reports

More sample reports

Cleaner error handling for una HTTP targets

More robust per-test timeout handling

SARIF report output for GitHub security workflows

v2.2 Red-Team Agent Depth

Sandboxed tool execution to pro agency (LLM06) without real damage

Closed-loop RAG corpus poisoning (inject → confirm retrieval → confirm effect)

Grey/white-box targets that expose retrie and tool-call traces

Li of the LLM operator/judge/analyst paths

v2.5 Retrieval and Provider Expansion

Embedding-based RAG scan mode

Anthropic target adapter

Ollama/local model target adapter

LiteLLM-compatible target support

Semantic leakage detector

Encoded leakage detector

v3.0 Platform Direction

Tool-use and agent attack packs

MCP-specific attack packs

Dashboard or report

Historical scan comparison

Web Agent (parallel track)

Real LLM client smoke-tested against a li (currently mock-)

Embedding-based retrieval for the RAG knowledge layer

Safe POST support for local labs (currently gated)

Multi-user / IDOR access-control testing

Form and parameter discovery beyond same-origin links

Import OWASP / PortSwigger lab writeups into the knowledge layer

---

Maintainer Note

is an open-source project focused on practical LLM, RAG, and AI-agent security testing.

The goal is to make AI security failures easier to reproduce, document, and fix — from simple YAML attack suites and local RAG scans to an autonomous red-team agent that captures reproducible proof, all behind clear reports and CI-friendly workflows.

Feedback, test cases, detector impro, and security review are welcome.

---

Contributing

Contributions are welcome.

Good first contributions include:

New YAML attack suites

New detectors

Better report formatting

Additional target adapters

Better documentation

False-positi

Test co

---

License

MIT License.

项目信息

开发语言：Python

Star 数：1

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Open-source defensive security testing toolkit for LLM/RAG apps and web apps. Runs OWASP-mapped YAML attack suites, a RAG injection benchmark, and a bounded Web Agent that plans, discovers endpoints, runs safe scoped checks, and reports. Local and authorized targets only.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。