vibeshield - Defensive security scanner for AI-generated code

项目说明

Defensi CLI security scanner purpose-built for AI-generated
("vibe coded") web app and API source code. Finds where user input enters a
codebase, detects concrete patterns, and gives a plain-language
risk explanation plus a concrete fix for each finding.

Hard constraint: detection, explanation, and remediation guidance only.
Ne exploit code, attack payloads, or offensi
tooling, regardless of how a request is framed.

---

Why this exists

AI coding assistants make it easy to ship working code without fully
understanding every security implication of what got generated.
is a safety net for that specific failure mode — point it at a folder,
get back the a human re would flag, in plain English,
with the actual fix.

Every claim in this README is backed by a real test, run against real code
(not just synthetic fixtures) — see Hardening history
below for the specific bugs this caught in its own de.

---

Setup

GROQ_API_KEY is optional. All 14 detection categories work with zero API
calls. A key only adds LLM-inferred risk surfaces (entry points regex can't
express — message queues, webhooks, custom parsers) and context-specific
remediation on top of the static results.

---

Running it

---

Architecture — 4-node pipeline

| Node | File | What it does |
|---|---|---|
| Scan Repo | src//nodes/scan_repo.py | Walks the target folder, skips .git/__pycache__/.venv/node_modules/binaries/archives, reads every file. Survi files, broken symlinks, and files remo without crashing — reports them instead. |
| Classify Risk Surface | src//nodes/classify_risk_surface.py | Finds where user input enters the system. Regex rules -> confirmed. Real Groq call proposes more -> inferred only if e is against the actual file, else inferred-un. Entry-point labels are normalized in code against a fixed , not left to whate the model returns. |
| Deep Read | src//nodes/deep_.py | Deterministic regex detectors across all 14 categories below. No LLM in in this node at all. Comment/docstring content is masked before matching so prose describing a pattern can't be mistaken for the pattern itself. |
| Explain + Remediate | src//nodes/explain_remediate.py | Static remediation template for every finding (always present). Real Groq call adds context-specific suggestion on top, when enabled. |

Supporting modules:

state.py — data model (FileRecord, RiskSurface, , GraphState)

file_reader.py — reads files with correct encoding detection (UTF-8, UTF-16, UTF-8-BOM), not a hardcoded UTF-8 assumption

comment_masking.py — masks Python docstring content before regex matching to prevent false positives from prose

baseline.py — accept-once suppression system so re-scans only surface genuinely new findings

hallucination_filter.py — (), checks an LLM-claimed snippet actually exists in the real file content

secret_redaction.py — redact_text(), strips secret-shaped strings before anything is sent to Groq

groq_client.py — real Groq API wrapper with retry/backoff on transient failures (rate limits, timeouts); enabled is True only when a key is present AND not in offline mode; records the real failure reason instead of silently returning None with no explanation

graph.py — wires the 4 nodes in sequence

cli.py — entry point, loads .env, prints the report, surfaces every failure mode (truncated files, unreadable files, LLM errors) instead of a silent "clean" result

reporting.py, de.py — report summary + guidance text

---

Detection categories (14, multi-language)

| Category | Se | What it catches | Languages covered |
|---|---|---|---|
| sql_injection | critical | String-concatenated SQL queries, including strings with a nested quote of the other type | language-agnostic pattern |
| nosql_injection | critical | Un request data passed into a query filter | JS/Node |
| hardcoded_secret | critical | API keys, passwords hardcoded in source. Never sent to the LLM, e | language-agnostic pattern |
| weak_hash | moderate | MD5/SHA-1 used for hashing | Python, JS/Node, PHP, Go, Java |
| xss | critical | HTML built by string concat, innerHTML =, Go's template.HTML() auto-escape bypass | Python, JS/Node, Go |
| code_injection | critical | eval()/new Function() on un request input — arbitrary code execution, not just a shell command | JS/Node, PHP |
| insecure_cors | moderate | Wildcard Access-Control-Allow-Origin (handles both KEY = val and dict['KEY'] = val styles) | language-agnostic pattern |
| command_injection | critical | Shell exec with input | Python, JS/Node, PHP, Go, Java |
| insecure_session | moderate | Cookie/session security flags disabled | Python (Flask/Django), JS/Node (Express) |
| error_disclosure | moderate | Raw exception/stack trace returned to the client | Python, JS/Node, Go |
| risky_dependency | low | Unpinned dependency (*, latest) in the manifest. Lockfiles are excluded from this check — their * entries are normal peerDependency metadata, not unpinned installs | package.json |
| dockerfile_root | low | Base image pinned to :latest | Dockerfile |
| insecure_auth | critical | TLS disabled, JWT algorithm="none" | Python, Go |
| debug_mode_enabled | moderate | Framework debug mode left on | Python |

Every finding carries: category, se, confidence
(confirmed/inferred/inferred-un), line, e (exact
matched text), remediation.

Known remaining gaps (named honestly, not hidden): Ruby has no dedicated
patterns in any category yet. nosql_injection, insecure_session,
error_disclosure, insecure_auth, debug_mode_enabled are not yet covered
for PHP/Go/Ja what's listed abo/docstring masking only
covers Python triple-quoted strings — single-line #/// comments and
block /* */ comments in other languages aren't masked yet, so prose in
those comment styles can still trigger a false positive the same way Python
docstrings used to. hardcoded_secret can miss secret containing
non-ASCII characters.

---

Baseline / suppression

Re-running a scanner against the same project fore the same
already-re findings, which trains people to ignore the tool. The
baseline system fixes that:

Findings are matched by file + category + exact e text, not line
number — so unrelated edits elsewhere in a file don't cause an already-accepted
finding to reappear as "new." A corrupt or unreadable baseline file fails
safe (treated as empty, e shown) rather than crashing the scan.

---

Privacy / data handling

Zero external calls by default. All 14 categories are fully detectable

regex alone, with no GROQ_API_KEY set.

--offline forces this even if a key IS present.

Redaction runs on every LLM call, unconditionally (secret_redaction.py)

— not a flag, not opt-in.

hardcoded_secret findings ne the LLM at all, redacted or

not — locked in by tests/test_privacy.py::test_hardcoded_secret_ne.

.env is loaded automatically by cli.py via python-dotenv.

Ne a real API key. If one is e exposed

(e.g. pasted somewhere), rotate it immediately at

---

Testing

Automated (48 tests):

tests/test_detection_accuracy.py — one test per category (including

negati: a safe eval('1+1') must NOT be flagged, a Dockerfile
string with + must NOT be flagged as SQL injection), a clean-file
negati, and a case-insensiti regression test

tests/test_privacy.py — redaction, offline gating, hardcoded_secret exclusion

tests/test_de.py — report summary/recommendations, and

a permission-denied-file simulation proving one bad file can't crash the scan

tests/test_file_encoding.py — UTF-16, UTF-8-with-BOM, and plain UTF-8

files are all correctly scanned (this locks in the most severe bug found
during hardening — see below)

tests/test_groq_retry.py — transient API failures (rate limits, timeouts)

are retried and recover; permanent failures (bad key) fail fast without
wasting retries

tests/test_baseline.py — accepted findings are suppressed on rescan, a

genuinely new finding still surfaces even with a baseline in place, and a
corrupt baseline file fails safe instead of crashing

Real API call, standalone (no pytest, no mock):

Confirms GROQ_API_KEY is actually loaded and a live Groq response comes
back — pro's no mock path left anywhere in groq_client.py.

---

Hardening history (found, fixed, and locked in)

This tool was ad tested against its own source, a real Node/React
project (FlowForge), and a real Python/LangGraph project () — not
just curated test fixtures. Every issue below was found on real code, fixed,
and is now covered by a permanent regression test.

The most severe finding: reading a UTF-16-encoded file with the UTF-8
codec does not raise an error — every ASCII byte and null byte is
independently valid UTF-8 — so it silently decodes into garbage (a null byte
between every character) and every regex match breaks with zero warning.
This produces a false "clean scan" result. This is not a hypothetical edge
case: Windows PowerShell's > redirection and several common Windows text
editors default to UTF-16LE, making this an entirely ordinary real-world
file. Fixed by BOM-sniffing before decode; on a real UTF-16 file
created via Out-File -Encoding unicode on an actual Windows machine.

Other real bugs found and fixed:

CLI argument-order parser bug ( <path> --offline failed, --offline <path> worked) — Typer's implicit command-group behavior was reserving a COMMAND slot; fixed by using a single registered command instead of a bare callback.

Files over the read-size cap were silently truncated with no indication a past the cutoff could have been missed — now warned explicitly, and the cap was raised substantially.

sql_injection regex failed on the single most common real-world shape of the : a query string containing a nested quote of the other type (e.g. a Python double-quoted string with a SQL single-quoted value) — missed even in this project's own test fixture until fixed.

Binary archives (.zip, .tar, etc.) were being read and scanned as text, producing wasted work and a risk of coincidental garbage matches.

Lockfiles (package-lock.json, npm-shrinkwrap.json) were flagged for risky_dependency on their peerDependencies entries — normal, expected metadata, not a real unpinned-dependency risk. via a real before/after scan of FlowForge: 6 false positives and 2 scanned archi disappeared, exactly as predicted.

Prose in Python docstrings describing a pattern (e.g. a comment explaining what NOT to do) was itself triggering a false positi regex matching had zero awareness of comments vs. code.

The docstring-masking fix above only checked .endswith(".py") case-sensitively, so an uppercase .PY extension reintroduced the exact same false positive it had just fixed.

One permission-denied, deleted, or otherwise unreadable file anywhere in a scan would crash the entire run instead of being skipped and reported.

Groq API calls had no retry logic: a single transient rate-limit or timeout partway through a scan silently degraded the rest of the run to offline-only results, with a report that looked identical to "LLM was ne." Now retries transient errors with backoff and records the real failure reason so the CLI can warn instead of staying silent.

code_injection (the eval()-on-user-input detector) had zero test co despite being a real, documented detection category.

LICENSE file was empty despite pyproject.toml declaring MIT, and the package was inconsistent between pyproject.toml and __init__.py.

---

License

MIT — see LICENSE.

项目信息

开发语言：Python

Star 数：3

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Defensive security scanner for AI-generated code展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。