Malicious-Coding-Intent-Dataset-Classifier - Security ML project for multilingual malicious-int

项目说明

Malicious Coding Intent Classifier

Classifier for detecting malicious-coding intent in AI prompts and code-like
inputs. The project is framed as a red-team e: build a lightweight
filter, then try to break it with realistic bypasses and document where it
fails.

The model uses BAAI/bge-m3 embeddings with small logistic-regression heads:
one binary malicious/benign head and one 12-category multilabel head. The
encoder is frozen, so training the heads is cheap and reproducible.

What This Tests

Safety filters can look strong on clean English prompts and still fail under
simple ad pressure. This repo focuses on three bypass axes:

1. Obfuscation - leetspeak, homoglyphs, punctuation, truncation.
2. Language pivot - the same malicious intent outside English.
3. Code- confusion - checking whether the classifier learned
malicious intent or merely learned that code-looking text is suspicious.

The third axis is the main finding. A code-hea reached very high recall,
but flagged too much normal benign code. Adding benign-code hard negati
most of that failure mode, with a measurable recall trade-off.

Headline Results

Recommended model: models/_clean_benign_code/.

| Check | Result |
|-------|-------:|
| In-distribution test | F1 99.80%, ROC-AUC 0.9997, FPR 0.40% |
| Obfuscated hold-out | Recall 99.35% |
| Malware-code hold-out | Recall 98.90% |
| Clean OSS benign-code hold-out | FPR 1.12% |
| Language-pi recall | Avg 99.49% across 16 languages |

The important sanity check is benign code. Before benign-code hard negatives,
the 50k code-hea flagged 26.7% of normal code as malicious. The
recommended v6 model reduces broader clean OSS benign-code FPR to 1.12%
while keeping malware-code recall at 98.90%.

v6 vs v8

v8 is a hard-negati, not a blanket replacement for v6.

| Model | Role | Malware-code recall | Obfuscated recall | HF CodeParrot benign-code FPR |
|-------|------|--------------------:|------------------:|------------------------------:|
| _clean_benign_code | balanced / recommended | 98.90% | 99.35% | 7.13% |
| _clean_plus_fp_pool | code-hardened ablation | 98.40% | 99.18% | 2.28% |

Interpretation: v8 cuts CodeParrot false positives by about 3.1x
(713/10,000 -> 228/10,000) but costs 0.50 points of malware-code recall.
That is an explicit operating trade-off, not an "e got better" claim.

Language Axis

The language-pi did not show a low-resource collapse in positive
recall. On a 500-example-per-language sample:

| Model | Canonical a | Obfuscated a | Worst obfuscated language |
|-------|---------------------:|----------------------:|--------------------------:|
| v6 | 99.85% | 99.49% | de 95.40% |
| v8 | 99.71% | 99.35% | de 94.40% |

Arabic, Chinese, Japanese, Korean, Hindi, and Bengali all stayed around
99.4-100.0% obfuscated recall in this check. Per-language FPR is still pending:
the repo needs matched multilingual benign prompts before precision/FPR by
language can be claimed honestly.

!Per-language obfuscated recall

Full e notes are in docs/E.md.

Quick Start

The CLI returns the binary malicious/benign label, the raw malicious-intent
score, a deri tier (low, suspicious, high), and the top
category scores. The routing tier is a policy layer over the binary score, not
a separately trained three-class label. Use --jsonl for gateway-friendly
structured output.

Repository Layout

Key artifacts:

| Artifact | Path |
|----------|------|
| Recommended model | models/_clean_benign_code/ |
| Hard-negati | models/_clean_plus_fp_pool/ |
| E report | docs/E.md |
| Language-axis report | docs/language_axis_eval.json |
| Label schema | config/categories.yaml |

Large generated classifier splits are ignored by git and can be rebuilt or
published separately to Hugging Face.

Published Artifacts

| Artifact | Link | What it is |
|----------|------|------------|
| Code + e story | GitHub | scripts, README, docs, small model heads, e |
| v6 dataset | HF dataset | train/val/test splits for the recommended v6 model |
| v6 model | HF model | balanced recommended classifier heads |
| v8 model | HF model | code-hard-negati heads |
| Base encoder | BAAI/bge-m3 | frozen embedding model used by all classifier heads |

The large generated data/external/malware_code_merged.json file is not
committed to GitHub. It is an intermediate pool used to build classifier splits;
publish generated splits to the HF dataset repo instead of adding the raw 425MB
JSON artifact to git.

Reproduce E

Training Pipeline

Known Limitations

Scope

This is a malicious-coding intent classifier, not a general toxicity benchmark.
It is meant to e security-relevant prompts/code and red-team bypass
patterns.

License

MIT. See LICENSE.

项目信息

开发语言：Python

Star 数：1

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Security ML project for multilingual malicious-intent detection with calibrated operating points, malware-code hold-outs, and defensive-security false-positive reduction展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。