modelaudit

项目说明

Secure your AI models before deployment. Static scanner that detects malicious code, potential backdoor indicators, and security in ML model files — without e or executing them.

![PyPI ]()
![Python ]()
![Code Style: ruff]()
![License]()
![Security policy]()

Full Documentation | Usage Examples | Supported Formats

Why ModelAudit

Models download from untrusted registries, pass through CI, and end up running in production. Traditional SAST tools do not look at pickle opcodes, HDF5 group layouts, ONNX proto graphs, or TensorFlow Sa signatures — ModelAudit does:

Scan statically. No model is e, unpickled, or executed.

Cover the formats you actually ship. 40+ scanners spanning pickle, PyTorch, SafeTensors, ONNX, TensorFlow, Keras, GGUF, archives, and configs.

Fit into CI. Machine-readable output (JSON, SARIF), strict mode, exit codes, and selectable scanners.

Surface co limits. Recognized scanners report bounded-analysis gaps such as truncated reads or exhausted budgets instead of presenting them as fully covered results.

Comparable tools: picklescan (pickle only, Python-based), fickling (pickle only, AST-based), modelscan (pickle + TensorFlow + Keras subset). ModelAudit is broader in co and ships a native Rust pickle engine via its companion package modelaudit-picklescan.

Quick Start

Requires Python 3.10-3.13

Scan e and source errors can include original credential , including in JSON, SARIF, SBOM, and shared reports. Credential normalization remains where it affects detection, grouping, or suppression.

What It Detects

Code execution attacks in Pickle, PyTorch, NumPy, and Joblib files

Potential backdoor indicators — suspicious weight patterns, anomalous tensors, or hidden-code signals

Embedded secrets — API keys, tokens, and credentials in model weights or metadata

Network indicators — URLs, IPs, and socket usage that could enable data exfiltration

Archi — path tra, symlink attacks in ZIP/TAR/7z files

Unsafe ML operations — Lambda layers, custom ops, TorchScript/JIT, template injection

Supply chain risks — tampering, license , suspicious configurations

Supported Formats

ModelAudit includes 45 registered scanners co model, archive, and configuration formats:

| Format | Extensions | Risk |
| ----------------------- | ------------------------------------------------------------------------------------------- | ------ |
| Pickle | .pkl, .pickle, .dill | HIGH |
| PyTorch | .pt, .pth, .ckpt, .bin | HIGH |
| Joblib | .joblib | HIGH |
| NumPy | .npy, .npz | HIGH |
| R Serialized | .rds, .rda, .rdata, signature-valid renamed workspace artifacts | HIGH |
| TensorFlow | .pb, .meta, Sa dirs | MEDIUM |
| Keras | .h5, .hdf5, .keras | MEDIUM |
| ONNX | .onnx | MEDIUM |
| CoreML | .mlmodel, structurally valid renamed artifacts | LOW |
| MXNet | *-symbol.json, *-NNNN.params, structurally valid renamed symbol JSON | LOW |
| NeMo | .nemo, renamed archives with root config | MEDIUM |
| CNTK | .dnn, .cmf, signature-valid renamed artifacts | MEDIUM |
| RKNN | .rknn, signature-valid artifacts under non-conflicting renamed suffixes | MEDIUM |
| Torch7 | Serialized artifacts (.t7, .th, .net or renamed) | HIGH |
| CatBoost | .cbm | MEDIUM |
| XGBoost | .bst, .model, .json, .ubj, extensionless UBJSON | MEDIUM |
| LightGBM | .lgb, .lightgbm, .model, signature-valid renamed artifacts | MEDIUM |
| Llamafile | Executable wrappers (.llamafile, .exe, extensionless or renamed) | MEDIUM |
| TorchServe | .mar | HIGH |
| SafeTensors | .safetensors | LOW |
| GGUF/GGML | .gguf, .ggml, .ggmf, .ggjt, .ggla, .ggsa, signature-valid renamed artifacts | LOW |
| JAX/Flax | .msgpack, .flax, .orbax, .jax, .checkpoint, .orbax-checkpoint | LOW |
| TFLite | .tflite, signature-valid artifacts under non-conflicting renamed suffixes | LOW |
| ExecuTorch | .ptl, .pte, signature-valid standalone artifacts under non-conflicting renamed suffixes | LOW |
| TensorRT | .engine, .plan, .trt | LOW |
| PaddlePaddle | .pdmodel, .pdiparams | LOW |
| OpenVINO | .xml | LOW |
| Skops | .skops | HIGH |
| PMML | .pmml | LOW |
| Compressed Wrappers | .gz, .bz2, .xz, .lz4, .zlib | MEDIUM |

Plus scanners for ZIP, TAR, 7-Zip, OCI layers, Jinja2 templates, JSON/YAML metadata, manifests, model cards, text files, and RAR recognition. RAR archives are reported as unsupported/fail-closed instead of being skipped.

Structurally valid TensorFlow Sa and MetaGraph protobufs are also recognized when renamed to non-model suffixes.
CoreML models can also be recognized when renamed, with incomplete co reported explicitly.
SafeTensors files with o but plausible framing are retained for bounded, inconclusi under
otherwise unclaimed suffixes such as .jpg.
Structurally plausible Flax/JAX MessagePack checkpoints are also recognized when renamed to non-model suffixes;
renamed structures that cannot be fully classified are reported as incomplete co.
Structured JAX/Orbax JSON checkpoint metadata is likewise recognized when renamed; o ambiguous
candidates are reported as incomplete co; observable security patterns in the bounded inspected prefix may
still be reported conser.

View complete format documentation

Remote Sources

Scan models directly from remote registries and cloud storage:

Authentication En

HF_TOKEN for private Hugging Face repositories

AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (and optional AWS_SESSION_TOKEN) for S3

GOOGLE_APPLICATION_CREDENTIALS for GCS

MLFLOW_TRACKING_URI for MLflow registry access

MODELAUDIT_MLFLOW_ALLOWED_ARTIFACT_URIS for non-local MLflow artifact roots (comma-separated URI prefixes)

Use concrete backend roots such as s3://bucket/prefix; logical models:/ or runs:/ URIs and percent-encoded remote paths are refused.

JFROG_API_TOKEN or JFROG_ACCESS_TOKEN for JFrog Artifactory

MODELAUDIT_JFROG_ALLOWED_HOSTS for comma-separated custom JFrog hostnames that may recei

MODELAUDIT_JFROG_ALLOWED_REDIRECT_HOSTS for comma-separated external redirect hostnames that may be downloaded without credentials

Store credentials in en or a secrets manager, and ne tokens/keys.

Installation

The ONNX extra, including the ONNX portion of modelaudit[all], is packaged for Python 3.10-3.12.
PyTorch-containing extras require Python 3.10-3.12 or standard (GIL-enabled) CPython 3.13; the patched PyTorch 2.13
wheels do not support free-threaded CPython 3.13t. Use standard CPython, since ModelAudit's nati scanner also
does not support CPython 3.13t. On Apple Silicon, PyTorch-containing extras require macOS 14 or newer because the wheels
target macosx_14_0_arm64; the core-only installation remains a for static scanning on macOS 11-13.

CLI Options

Primary commands:

Common scan options:

Reports retain raw source identifiers and credential-bearing e. Consumers of file_metadata
should use the emitted keys; pre masked source keys are no longer lookup aliases.
Very long source identifiers are shortened consistently within each report.
The legacy redacted_value key is retained for compatibility; it now contains bounded raw e and provides no masking guarantee.

The redact_huggingface_url_for_display and redact_huggingface_urls_in_text helpers are no longer
a from modelaudit.utils.sources.huggingface or huggingface_paths. Remo imports to
retain raw e, or apply your application's own masking policy before sharing output.

Targeted scanner selection:

--scanners starts from an explicit allowlist. --exclude-scanner subtracts scanners from either that allowlist or the default scanner set. Scanner selection is reflected in JSON output under scanner_selection.

For remote folders, ModelAudit narrows downloads by selected scanner extensions when safe. Content-based renamed-wrapper routing applies after acquisition; scan a direct file URL when repository filenames may be intentionally misleading.

Metadata Extraction

--trust-loaders enables scanner metadata loaders that may deserialize model content. Only use this on trusted artifacts in isolated en.

Exit Codes

0: No security issues detected

1: Security issues detected

2: Scan errors

Telemetry and Privacy

ModelAudit includes telemetry for product reliability and usage analytics.

Collected metadata can include command and feature usage, scan and download timing or size metadata, scanner/file-type usage, stable rule or C, se aggregates, registered file extensions, coarse source or provider categories, and runtime , platform, and CI metadata.

Telemetry does not include raw local paths, filenames, model IDs or names, repository or artifact identifiers, bucket or object keys, remote URL paths, free-form issue or error text, URL userinfo, query strings, fragments, or model binary contents.

Packaged installs enable telemetry by default. Telemetry is disabled automatically when CI=true is set or IS_TESTING=true is set, and in editable de installs unless MODELAUDIT_TELEMETRY_DEV=1. Events that are sent from other CI pro (TeamCity, CodeBuild, Bitbucket Pipelines, Jenkins) are tagged with isRunningInCi=true so they can be filtered downstream.

A persistent pseudonymous user identifier is stored in ~/.promptfoo/promptfoo.yaml for cross-tool correlation with Promptfoo, and each telemetry session uses a new session identifier. Existing IDs from ~/.modelaudit/user_config.json are migrated on first run after upgrade. A legacy email value in that file, if present, may be attached to the analytics user profile.

Opt out explicitly with either en :

To opt in during editable/de installs:

Output Examples

Troubleshooting

Run modelaudit doctor --show-failed to list una scanners and missing optional deps.

Run modelaudit debug --json to collect en/config diagnostics for bug reports. Inspect the raw output and remo before sharing it.

Use modelaudit cache cleanup --max-age 30 to remo cache entries safely.

If pip installs an older release, Python is supported (python --; ModelAudit supports Python 3.10-3.13).

For additional troubleshooting and cloud auth guidance, see:

-
-

Documentation

Full docs — setup, configuration, and ad usage

Usage examples — CI/CD integration, remote scanning, SBOM generation

Supported formats — detailed scanner documentation

Support policy — supported Python/OS and maintenance policy

Security model and limitations — what ModelAudit does and does not guarantee

Compatibility matrix — file formats dependencies

Scanner selection — targeted scanner allowlists and exclusions

Metadata extraction guide — safe metadata workflows and --trust-loaders guidance

Offline/air-gapped guide — secure operation without internet access

Troubleshooting — run modelaudit doctor --show-failed to check scanner a

Related Packages

modelaudit-picklescan — the standalone Rust-backed pickle scanner used by ModelAudit's pickle, PyTorch, ExecuTorch, and PyTorch-ZIP scanners. Install it directly if you only need pickle analysis (as a library, not a CLI) and do not want the full scanner bundle.

Reporting

Do not open public issues for suspected . See SECURITY.md for coordinated disclosure.

Contributing

Issues, feature requests, and PRs are welcome. See CONTRIBUTING.md.

License

MIT License — see LICENSE for details.

项目信息

开发语言：Python

Star 数：76

技术要点

该项目属于安全软件方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Security scanner for AI/ML model files. Detects malicious code, backdoors, and vulnerabilities before deployment展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。

登录后才能参与讨论

下载链接

会员才能下载

登录后开通会员

同分类推荐

暂无同类资源

资源标题	modelaudit - Security scanner for AI/ML model files. Detects ma
所属分类	安全软件
发布作者	admin
发布时间	2026-10-05 22:43:26
下载权限	免费下载
附件数量	2 个下载链接
浏览次数	186
讨论数量	0
主题状态	正常讨论中

modelaudit - Security scanner for AI/ML model files. Detects ma