cybergym - CyberGym is a large-scale, high-quality cybersecur

项目说明

CyberGym: E AI Agents' Real-World Cybersecurity Capabilities at Scale

![Website]()
![ArXiv]()
![Hugging Face]()

CyberGym is a large-scale, high-quality cybersecurity e framework designed to rigorously assess the capabilities of AI agents on real-world analysis tasks.

> [!NOTE]
> - FAQ.md: frequently asked questions about benchmark setups, such as network access, local deployment, pre-/post-patch , interpreting outcomes.
> - SUBMISSION.md: guidelines for leaderboard submissions, including cost reporting, writeup requirements, etc.

> [!WARNING]
> Deploy e locally. Do not expose any part of CyberGym to the public internet.

Installation

Require python and docker en.

Install the dependencies for the server and the task generation

Download the benchmark data (~240GB)

Download Server Data (binary only mode)

If you only need static analysis and don't require the dynamic compilation en, you can use this binary mode which only takes ~130GB.

Download Server Data (full data)

Download the docker images with the compilation en:
1. Full data

2. Subset data

The full server data is large (~10TB). We provide a subset with the following 10 tasks, which include 5 tasks that the agent can successfully generate the PoC and 5 tasks that are not easy for the agent.

Download the subset data

E

The server only needs to be reachable from the agent containers and from the host. 0.0.0.0 binds every interface, which on a machine with a public IP exposes a partly unauthenticated service to the internet.

Pick the bind address

The right address is the gateway of the Docker network your agent containers run on — that address is a host-side interface reachable from those containers and from the host, but not routable from outside the machine.

Notes:

The two gateways are not interchangeable. cybergym-internal is created with internal=True, so containers on it have no route to the default bridge — a ser to 172.17.0.1 is unreachable from firewalled agents.

Docker assigns the cybergym-internal subnet when the network is created, so the gateway is host-specific. Query it rather than hardcoding it. python3 -m cybergym.firewall start and status both print it as host_gateway.

FirewallProxyManager.start() already adds this gateway to NO_PROXY and to the proxy's IP allowlist, so agent traffic to the ser Squid.

Use the same $HOST value in --server when generating tasks and PoCs below.

Start the PoC submission server:

Start the PoC submission server (binary only mode):

Test:

the PoCs Submitted by the Agent

After running the agent, you can get the agent_id from the logs/args.json.
You can the PoCs submitted by:

Firewall (Restrict Agent Internet Access)

CyberGym includes a domain-allowlist proxy that restricts agent containers to appro only. It creates an isolated Docker network with no direct internet route — all outbound traffic must pass through a Squid proxy filtered by domain.

Start the proxy (uses the built-in allowlist for apt, pip, and LLM APIs):

Check status, stop the proxy, or tear down e:

Agent containers must be started on the cybergym-internal network and use the proxy env vars (see ProxyManager.en()). The default allowlist is at src/cybergym/firewall/default_allowlist.txt.

Note that the firewall only filters outbound traffic from agent containers. It does nothing about inbound reachability of the submission server — that is still up to how you bind and firewall the host.

See python3 -m cybergym.firewall --help for more usage.

Example Agents

The four example agents can be installed as:

Then check the instructions in the folder: Example Agents

Citation

If you use this project in your research, please cite:

License

This project is licensed under the Apache License 2.0.

项目信息

开发语言：Python

Star 数：933

技术要点

该项目属于漏洞知识库方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕CyberGym is a large-scale, high-quality cybersecurity evaluation framework designed to rigorously assess the capabilities of AI agents on real-world vulnerability analysis tasks.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。