NYU_CTF_Bench - CTF思路相关项目

项目说明

NYU CTF Bench

This repository hosts the NYU CTF Bench, a collection of CTF challenges from the CSAW CTF competitions, designed for e of LLM agents.
The CTF challenges are dockerized and easily deployable to allow an LLM-based automation framework to interact with the challenge and attempt a solution.
The main benchmark dataset contains 200 challenges across 6 CTF categories: web, binary exploitation (pwn), forensics, reverse engineering (re (crypto), and miscellaneous (misc).

Benchmark structure

The test/ folder contains the main benchmark dataset of 200 challenges. A smaller de set of 55 challenges is present in the de/ folder.
The de set can be treated equi to a "train" split and used for building the agent, so that design decisions made to improve the agent do not bias the test scores.

The folder structure is as follows: <year>/<event>/<category>/<challenge>.
<year> is the year of the competition, <event> is either "CSAW-Quals" or "CSAW-Finals", <category> is among the 6 categories, and <challenge> is the challenge name.
Note that the challenge name may ha and single-quotes, so it is ad to wrap it in double-quotes when using in scripts.

Each challenge contains a challenge.json containing the metadata of the challenge, and the corresponding challenge files.
Challenges that require a server to host some challenge files are set up with a docker image, and a docker-compose.yaml file.
The docker image is loaded directly using docker compose up.

Setup

Install the python package:

The repository is automatically cloned when the CTFDataset is first instantiated with the split argument.
If needed, you can manually clone it by running:

Usage

The following python snippet shows how to load challenge details using the python module:

Tests

Run tests on the challenges, for docker setup and network connection.
Requires the docker network to be setup.

Optionally filter the tests with the unittest -k flag.

项目信息

开发语言：Python

Star 数：178

技术要点

该项目属于CTF思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕CTF思路相关项目展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。