Sarracenia - A low resource anti-scraper tarpit inspired by Nep

项目说明

<di="center">
<img src="icon.svg" width=128 height=128 alt="Logo">
<h1>Sarracenia</h1>

![MIT License]()
![Go Report Card]()
![Go ]()
![GitHub release (latest by date)]()
![Docker Image]()
![Repo size]()

![Sarracenia Test / Build / Release]()
![CodeQL]()

</div>

A high-performance, configurable anti-scraper tarpit ser in Go.

Sarracenia is meant to serve as a defensi against web scrapers by serving generated, endless, and
good-enough web content to be believable. It's primary goal is to trap scrapers and keep them away from your actual
web content, or as a more strict enforcer for those who don't listen to robots.txt.

Sarracenia is made to use a very low amount of resources while remaining performant, and uses SQLite databases to hold
data when it's not being used. This allows Sarracenia to ha like multiple marko, each trained
on hundreds of MB of text data or e, while keeping its memory footprint in the double digits at most.

Please note that Sarracenia is currently feature complete, as I ha e that I wanted to.

If you want new features, or encountered a bug that needs fixing, please open a github issue or a pull request.
Contributions are welcome!

---

Core Libraries

* pkg/markov: A persistent Marko library supporting streaming generation, database-backed storage, and
ad sampling techniques.
* Documentation
* pkg/templating: A dynamic HTML generation engine capable of producing complex, randomized DOM structures and
executing logic-hea.
* Documentation

---

Installation

1. From Release (Recommended)

1. Download the latest binary for your OS from
the Releases Page.
2. Download the Source code archive (zip/tar.gz) from the same release.
3. Extract the archive and copy the example directory contents to your working folder:

4. Run the binary:
* Linux/macOS: ./sarracenia
* Windows: .\sarracenia.exe

2. Docker

A pre-built image is a on the GitHub Container Registry.

3. From Source

Prerequisites: Go 1.24+

---

Testing

---

Initial Setup

1. Access the Dashboard:
By default, the dashboard runs on port :7278. Open a browser and na to .
* Note: The port for the dashboard does not implement rate-limiting. If you are not behind a service to circumvent
that such as Cloudflare, do not expose the API to the internet.

2. Get the Master API Key:
On first launch, the ser the master key automatically at startup. It gets the master key from one of two
sources.
* If the sarr-master-key en is set, that value becomes the master key.
* Otherwise, a key is generated and logged at warn level:
level=WARN msg="No API keys existed, generated master API key:" key=sarr_...
* You can then use this key to access the dashboard.

---

Configuration

Configuration is managed via config.json.

Ser (ser)

| Key | Description | Default |
|:------------------------|:--------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------|
| ser | Tarpit ser address. | :7277 |
| api_addr | API/Dashboard ser address. | :7278 |
| log_level | Logging (debug, info, warn, error). | info |
| trusted_proxies | List of CIDRs or IPs to trust for CF-Connecting-IP, X-Real-IP, and X-Forwarded-For headers. | [] |
| data_dir | Base directory for data files. | ./data |
| marko | Path to the Marko database. | ./data/sarracenia_markov.db?_journal_mode=WAL&_busy_timeout=5000 |
| auth_database_path | Path to the Auth/Whitelist database. | ./data/sarracenia_auth.db?_journal_mode=WAL&_busy_timeout=5000 |
| stats_database_path | Path to the Statistics database. | ./data/sarracenia_stats.db?_journal_mode=WAL&_busy_timeout=5000 |
| dashboard_tmpl_path | Path to dashboard templates. | ./data/dashboard/templates/ |
| dashboard_static_path | Path to dashboard static assets. | ./data/dashboard/static/ |
| enabled_templates | List of templates enabled for random selection. | ["page.tmpl.html"] |

Tarpit Configuration (tarpit_config)

Controls the behavior of the tarpit response mechanism.

| Key | Description | Default |
|:-------------------------|:---------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| enable_drip_feed | If true, responses are sent in slow chunks to hold connections open. | false |
| min_initial_delay_ms | Minimum delay before sending the first byte. | 0 |
| max_initial_delay_ms | Maximum delay before sending the first byte. | 15000 |
| min_drip_feed_delay_ms | Minimum delay between subsequent chunks. | 500 |
| max_drip_feed_delay_ms | Maximum delay between subsequent chunks. | 1000 |
| min_drip_feed_chunks | Minimum total chunks to split the response into. | 1 |
| max_drip_feed_chunks | Maximum total chunks to split the response into. | 20 |
| headers | HTTP headers that the tarpit replies to each request with. | {"Cache-Control":"no-store, no-cache","Pragma":"no-cache","Expires":"0","Content-Security-Policy": "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';","Content-Type":"text/html; charset=utf-8",} |

Statistics Configuration (stats_config)

| Key | Description | Default |
|:-------------------------|:------------------------------------------------------------------------------------------------------------|:--------|
| sync_inter | Frequency of flushing stats from memory to disk. | 30 |
| forget_threshold | Minimum hits required to retain an IP record. | 10 |
| forget_delay_hours | Time without activity before a record is pruned. | 24 |
| max_user_agent_bytes | Truncates user-agent strings to this many bytes. 0 disables. | 512 |
| max_unique_user_agents | Keeps at most this many distinct user agents; oldest by last_seen are dropped on each sync. 0 disables. | 1000 |

Log Configuration (logs_config)

| Key | Description | Default |
|:------------------------|:--------------------------------------------------------------------------|:----------------------|
| filename | File to write the log to; rotated backups are kept in the same directory. | ./logs |
| max_size | Maximum log file size in megabytes before rotation. 0 means 100. | 500 |
| max_age | Days to retain rotated log files. 0 retains them all. | 28 |
| max_backups | Number of rotated log files to retain. 0 retains them all. | 3 |
| local_time | Use local time in backup timestamps. | true |
| compression | Backup compression: none, gzip, or zstd. | gzip |
| rotation_interval | Maximum duration between rotations, in hours. 0 disables time rotation. | 24 |
| rotate_at_minutes | Minutes within the hour (e.g. [0, 30]) at which to rotate. | [] |
| rotate_at | Clock times (e.g. ["00:00", "12:00"]) at which to rotate. | [] |
| backup_time_format | Time layout used in backup file names. | 2006-01-02-15-04-05 |
| append_time_after_ext | Place the timestamp after the file extension. | true |
| file_mode | File mode for new log files (decimal; 420 = 0644). 0 means 0640. | 420 (0644) |

Template Configuration (template_config)

| Key | Description | Default |
|:-----------------------------|:----------------------------------------------------------------------------------------|:----------------|
| marko | Controls whether markov functions use the generator. Falls back to random if false. | true |
| marko | Separator used by the marko. | "" |
| markov_eoc | End-of-chain marker used by the marko. | "" |
| marko | Regex for splitting tokens in the marko. | "" |
| marko | Regex for detecting EOC tokens. | "" |
| marko | Regex for tokens that should not have a separator prefix. | "" |
| marko | Regex for tokens that should not have an EOC suffix. | "" |
| path_whitelist | URL paths considered safe; excluded from random link generation. | [] |
| min_subpaths | Minimum number of segments in generated URL paths. | 1 |
| max_subpaths | Maximum number of segments in generated URL paths. | 5 |
| max_json_depth | Hard limit on recursion depth for randomJSON. | 8 |
| max_nest_divs | Hard limit on recursion depth for nestDivs. | 50 |
| max_table_rows | Maximum rows for randomComplexTable. | 100 |
| max_table_cols | Maximum columns for randomComplexTable. | 50 |
| max_form_fields | Maximum fields for randomForm. | 75 |
| max_style_rules | Maximum complex CSS rules for randomStyleBlock. | 200 |
| max_css_vars | Maximum interdependent CSS for randomCSSVars. | 100 |
| max_s | Complexity limit for randomSVG. | 7 |
| max_js_content_size | Maximum content size (bytes) for jsInteracti. | 1048576 (1MB) |
| max_js_waste_cycles | Maximum waste loop iterations for jsInteracti. | 1,000,000 |

Threat Configuration (threat_config)

Configures the heuristic threat assessment system.

| Key | Description | Default |
|:---------------------|:------------------------------------------------|:--------|
| base_threat | Initial score for any request. | 0 |
| ip_hit_factor | Score added per IP hit. | 1.0 |
| ua_hit_factor | Score added per User Agent hit. | 0.5 |
| ip_hit_rate_factor | Multiplier for IP hit rate (hits/min). | 10.0 |
| ua_hit_rate_factor | Multiplier for UA hit rate (hits/min). | 5.0 |
| max_threat | Maximum possible threat score. | 1000 |
| fallback_level | Default threat stage (0-4) if no threshold met. | 0 |

Threat Stages:
Stages define thresholds for triggering increasingly aggressi templates.

| Stage | Enabled | Threshold |
|:----------|:--------|:----------|
| stage_1 | True | 0 |
| stage_2 | False | 25 |
| stage_3 | False | 50 |
| stage_4 | False | 75 |
| stage_5 | False | 100 |

---

API Reference

Note: The API is designed for internal use by the dashboard. It does not implement rate limiting. Do not expose the
API port directly to the public internet.

All endpoints require the sarr-auth header containing a valid API key.

Authentication (/api/auth)

| Method | Endpoint | Scope | Description |
|:---------|:----------------------|:--------------|:---------------------------------------------------|
| GET | /api/auth/me | *Any* | current session. |
| GET | /api/auth/keys | auth:manage | Lists API keys. |
| POST | /api/auth/keys | auth:manage | Creates a new key. First key is always Master. |
| DELETE | /api/auth/keys/{id} | auth:manage | Deletes a key. |

Marko (/api/markov)

Do note: Only one model can be trained at a time. Simultaneous training jobs will result in database
lock errors.

| Method | Endpoint | Scope | Description |
|:---------|:-------------------------------------|:---------------|:----------------------------------|
| GET | /api/marko | markov:read | Lists a models. |
| POST | /api/marko | marko | Creates a new model. |
| DELETE | /api/marko/{name} | marko | Deletes a model. |
| POST | /api/marko/{name}/train | marko | Trains a model (Text/Plain body). |
| POST | /api/marko/{name}/prune | marko | Prunes model data. |
| GET | /api/marko/{name}/export | markov:read | Exports model as JSON. |
| POST | /api/marko/{name}/generate | markov:read | Generates text. |
| POST | /api/marko | marko | Imports a model from JSON. |
| POST | /api/marko/prune | marko | Global pruning. |
| GET | /api/marko/status | markov:read | Checks training status. |

Ser (/api/server)

| Method | Endpoint | Scope | Description |
|:-------|:-----------------------|:-----------------|:---------------------|
| GET | /api/health | *None* | Health check. |
| GET | /api/ser | stats:read | Ser info. |
| GET | /api/ser | ser | Get current config. |
| PUT | /api/ser | ser | Update config. |
| POST | /api/ser | ser | Restart server. |
| POST | /api/ser | ser | Shutdown server. |

Statistics (/api/stats)

| Method | Endpoint | Scope | Description |
|:---------|:-----------------------------|:-----------------|:--------------------------|
| GET | /api/stats/summary | stats:read | Global request summary. |
| GET | /api/stats/top_ips | stats:read | Top 100 IPs by hit count. |
| GET | /api/stats/top_user_agents | stats:read | Top 100 User Agents. |
| DELETE | /api/stats/all | ser | Reset all statistics. |

Templates (/api/templates)

| Method | Endpoint | Scope | Description |
|:---------|:-------------------------|:------------------|:----------------------------|
| GET | /api/templates | templates:read | List all templates. |
| GET | /api/templates/{name} | templates:read | Get template content. |
| PUT | /api/templates/{name} | templates:write | Create/Update template. |
| DELETE | /api/templates/{name} | templates:write | Delete template. |
| POST | /api/templates/refresh | templates:write | Reload templates from disk. |
| POST | /api/templates/test | templates:read | Test template syntax. |
| GET | /api/templates/preview | templates:read | Render template preview. |

Whitelist (/api/whitelist)

| Method | Endpoint | Scope | Description |
|:---------|:---------------------------|:------------------|:----------------------------------|
| GET | /api/whitelist/ip | whitelist:read | List whitelisted IPs. |
| POST | /api/whitelist/ip | whitelist:write | Add IP to whitelist. |
| DELETE | /api/whitelist/ip | whitelist:write | Remove IP from whitelist. |
| GET | /api/whitelist/useragent | whitelist:read | List whitelisted User Agents. |
| POST | /api/whitelist/useragent | whitelist:write | Add User Agent to whitelist. |
| DELETE | /api/whitelist/useragent | whitelist:write | Remove User Agent from whitelist. |

---

A special thank you to Nepenthes for the inspiration.

This project is *technically* a fork from the initial of
Chunchunmaru, which I majorly contributed to before the fork there
was made. This is the reason for the similarities in functions for template generation and other project structure.
However, I started from scratch and wrote Sarracenia from the ground up to fit my original for the project.

Gemini CLI was used to make the dashboard, as I am not an experienced frontend/html programmer, nor do I plan on being
one. If anyone would like to make their own, impro, I would be glad to accept it as a replacement.

项目信息

开发语言：Go

Star 数：12

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕A low resource anti-scraper tarpit inspired by Nepenthes.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。