AnchorWeight - Defensive reverse proxy and crawler-identification

项目说明

AnchorWeight v2.9.2

Self-hosted adapti defense with Proof-of-Crawl, Bot DNA, silent quarantine, re protection, in tooling, and shadow-first adapti.

> Let bots identify themselves.

![CI]()
![CodeQL]()

AnchorWeight is a self-hosted defensi proxy and crawler-identification platform built around a simple idea:

Do not rely only on guessing whether a is automated. Gi automation opportunities to prove it through observable behavior.

AnchorWeight combines cryptographically signed procedural lures, robots-blackhole detection, Bot DNA e, client-en consistency checks, adapti, traffic telemetry, and bounded quarantine controls. A client that successfully tra the signed lure to the configured depth establishes Proof-of-Crawl.

AnchorWeight can observe that e in Shadow Mode or, when explicitly configured for enforcement, temporarily isolate con crawler traffic behind deterministic inert HTTP 200 decoy responses while normal traffic continues to the protected origin.

!AnchorWeight Dashboard

> The maze is not the weapon. The maze is the test.

AnchorWeight is intentionally not an infinite tarpit, bandwidth sink, huge-file generator, connection-exhaustion system, or offensi. Its design emphasizes bounded defensi, explainable e, re enforcement, and conser defaults.

Current v2.9.2 capabilities

Proof-of-Crawl and deception

Signed procedural lure — cryptographically deri must be tra sequentially to establish Proof-of-Crawl.

Robots Blackhole — hidden robots-disallowed paths provide an independent crawler-behavior signal.

Independent canary branches — signed branches preser lineage and help distinguish repeated or distributed crawler behavior.

Adapti — observable behavior can dri recommendations such as shadow branches, signed decoys, depth extension, and rate friction.

Adapti Engine — recommendations pass through inter allowlists, cooldowns, prior effecti, and deterministic fallback selection.

Adapti remains shadow-first and does not independently o explicit quarantine or authentication policy.

Bot DNA and e

Bot DNA 2.0 separates automation indicators, signed-trap e, crawler identity, client-en obser, and signal limitations.

Client En Consistency adds pri display and consistency e.

Impossible or contradictory client-en can contribute supporting e without treating unusual but valid devices as bots.

Optional trusted JA4-compatible obser can be accepted from a trusted TLS-terminating proxy.

JA4 metadata is informational and is not treated as standalone proof of malicious automation.

Bounded e histories support in without retaining unnecessary raw client data.

Proxy Identity Safety — v2.9.2

AnchorWeight detects re deployments where Node sees only a shared loopback or private proxy hop instead of the real client identity.

When client identity is degraded:

Bot DNA client-scoped quarantine is suppressed.

Score enforcement is suppressed.

Gateway IP rules are suppressed.

Gateway per-client rate enforcement is suppressed.

Ordinary proxy and route telemetry continues to operate.

Signed lure tra remains a through session-only e identities.

Unrelated are not merged into one shared Bot DNA profile.

Caller-supplied client-address headers are stripped before traffic is forwarded to the protected origin.

Identity health is exposed through /live, /ready, and authenticated statistics.

See CLIENT-IDENTITY-SAFETY.md.

Silent Quarantine

Con crawler traffic can be temporarily isolated behind deterministic inert HTTP 200 responses while legitimate traffic continues to the real application.

Quarantine is:

bounded;

temporary;

re;

separately controlled from detection;

disabled when AnchorWeight cannot safely establish client identity.

AnchorWeight intentionally avoids permanent IP bans based solely on heuristic classifications.

In and operations

In Console 2.0 provides bounded timelines, campaign relationships, e reports, analyst reviews, and notes.

Li Obser reports route traffic, status distributions, upstream failures, and approximate p50/p95/p99 latency.

Optional authenticated WebSocket telemetry supports li .

Webhook Alerts can report configured upstream failures, 5xx thresholds, and circuit transitions using signed HTTPS payloads.

Setup Wizard and npm run doctor help deployments.

Declarati configuration supports , import, export, rollback, and operator-controlled changes.

Backup, restore, state , migration, and release- workflows support production operation.

Gateway and application protection

AnchorWeight also includes optional:

IPv4/IPv6 CIDR access policies;

path-specific gateway rate limits;

HTTP Basic authentication;

API-key authentication;

multi-origin routing;

upstream health probes;

circuit breakers;

safe GET/HEAD fallback routing;

maintenance responses;

gateway telemetry.

These controls remain separate from crawler e so operators can choose which capabilities to enable.

Persistence and distributed intelligence

JSON remains the default persistence backend.

Optional SQLite persistence is a on supported Node.js .

The distributed-intelligence foundation can send signed, pri e to a separately deployed AnchorWeight hub.

The hub does not automatically create global identities or remotely enforce quarantine.

Safe-by-default philosophy

AnchorWeight deliberately separates:

A suspicious signal does not automatically become a block.

The recommended operating model remains:

This matters because NAT gateways, VPNs, mobile networks, enterprise proxies, privacy relays, and hosting infrastructure can cause multiple unrelated to share network characteristics.

Security boundary

For whole-site Silent Quarantine to provide meaningful protection, clients must not be able to bypass AnchorWeight and reach the protected origin directly.

If the origin remains directly reachable from the public Internet, AnchorWeight cannot enforce the intended re security boundary.

Current release documentation

Proxy Identity Safety

Adapti

Adapti

Client En Consistency

Bot DNA 2.0

In Console 2.0

Li Obser

Alerts and Webhooks

Installation and Operations

Architecture

Threat Model

Changelog

---

Historical release capabilities

v2.3 Upstream Resilience (opt-in)

Per-origin, process-local circuit breakers with independent failure thresholds and cooldowns.

Optional bounded HEAD probes (two at a time), including configured fallback origins.

Optional exact-match, allowlisted fallback mapping in a private 0600 JSON file.

Only NEW bodyless GET/HEAD requests switch to fallbacks while a primary circuit is open. POST/PUT/PATCH/DELETE are ne or rerouted.

HTTP 5xx responses contribute to opening a circuit but are returned unmodified; an in-flight failure is ne.

Per-path breaker state and fallback counts appear in the authenticated dashboard without origin URLs or secrets.

Disabled by default; no state-schema migration or runtime dependencies. See UPSTREAM-RESILIENCE.md.

v2.1 Unified Gateway Configuration & Route Wizard

Optional, private JSON or safe-subset YAML config for the proxy switch, appro origin and up to 16 path routes. No additional npm dependencies or state migration.

Authenticated Setup Wizard route editor, bounded upstream HEAD , CSRF-protected li and pre rollback; file writes remain disabled until explicitly enabled.

Legacy en, v1.7 operator settings and v1.9 route files remain supported with declarative mode off; conflicting modes fail if accidentally enabled together.

Secrets, crawler enforcement, gateway rate/IP rules and health policy are not editable through the declarative file. See DECLARATI.md for migration and rollback.

v2.2 Application Authentication & Security Policies

Opt-in per-path HTTP Basic Auth and API keys for proxied applications; independent of dashboard authentication and Bot DNA.

Offline credential CLI produces scrypt password or one-time 256-bit API keys; private on-disk credentials file is fail-closed when enabled.

Longest segment-prefix policy selection, bounded async hashing, HTTP 401 challenges, stripped credentials before forwarding, and authenticated policy-only dashboard counters.

Existing per-path gateway rate-limit controls continue to work independently. No new runtime dependencies or state migration.

See APPLICATION-AUTH.md for installation, rotation, trust boundaries and rollback.

v2.4 Li Obser

In-memory, per-process, bounded per-route response metrics with approximate p50/p95/p99 latency and sanitized transport-error counts.

Redacted last 50 proxied-response events in authenticated stats; no raw paths, IPs, authorization data or upstream URLs.

Optional, read-only WebSocket event stream with one-use, 30-second tickets issued by the authenticated admin API.

Disabled by default. No new runtime dependencies, state-schema changes or required routing changes.

Installation and operator guide.

v2.0 Integrated Defensi Platform

The v2.0 milestone consolidates local crawler e, opt-in multi-origin gateway controls, operator setup, telemetry, and optional authenticated intelligence reporting. The hub is an e collector, not global crawler identity or a remote enforcement system. Legacy JSON is still the default; SQLite requires Node 22.13+.

Source integrity: npm run release:check writes the manifest; npm run release: checks without changing files and is used by CI.

Operator configuration: bounded (16 KiB) descriptor-backed local settings reads, refusing symlinks where the platform supports O_NOFOLLOW.

v1.9's pre uncommitted test/multi-origin.integration.test.js is included in the v2.0 release.

No automatic proxy enabling, new runtime dependencies, or state schema changes.

See .md for the upgrade and rollback process and limitations.

v1.7 Operator Setup Wizard & Traffic Telemetry

Authenticated /setup.html wizard with an explicit opt-in before browser

changes can apply; no en rewriting or arbitrary origin access.

cPanel-approved, exact-match origins, ser configuration ,

bounded origin HEAD probe, CSRF-protected apply, and private atomic local
settings for restart persistence. Credentials and enforcement policy stay in
cPanel. Disabling the feature restores en routing.

New bounded per-process telemetry reports request totals, final status

classes, acti/upstreams, mean response time, upstream first-header
latency and fi traffic trends. No request identities are stored.

Existing JSON/SQLite schema v5, cPanel startup compatibility, distributed

e reporting and dashboard in features remain unchanged.

See OPERATOR-GATEWAY.md before enabling proxy mode.

v1.8 Gateway Protection & Reliability

Optional, independent access-rule and per-IP rate-limit gates for proxied traffic only, disabled by default.

IPv4/IPv6 CIDR allow and deny rules, shadow-mode counters and optional bounded path-specific request limits.

Optional bounded upstream HEAD health probes and opt-in 503 maintenance responses for transport errors on GET/HEAD.

Authenticated dashboard displays access, rate and upstream status without leaking IPs or paths.

No new runtime dependencies or state migration; operator guide.

v1.9 Multi-Origin Routing

Optional private route-file mapping of approved URL path prefixes to up to 16 distinct origins.

Existing AW_ORIGIN_URL remains the default fallback, including via the existing setup wizard.

Route table is at startup and is restart-only; no external HTTP request can select an arbitrary upstream.

Authenticated dashboard displays path-only route status; /ready checks the default and each configured route.

Existing client e and JSON/SQLite state schema v5 are unchanged.

See Multi-origin routing and rollback.

v1.6 Distributed Intelligence Foundation

Separate, authenticated e hub, disabled by default.

Explicit per-site registration; independently configured federation secret.

Private signed paths and internal IP keys ne the federation boundary.

Locally e summaries, bounded asynchronous queue, and hub ingestion deduplication.

Sensor dashboard displays distributed publisher queue/sent/drop status.

No automatic cross-site identity inference or remote enforcement.

Distributed intelligence deployment guide.

v1.5 In Console 2.0

Searchable, bounded Bot DNA and campaign in timelines and relationship e.

Authenticated JSON and plain-text case reports with signed-path redaction.

Explicit, CSRF-protected analyst reviews separate from enforcement policy.

In Console 2.0 guide.

Security Engineering E

AnchorWeight is designed as a bounded defensi, not as a resource-exhaustion mechanism. Its security model emphasizes observable crawler behavior, explicit trust boundaries, re enforcement, and conser defaults.

| Area | E |
| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| Threat model | THREAT-MODEL.md documents protected assets, ad, trust assumptions, controls, abuse cases, and residual risks |
| Architecture / trust boundaries | ARCHITECTURE.md documents request flow, origin isolation, administrati, crawler e, and enforcement boundaries |
| Safe rollout | Shadow Mode is enabled by default; proxy and score enforcement remain disabled until an operator deliberately enables them |
| Crawler e | Proof-of-Crawl uses signed sequential paths rather than treating ordinary browsing behavior as automatic proof of abuse |
| Good-bot | Claimed Google/Bing identities are using reverse and forward DNS rather than trusted from User-Agent strings alone |
| Bounded enforcement | Con traffic recei inert HTTP 200 decoys rather than unbounded files, connection exhaustion, or infinite resource consumption |
| State integrity | Schema compatibility checks and explicit migrations fail closed on corrupt, unsupported-old, or future- state |
| Release gates | Tests, npm audit, release , CodeQL, container builds, configuration checks, and runtime diagnostics support production readiness |

Safe-by-Default Operating Model

AnchorWeight intentionally separates detection e from enforcement policy.

A crawler classification, beha score, or campaign correlation does not need to become an automatic permanent block. This matters because shared NAT, VPN, mobile, and enterprise egress addresses can represent multiple unrelated users.

The preferred operating model is therefore:

* obser enforcing;
* use layered e rather than one heuristic;
* distinguish good bots from un identities;
* make quarantine temporary and re;
* keep the real origin private;
* preser and audit history;
* fail safely when configuration or state integrity cannot be established.

Security Boundary

For Silent Quarantine to provide meaningful isolation, the protected origin must not remain directly reachable from the public Internet.

If clients can bypass AnchorWeight and reach the origin directly, the reverse proxy cannot enforce the intended boundary.

v1.0.0 highlights

Signed, sequential Proof-of-Crawl procedural lure.

Whole-site fixed-origin HTTP/HTTPS reverse proxy.

Silent Quarantine with deterministic inert HTML/JSON responses.

Pri Bot DNA beha profiles.

Google/Bing crawler claims using reverse + forward DNS confirmation.

Distributed-crawler Bot Campaign correlation using shared signed-canary e.

In dashboard with event explorer, campaign map, timelines, and explicit allow/quarantine policy controls.

Zero-runtime-dependency operator CLI.

Named Shadow/Production configuration profiles.

Log rotation/retention, backup/restore, e reports, li/readiness, and graceful shutdown.

Bearer-only admin authentication by default, CSRF-protected mutations, admin rate limiting, audit logging, and request-body bounds.

Formal configuration schema v1 and state schema v5 with supported migrations.

state-check, migrate, and release-check upgrade/release gates.

Expanded unit, security, integration, persistence, and proxy test co.

GitHub Actions CI, npm audit, CodeQL, and container-build .

Non-root multi-stage Docker image plus cPanel, systemd, Nginx/Apache, and Docker deployment examples.

v1.1.0 highlights

Robots Blackhole detection using a hidden link explicitly disallowed by robots.txt.

Blackhole e feeds the existing Bot DNA and temporary quarantine system.

good bots and explicit allow policies remain exempt.

Shadow Mode can measure blackhole activity before enforcement.

Dashboard metrics for blackhole hits, con, and would-quarantine events.

Existing signed sequential Proof-of-Crawl remains the stronger deep-tra signal.

v1.2.0 highlights

Optional SQLite persistence on Node 22.13+, with zero additional npm runtime dependencies.

Existing JSON deployment remains the default; Node 20 installations are not forced to upgrade.

Transactional JSON-to-SQLite import; the legacy JSON file remains unchanged for rollback.

Changed-row persistence and SQLite-aware backup, restore, inspection, and reporting.

Local benchmark script, expanded regression tests, and storage documentation.

Start with SQLITE-MIGRATION.md before enabling SQLite on a live site.

Bot DNA 2.0 (v1.4.0)

See BOT-DNA-2.md for explainable e and optional trusted TLS fingerprint ingestion. The additional obser do not alter risk points or enforcement defaults.

v1.3.0 highlights

Independently signed canary branches: each displayed lure link has its own authenticated token; branch zero remains backward compatible with v1.2 signed paths.

Session lineage: bounded event records capture branch, depth, elapsed time and opaque session IDs without logging signed URL tokens or raw IPs.

E ledger: recent structured obser appear on the existing authenticated in API and persist with Bot DNA profiles in both JSON and SQLite.

Repeat-offender context: profile offense counts and last-offense timestamps survi through the existing state backends.

Stronger campaign attribution: cross-client campaign correlation requires a valid signed chain, and repeated use of the same session cannot repeatedly inflate campaign confidence.

Configurable traps: AW_CANARY_BRANCH_COUNT (2–12) and AW_CANARY_ (default true). Shadow Mode and manual allow policies are unchanged.

See CANARY-E.md for rollout, privacy boundaries, limitations and rollback.

Architecture

For the detailed module/trust-boundary description, see ARCHITECTURE.md and THREAT-MODEL.md.

Safe defaults

The recommended rollout is Shadow Mode first, inspect real traffic, legitimate automation/good bots, then deliberately enable Proof-of-Crawl enforcement. Score-only enforcement remains opt-in.

Quick start

Requirements: Node.js 20+.

Then visit:

For cPanel hosting, see deploy/cpanel/README.md; normal setup uses Setup Node.js App with app.js as the startup file and en configured in the cPanel UI.

CLI

See CLI.md for flags and examples.

Upgrading from v0.8/v0.9

Back up before migration:

Supported old state are migrated sequentially to the v1.0 state schema. A corrupt, unsupported-old, or future- state file causes startup to fail instead of silently starting with empty protection history.

Production release gate

The real origin must be private or otherwise inaccessible from the public Internet for whole-site Silent Quarantine to be meaningful.

Testing and security automation

npm test runs isolated module/security tests plus the existing end-to-end proxy, persistence, Proof-of-Crawl, Bot DNA, good-bot, campaign, CLI, and operations regression tests.

GitHub workflows provide:

Node.js 20/22/24 test matrix.

npm audit --audit-level=high software-composition gate.

release/source-manifest .

CodeQL Ja/TypeScript scanning.

Docker image build .

AnchorWeight has no runtime npm dependencies in v1.2.0, but the audit gate is kept in place so future dependency additions do not silently bypass SCA.

Documentation

ARCHITECTURE.md — modules, request path, trust boundaries.

THREAT-MODEL.md — assets, ad, controls, residual risks.

CONFIGURATION.md — configuration reference and precedence.

CLI.md — operator command/flag reference.

OPERATIONS.md — profiles, retention, backup/restore, lifecycle.

DEPLOYMENT.md — proxy and hosting deployment guidance.

SECURITY.md — security posture and reporting guidance.

RELEASE.md — v1 release/upgrade checklist.

CHANGELOG.md — release summary.

Important limitations

AnchorWeight v1.0 supports a single process / single local state writer. The JSON persistence layer is not a distributed datastore. Shared NAT/ IPs can represent multiple users, so AnchorWeight uses temporary quarantine and layered e rather than permanent IP bans. Good-bot DNS failure means un, not automatically malicious. Bot Campaign correlation is informational by default and does not automatically quarantine an entire cluster.

License

MIT

Optional distributed intelligence (v1.6.0)

Run node bin/intelligence-hub.js as a separate Node application with its own
credentials and storage; see DISTRIBUTED-INTELLIGENCE.md.
The normal sensor is unchanged by default (AW_INTELLIGENCE_ENABLED=false).
The hub aggregates site-scoped e but does not automatically associate
profiles across different sites or apply policies. These are local attestations,
not independent proof of common operator identity.

项目信息

开发语言：JavaScript

Star 数：3

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Defensive reverse proxy and crawler-identification platform with Proof-of-Crawl, safe Shadow Mode, bounded quarantine, audit logging, and security release gates.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。