MalwareWorld - System based on +500 blacklists to detect internet

项目说明

!MalwareWorld

MalwareWorld aggregates public threat-intel blacklists into a worldwide static dataset (IPs, domains, IP ranges, E.164 phone numbers, mobile app IDs, file hashes, TLS/code-signing certificate fingerprints, and known exploited CVEs) and publishes it as a website on GitHub Pages.

Website:

HackTricks tools:

Blacklists used (URLs):

What this repo contains

docs/: the GitHub Pages web UI (no backend).

scripts/: a generator that downloads blacklists and produces:

- text lists to download (suspiciousIPs.txt, suspiciousDomains.txt, suspiciousRanges.txt, suspiciousPhones.txt, suspiciousAppIds.txt, suspiciousFileHashes.txt, suspiciousSigningCertificates.txt, and knownExploited.txt)
- per-category downloads (type_<Category>_<indicator-kind>.txt)
- sharded JSON for exact lookups from the UI (IP/domain/range/phone/app ID/file hash/certificate fingerprint/CVE)
- maps + stats per category
- monthly archi (optional; see below)

Generation uses SQLite on disk to keep memory usage low.

Supported indicators and categories

The generated dataset supports these first-class searchable indicator kinds:

| Indicator | Accepted search input | Main download |
| --- | --- | --- |
| IPv4 address | Exact IPv4 address | suspiciousIPs.txt |
| IPv4 CIDR range | Exact CIDR | suspiciousRanges.txt |
| Domain | Domain or subdomain; URL input is reduced to its hostname | suspiciousDomains.txt |
| International phone number | E.164 or a formatted international number; country codes are ne | suspiciousPhones.txt |
| Mobile app ID | Android package name or iOS bundle ID | suspiciousAppIds.txt |
| Malware file hash | SHA-1 or SHA-256 | suspiciousFileHashes.txt |
| TLS or code-signing certificate | SHA-1 or SHA-256 fingerprint, with optional colon separators | suspiciousSigningCertificates.txt |
| Known exploited | C | knownExploited.txt |

The web search displays a current, clickable example for every indicator kind plus URL input. All
lookups are static and download only the relevant JSON shard. URLs are checked using their host;
domain lookup checks the exact hostname and then its parent domains.

Indicators can belong to more than one category. Current categories are:

BadReputation, Malware, Ransomware, KnownAttacker, Spammer, Phishing, Adware, DGA, HideSource, CryptoCurrencies, and Whitelist

PhoneSpamSmishing for worldwide spam-call, robocall, SMS-spam, and smishing numbers

EmailSpamPhishing for domains used in spam and phishing email

CommandAndControl for C2 domains and IP addresses

ScamFraud for scam and fraud infrastructure

DisposableEmail for temporary-email domains

BruteForce for brute-force source IP addresses

MobileSpyware for malicious Android/iOS infrastructure, app IDs, binaries, and signing certificates

MaliciousCertificate for malware C2 TLS certificates and manually malicious code-signing certificates

KnownExploited for the compact CISA catalog of with e of exploitation in the wild; entries marked by CISA as used in known ransomware campaigns also receive Ransomware

New sources are selected for clear malicious semantics, redistribution-compatible licensing,
continued maintenance, and bounded size. MalwareWorld deliberately avoids giant combined feeds when
they would mostly duplicate existing data or make the generated GitHub Pages and Release assets
unmanageable.

The NextDNS catalog review records decisions for all
84 entries checked on 12 September 2026. Se DNS feeds were added with source/license
metadata. Mixed ad, tracking and threat feeds are categorized as BadReputation or
Adware; membership alone does not confirm malware. Their strict DNS importer supports plain
domains, apex/subdomain wildcards and blackhole hosts records, and rejects browser-only filter rules.

Per-category text files use type_<Category>_<kind>.txt, where <kind> is domains, ips,
ranges, phones, apps, hashes, certificates, or . Empty combinations are still emitted as
valid text files so automated consumers can use a stable filename matrix.

Browser and client lookup shards

manifest.json describes the current shard patterns; clients should prefer it o
filenames. The default layout is:

| Indicator | Shard layout |
| --- | --- |
| Domains | domains_<first-character>.json |
| IPv4 addresses | ips_<first-octet-group>.json |
| IPv4 ranges | ranges_<first-octet-group>.json |
| Phone numbers | phones_<first-two-E.164-digits>.json |
| Mobile app IDs | apps_<first-character>.json |
| File hashes | hashes_<first-hex-character>.json |
| Certificate fingerprints | certificates_<first-hex-character>.json |
| Known exploited CVEs | <C>.json |

This keeps exact browser and library lookups bounded to one small shard instead of downloading a
complete list. File-hash input is checked against both the file-hash and certificate shards
because both use hexadecimal fingerprints.

GitHub Pages setup

1) Repo → Settings → Pages
2) Source: GitHub Actions
3) Run the workflow .github/workflows/publish-release-assets.yml once (or wait for the schedule).

The workflows publish a GitHub Release with the generated artifacts and also deploy the same artifacts under the Pages site at /data/ so the UI can fetch them without CORS issues.

Run locally (web UI + data generation)

1) Install:

npm ci

2) Generate the site data:

npm run generate:site

Useful env vars:

MW_LIMIT_BLACKLISTS=10 MW_CONCURRENCY=10 MW_OUTPUT_DIR=release-assets npm run generate:site

The parser does not impose a global per-blacklist match cap. Indi limit in
blacklists_list.js are still honored where a source intentionally selects only recent entries.

To avoid removing a malicious IP or domain after a transient feed omission, generation writes
retention-state.sqlite. Seed the next run with it:

MW_RETENTION_SEED_PATH=path/to/retention-state.sqlite npm run generate:site

An item absent from all of its successfully downloaded sources is retained after the first
absence and remo the second consecuti. Failed or skipped sources do not
advance that counter. A source that fails six consecuti downloads expires, and data
attributable only to that source is removed. Configure this threshold with
MW_RETENTION_MAX_SOURCE_FAILURES. The publish workflow downloads and decompresses the state
automatically from the latest release. Releases store it losslessly as
retention-state.sqlite.zst; it is excluded from the Pages artifact.

The same state database tracks source freshness. For GitHub-hosted lists, generation records the
latest commit affecting the feed path. Other feeds use HTTP Last-Modified metadata when
a, with content-hash changes as the uni fallback. The website exposes these
in an expandable table sorted by most recent update and flags repeatedly unchanged or failing
sources for review.

For flaky sources (common on CI):

MW_CONCURRENCY=12 MW_RETRY_COUNT=3 MW_RETRY_DELAY_MS=90000 npm run generate:site

Monthly archi of all non-whitelist items seen during the month):

MW_MONTHLY=1 MW_MONTHLY_DB_PATH=release-assets/monthly-YYYY-MM-archi npm run generate:site

Optional seed to merge the current run into an existing monthly archive:

MW_MONTHLY_SEED_PATH=path/to/monthly-YYYY-MM-archi

3) Serve the UI:

python3 -m http.server 4173 --directory docs

Option A (recommended local de into docs/data/ so the UI auto-detects it:

MW_OUTPUT_DIR=docs/data npm run generate:site

Open

Option B: serve release-assets/ separately and point the UI to it:

python3 -m http.server 4174 --directory release-assets

Open

4) Quick end-to-end smoke test (generates a small subset and outputs + HTTP fetches):

npm test

CLI lookup tools (Node + Python)

The same sharded JSON layout used by the UI can be queried from the command line. Scripts live in tools/lookup/.

Local data (uses docs/data/ if present):

node tools/lookup/lookup.js example.com

python3 tools/lookup/lookup.py 1.1.1.1

node tools/lookup/lookup.js '+34 600 000 000'

node tools/lookup/lookup.js com.example.suspicious

node tools/lookup/lookup.js 203.0.113.0/24

node tools/lookup/lookup.js

python3 tools/lookup/lookup.py <sha256-or-certificate-fingerprint>

node tools/lookup/lookup.js C

Remote release assets:

node tools/lookup/lookup.js example.com --base

python3 tools/lookup/lookup.py 1.1.1.1 --base

Note about removals

This repo pre exposed a Node.js “library” API (including external intelligence lookups). That code path is removed: MalwareWorld is now focused on static data generation + GitHub Pages.

Monthly archives (how it works)

MalwareWorld can keep a monthly union of all non-whitelist indicators and their associated URLs,
categories, and a IP geolocation. When enabled:

scripts/generate-site-data.js attaches/creates a monthly SQLite archive:

monthly-YYYY-MM-archi.

Each run merges the current dataset into the archive (non‑whitelist only).

It also generates monthly map and stats assets:

monthly-YYYY-MM-map_{Type}.geojson and monthly-YYYY-MM-stats_{Type}.json.

The workflow .github/workflows/publish-release-assets.yml publishes those files as a release

and publishes monthly-index.json with the month → release URL mapping. Pages exposes that
asset as /data/monthly/index.json.

The UI reads /data/monthly/index.json and shows a Month selector in the Maps section.
When a month is selected, the UI loads the monthly map/stats files from the release URL.
Because GitHub Releases do not provide CORS headers, the UI fetches monthly files through a
configurable CORS proxy (default: ). O via:

?corsProxy= or disable with ?corsProxy=none.

Downloadable files (from the web UI)

These files are published under /data/ on GitHub Pages and also as release assets:

blacklists.txt (all source blacklist URLs):

blacklists.json (run stats + blacklist metadata):

suspiciousIPs.txt:

suspiciousDomains.txt:

suspiciousRanges.txt:

suspiciousPhones.txt:

suspiciousAppIds.txt:

suspiciousFileHashes.txt:

suspiciousSigningCertificates.txt:

knownExploited.txt:

type_KnownExploitedities.txt:

type_Ransomware_.txt:

type_MaliciousCertificate_certificates.txt:

type_MobileSpyware_domains.txt:

type_MobileSpyware_ips.txt:

type_MobileSpyware_apps.txt:

type_MobileSpyware_hashes.txt:

type_MobileSpyware_certificates.txt:

type_Malware_apps.txt:

type_Malware_hashes.txt:

type_Malware_certificates.txt:

type_PhoneSpamSmishing_phones.txt:

type_EmailSpamPhishing_domains.txt:

type_DisposableEmail_domains.txt:

type_ScamFraud_domains.txt:

type_CommandAndControl_domains.txt:

type_CommandAndControl_ips.txt:

type_CommandAndControl_certificates.txt:

type_BruteForce_ips.txt:

type_BadReputation_domains.txt:

type_BadReputation_ips.txt:

type_BadReputation_ranges.txt:

type_Malware_domains.txt:

type_Malware_ips.txt:

type_Malware_ranges.txt:

type_Ransomware_domains.txt:

type_KnownAttacker_domains.txt:

type_KnownAttacker_ips.txt:

type_KnownAttacker_ranges.txt:

type_Spammer_domains.txt:

type_Spammer_ips.txt:

type_Spammer_ranges.txt:

type_Phishing_domains.txt:

type_Phishing_ips.txt:

type_Phishing_ranges.txt:

type_CryptoCurrencies_domains.txt:

type_CryptoCurrencies_ips.txt:

type_CryptoCurrencies_ranges.txt:

type_HideSource_domains.txt:

type_HideSource_ips.txt:

type_HideSource_ranges.txt:

type_Adware_domains.txt:

type_Adware_ips.txt:

type_Adware_ranges.txt:

type_DGA_domains.txt:

type_DGA_ips.txt:

type_DGA_ranges.txt:

Monthly archi (not in Pages; only in Releases):

monthly-YYYY-MM-archi.zst (lossless Zstandard-compressed SQLite): use the release base URL from

monthly-YYYY-MM-map_<Category>.geojson: use the release base URL from

monthly-YYYY-MM-stats_<Category>.json: use the release base URL from

Monthly archi notes

The Pages site always serves the latest dataset under /data/.

Monthly archives and monthly map/stats are kept in Releases (not in Pages) to avoid

unbounded Pages growth.

The monthly index is a release asset copied to /data/monthly/index.json during deployment, so

the UI can disco without calling the GitHub API or creating an extra Git commit/run.

License

The MalwareWorld code is MIT licensed. The aggregated blacklist data remains subject to each
upstream provider's license and usage terms; the source URL and a license metadata are
published in blacklists.json. Some feeds restrict commercial use or require attribution and
share-alike distribution, so downstream users must review those terms before redistributing the
generated datasets.

项目信息

开发语言：JavaScript

Star 数：158

技术要点

该项目属于病毒分析方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕System based on +500 blacklists to detect internet potencialy malicious hosts展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。