osint-d2 - Agentic OSINT toolkit. Autonomous identity triangu

项目说明

<p align="center">
<img src="assets/demo.png" alt="OSINT-D2 Demo" width="700" />
</p>

<h1 align="center">OSINT-D2</h1>

<p align="center">
<strong>Agentic OSINT • Identity Intelligence • Cogniti</strong>
</p>

<p align="center">
<a href="#installation"><img src=" alt="Python" /></a>
<a href="#installation"><img src=" alt="Poetry" /></a>
<a href="#ai-analysis--language"><img src=" alt="AI" /></a>
<a href="#-agent-mode--agentic-ai"><img src=" alt="Agentic AI" /></a>
<a href=" src=" alt="ScrapingAnt" /></a>
<a href="LICENSE"><img src=" alt="License" /></a>
</p>

---

OSINT-D2 is an ad open-source intelligence platform that transforms usernames and emails into structured identity dossiers. Powered by agentic AI and backed by ScrapingAnt's enterprise-grade proxy infrastructure, it delivers intelligence-grade identity correlation, cogniti, and breach analysis — all from a single CLI command.

Why OSINT-D2?

| Capability | What it does |
|---|---|
| 🔍 Multi-source correlation | Links usernames, emails, and deri across 30+ platforms in a single run |
| 🤖 Agentic AI | Autonomous in — the LLM decides which tools to invoke and pivots across identities automatically |
| 🧠 6-dimension cogniti | Identity, Geo-temporal, OCEAN psychology, Technical stack, Ideology, and OpSec analysis |
| 🛡️ Trust anchors | Define identity sources to filter false positives and focus on real targets |
| 🌐 ScrapingAnt proxy integration | Residential & datacenter proxies to bypass rate limits and anti-bot blocks worldwide |
| 📊 Premium PDF dossiers | Classified-style reports with identity cards, OCEAN profiles, OpSec risk matrices, and rendered Markdown |
| 💀 Breach exposure (HIBP) | Query Ha to find data breaches linked to discovered emails |
| 🌍 5 languages | English, Spanish, Portuguese, Arabic, and Russian — CLI, wizard, AI, and reports |
| 📦 Cross-platform binaries | Ship standalone executables for Linux, macOS, and Windows |

---

Table of Contents

---

Quick Start

!Command walkthrough showing the initial hunt in

---

Installation

Prerequisites

1. System dependencies (PDF export)

PDF dossiers are rendered , which requires nati:

<details>
<summary><strong>macOS (Homebrew)</strong></summary>

</details>

<details>
<summary><strong>Ubuntu / Debian</strong></summary>

</details>

<details>
<summary><strong>Fedora</strong></summary>

</details>

<details>
<summary><strong>Arch</strong></summary>

</details>

2. Install OSINT-D2

3. Configure en

Edit .env with your keys (or use the interacti):

En

| | Required | Description |
|---|---|---|
| OSINT_D2_AI_API_KEY | For AI | API key for DeepSeek, Groq, OpenRouter, or any OpenAI-compatible provider |
| OSINT_D2_AI_BASE_URL | No | Custom API endpoint (auto-detected from provider preset) |
| OSINT_D2_AI_MODEL | No | Model name o (default: deepseek-chat) |
| OSINT_D2_AI_TIMEOUT_SECONDS | No | API timeout (default: 120) |
| OSINT_D2_PROXY_API_KEY | For proxy | ScrapingAnt API key |
| OSINT_D2_PROXY_MODE | No | residential or datacenter (default: residential) |
| OSINT_D2_PROXY_COUNTRY | No | 2-letter country code for geo-targeted proxy |
| OSINT_D2_DEFAULT_LANGUAGE | No | Default language: en, es, pt, ar, ru |

---

🌐 Proxy: Powered by ScrapingAnt

<p align="center">
<a href="
<img src=" alt="ScrapingAnt" />
</a>
</p>

OSINT-D2's web scraping capabilities are powered by ScrapingAnt — an enterprise-grade proxy and web scraping platform trusted by thousands of de worldwide.

Why ScrapingAnt?

| Feature | Benefit for OSINT-D2 |
|---|---|
| 🏠 Residential proxies | Access social platforms like Instagram, X/Twitter, and LinkedIn that block datacenter IPs |
| 🌍 190+ countries | Geo-targeted requests to see region-specific content |
| ♻️ Automatic rotation | Fresh IPs on every request — no bans, no CAPTCHAs |
| 🛡️ Anti-bot bypass | Built-in Ja rendering and browser fingerprint rotation |
| ⚡ 99.9% uptime | Enterprise reliability for production OSINT workflows |
| 💰 Free tier a | Start in immediately — upgrade when you scale |

Setup

When configured, all HTTP requests are automatically routed through ScrapingAnt — no code changes needed. The proxy status is in the doctor diagnostics:

Per-run o

> 💡 Get your ScrapingAnt API key → scrapingant.com

---

Command Reference

| Command | Description |
|---|---|
| wizard | 🧙 Interacti workflow — choose mode, sources, language, trust anchors, and exports step by step |
| scan | ⚡ Lightweight username sweep across 18+ built-in social networks |
| scan-email | 📧 Email-centric scan with optional local-part username pivot |
| hunt | 🎯 Full pipeline: usernames + emails + Sherlock + site-lists + AI + PDF |
| agent | 🤖 Autonomous AI in — the LLM decides e |
| analyze | 🔄 Re-run AI analysis on a pre exported JSON dossier |
| doctor | 🩺 En diagnostics, AI setup wizard, connectivity checks |

Key flags

| Flag | A in | Description |
|---|---|---|
| --site-lists / --no-site-lists | hunt, wizard | Enable WhatsMyName-style data-driven site manifests |
| --sherlock / --no-sherlock | hunt, wizard | Download and execute the Sherlock manifest (~400 sites) |
| --strict / --no-strict | hunt, wizard | Apply defensi to reduce false positives |
| --export-json | hunt, agent, wizard | Write JSON dossier to reports/ |
| --export-pdf | hunt, agent, wizard | Write PDF/HTML dossier to reports/ |
| --format json | hunt, scan, scan-email | Machine-readable JSON output for automation |
| -l, --language | All | en, es, pt, ar, ru — affects CLI, AI, and reports |
| --breach-check | hunt, agent | Enable Ha breach queries |
| --trust | hunt, agent | Define trusted identity sources (repeatable) |
| --proxy / --no-proxy | All | O or disable ScrapingAnt proxy |

Usage examples

!Social footprint list for torvalds without Sherlock

---

🤖 Agent Mode — Agentic AI

Agent mode is where OSINT-D2 truly shines. Instead of manually specifying what to scan, the AI decides e autonomously: which usernames to scan, which emails to in, which URLs to fetch, and when to pivot to new leads.

Agent tools

The agent has access to 5 tools that it orchestrates autonomously:

| Tool | Description |
|---|---|
| scan_username | Scan 18+ social networks for a username (GitHub, X, Instagram, Twitch,
| scan_email | Scan Gravatar, PGP keyservers + pivot the local-part as a username |
| fetch_url | Scrape any URL and extract social links, emails, phone numbers, and metadata |
| breach_check | Query Ha for data breaches (requires --breach-check) |
| generate_report | Submit the final 6-dimension intelligence report (ends in) |

How it works

1. You provide an objective (username, email, or free text)
2. The LLM receives the objective + a tools
3. The LLM calls tools (e.g., scan_username("doble-2")) and recei results
4. Based on findings, the LLM pivots — discovers new aliases, emails, and linked profiles
5. When enough e is gathered, the LLM generates a structured 6-dimension cogniti

Agent examples

The wizard also supports agent mode — select "agent" when prompted for hunt type.

!Agent mode step-by-step in

---

🛡️ Trust Anchors

Trust anchors let you define sources of truth for an identity. When acti uses these anchors to automatically discard false positives — profiles on other networks that don't match the identity.

How it works

If you know that janedoe is the real Instagram handle, you can tell OSINT-D2:

OSINT-D2 will:
1. Extract the real name from trusted profiles (e.g., from Instagram bio)
2. Compare discovered profiles against the trusted identity
3. Discard profiles that belong to a different person (e.g., a Pinterest user named "John Smith" when the target is "Jane Doe")
4. Report how many false positives were filtered: 🛡️ Trust anchors discarded 26 false positive(s)

Supported formats

In the wizard, you can enter multiple anchors per line (comma or space separated):

!Trust anchors filtering false positives

---

AI Analysis & Language

OSINT-D2 produces structured intelligence reports using any OpenAI-compatible LLM. The AI generates a 6-dimension cogniti:

| Dimension | What it covers |
|---|---|
| 🆔 Identity | Real name, aliases, emails, phone numbers, demographics |
| 🌍 Geo-temporal | Location inference, timezone, activity patterns, sleep schedule |
| 🧠 Psychological (OCEAN) | Openness, Conscientiousness, Extra, Agreeableness, Neuroticism |
| 💻 Technical/Professional | Tech stack, seniority level, professional archetype |
| ⚖️ Ideology | Political/ethical leanings, , cultural affiliations |
| ⚠️ OpSec | Attack surface, social engineering susceptibility, exposure risks |

Supported AI pro

| Preset | Model | Best for |
|---|---|---|
| deepseek | deepseek-chat | Best quality/cost ratio (recommended) |
| groq | llama-3.1-70b- | Fast, high quality |
| groq-fast | llama-3.1-8b-instant | Ultra-fast, lower quality |
| openrouter | openai/gpt-4o-mini | OpenAI models |
| huggingface | meta-llama/Llama-3.1-8B-Instruct | Open-source models |

Key AI flags

Fallback behavior

Languages

Full support for 5 languages — CLI prompts, wizard, AI analysis, and PDF reports:

!CLI AI analysis panel screenshot

---

📄 Premium PDF Dossiers

OSINT-D2 generates classified-style intelligence dossiers in PDF and HTML format. The reports include:

Reports are sa reports/ with sanitized filenames.

> PDF rendering requires WeasyPrint system libraries. See Installation.

!PDF dossier co with the classified layout

---

💀 Breach Checking (HIBP)

OSINT-D2 queries Have I Been Pwned to find data breaches linked to discovered email addresses.

Results appear in:

---

🧙 Interacti

The wizard guides you through a complete in without memorizing flags:

The wizard prompts for:

1. Hunt type: username, email, both, or agent
2. Target(s): one or more usernames/emails
3. Language: English, Spanish, Portuguese, Arabic, Russian
4. Sherlock: enable/disable the ~400-site Sherlock manifest
5. Site lists: enable/disable WhatsMyName-style data-dri
6. Agent mode: max steps, breach checking
7. Proxy: enable/disable ScrapingAnt, mode, and country
8. Trust anchors: define identity sources
9. Exports: JSON and/or PDF to reports/

All features a via CLI flags are also accessible through the wizard.

---

Architecture

The CLI delegates business logic to the service layer, keeping commands thin and testable. Async operations use asyncio.run() for Typer compatibility. The architecture follows hexagonal/ports-and-adapters design — core logic has zero dependencies on CLI or HTTP libraries.

---

Packaging (PyInstaller)

Ship OSINT-D2 as a standalone executable — no Python required for end users.

<details>
<summary><strong>Linux build</strong></summary>

</details>

<details>
<summary><strong>macOS build</strong></summary>

</details>

<details>
<summary><strong>Windows build (PowerShell)</strong></summary>

</details>

Output: dist/osint-d2/ — ship this folder as a .zip or .tar.gz.

First-run setup for end users

GitHub Releases

This repo includes a GitHub Actions workflow that builds PyInstaller bundles for Linux, macOS, and Windows automatically. Push a tag to publish:

---

Troubleshooting

| Issue | Solution |
|---|---|
| WeasyPrint could not import... | Install system libs: brew install cairo pango gdk-pixbuf libffi (see Installation) |
| AI returns heuristic instead of full analysis | Check API key: osint-d2 doctor run. May also be rate limiting — wait 30s and retry |
| RateLimitError / HTTP 429 | OSINT-D2 auto-falls back to heuristic. Wait 30-90s or use a different provider |
| Instagram shows "blocked" | Enable ScrapingAnt residential proxy: --proxy residential --proxy-country us |
| Aborted when piping output | Use --format json and redirect to a file instead of piping Rich tables |
| PDF is empty/malformed | Run osint-d2 doctor run to WeasyPrint status |

Diagnostics

!Doctor diagnostics showing all checks passing

---

Disclaimer

OSINT-D2 is intended for lawful defensive and in use: incident response, fraud detection, brand protection, competiti, and personal security audits.

---

<p align="center">
<strong>Built with ❤️ by <a href="
<br />
Proxy infrastructure powered by <a href="
</p>

<p align="center">
Released under the <a href="LICENSE">MIT License</a>
</p>

项目信息

开发语言：Python

Star 数：269

技术要点

该项目属于红蓝攻防方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Agentic OSINT toolkit. Autonomous identity triangulation, 6-dimension cognitive profiling, breach analysis. CLI-first. 30+ platforms. 5 languages.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。