soc-insight-Sid - SOC-Insight — A defensive security investigation w

项目说明

SOC-Insight

AI-assisted security in & threat hunting workbench.

SOC-Insight is a local, self-contained lab that simulates the core workflow
of a SOC (Security Operations Center) analyst: take raw security events,
correlate them, detect suspicious behavior with explainable rules, score
risk transparently, extract indicators of compromise, map behavior to
MITRE ATT&CK, build an incident timeline, and produce a professional
incident report — all without depending on a paid external API.

This is an educational / portfolio security lab, not a production SIEM,
antivirus, EDR, or SOC replacement. See Limitations below.

---

Table of contents

O

Architecture

Features

Installation

Usage

Dataset description

Detection engine

Scoring methodology

MITRE ATT&CK integration

Security considerations

Testing

Limitations

Future impro

---

O

The pipeline follows one direction, matching the mental model of an actual
SOC:

Every stage is a plain Python module with no hidden magic: the same code
path backs both the CLI (main.py) and the web UI (app.py).

Architecture

See docs/architecture.md for the full breakdown.
In short:

Detection logic is ne in the UI layer; the frontend only calls
the JSON API in app.py, which in turn only calls pipeline.py.

Features

Multi-format ingestion (JSON, NDJSON, CS key=value logs)

Tolerant normalization with reported (ne) parse issues

Entity + time-window correlation

7 modular, independently testable detection rules co brute force /

account compromise, unusual logins, encoded PowerShell, persistence,
lateral mo, data collection / exfiltration, and account
manipulation

Fully transparent, additive risk scoring with a breakdown per

detection (no black-box "AI confidence" number)

IOC extraction with honest risk labeling (OBSERVED / SUSPICIOUS /

ASSOCIATED_WITH_INCIDENT — never a claim of confirmed malice)

Local MITRE ATT&CK dataset scoped to what the rules can actually detect

Incident management with status, classification, analyst notes, and

per-e classification (CONFIRMED / SUSPICIOUS / BENIGN / UNKNOWN)

Safe threat-hunting query language (field = "value", contains,

AND / OR — no eval, no injection path)

AI-assisted analysis layer (incident summary, e explanations,

in suggestions, executive & technical summaries) built on a
pluggable AIProvider interface; the default implementation is
template-based and requires no external API

Professional Markdown incident report generation

Dark, dense, professional in UI: O, Incidents,

Incident In, Event Explorer, Threat Hunting, Detection
Rules, MITRE ATT&CK, IOC Explorer, Reports, Settings

Upload your own synthetic log file and re-run the whole pipeline live

Installation

Requires Python 3.10+.

Usage

Web UI

Then open . The demo en (5 synthetic
incidents) loads automatically on first launch.

CLI

Importing your own data

From the Settings page in the UI, or via POST /api/ingest
(multipart form field file), you can upload a .json, .ndjson,
.csv, or .log/.txt file. It is parsed with the same safe parsers as
the bundled data, normalized, and immediately run through the full
detection pipeline. Files are capped at 10 MB and are always treated as
inert data — nothing in a log file is e.

Dataset description

Five safe, coherent, synthetic attack scenarios ship in data/scenarios/,
one per required format family (see
docs/demo-scenarios.md for the full narrative
of each):

| File | Format | Scenario |
|---|---|---|
| scenario_a_bruteforce.json | JSON | Brute force -> account compromise |
| scenario_b_powershell.ndjson | NDJSON | Encoded PowerShell -> C2-style connection |
| scenario_c_persistence.csv | CS drop -> scheduled task persistence |
| scenario_d_lateral_mo.log | text log | Remote auth -> remote service execution |
| scenario_e_exfiltration.json | JSON | File collection -> archive -> simulated outbound transfer |

All data is fabricated. IPs used to represent "external" infrastructure
are arbitrary, non-attributable addresses chosen only to be outside
private/reser; no real organization or indi is
represented.

Detection engine

See docs/detection-engine.md. Rules combine
indi event signatures, thresholds, and temporal/entity correlation
rather than keyword matching alone — e.g. 5 failed logins alone score low,
but 20 failed logins followed by a successful login and pri
activity score CRITICAL.

Scoring methodology

score = base_points_for_the_rule + sum(e), clamped
to [0, 100], mapped to a se band:

| Score | Se |
|---|---|
| 0–24 | INFO |
| 25–49 | LOW |
| 50–74 | MEDIUM |
| 75–89 | HIGH |
| 90–100 | CRITICAL |

Every point in the breakdown is labeled and shown in the UI's Detections
tab — there is no hidden or unexplainable component.

MITRE ATT&CK integration

A small, local dataset (src/soc_insight/mitre/dataset.py) covers only
the techniques the bundled rules can produce e for (see
docs/detection-engine.md for the full rule -> technique table). No
external ATT&CK service is called.

Security considerations

See docs/security.md for the full write-up.
Highlights: uploaded logs are always treated as data, ne;
strict file size and record-count caps; safe JSON/CS via the
standard library only; a small, eval-free threat-hunting query
grammar; no secrets or API keys anywhere in the codebase; en
for the one optional integration point (a local model
endpoint).

Testing

The suite (60+ tests) covers parsing/malformed input, detection rules
(including explicit false-positi cases), scoring, IOC
extraction, MITRE mapping, incident creation, threat-hunting search,
report generation, and security-relevant behavior (size limits, no
code execution, no query-injection path).

Limitations

SOC-Insight is a security laboratory / educational system. It is
explicitly not:

a production SIEM

an antivirus product

an EDR (Endpoint Detection and Response) agent

a malware scanner

a replacement for a real SOC or a licensed security professional

State is in-memory only (a restart clears e except the bundled
demo data); there is no authentication, multi-user support, or
persistent database, because this is a single-analyst local lab, not a
deployable product. Detection rules are intentionally few and are scoped
to the bundled scenarios rather than aiming for broad real-world
co. IOC "risk" labels reflect only what the local rules observed —
there is no external threat-intelligence reputation lookup.

Future impro

Persistent storage (SQLite) so incidents survive a restart

Additional detection rules (DNS tunneling, credential dumping signatures, etc.)

A pluggable rule-definition format (YAML/JSON) so rules can be added

without editing Python

Multi-analyst support with per-user attribution on notes/status changes

A real local-model integration example (e.g. a small local LLM server)

for richer narrati

项目信息

开发语言：Python

Star 数：1

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕SOC-Insight — A defensive security investigation workbench that correlates security events, detects suspicious behavior, extracts IOCs, maps activity to MITRE ATT&CK, and assists analysts in investigating simulated security incidents.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。