prompt-sentinel - A defensive security microservice for detecting an

项目说明

🛡️ PromptSentinel

![CI/CD Pipeline]()
![Docker Pulls]()
![Python 3.11+]()
![FastAPI]()
![License: ELv2]()

A production-ready defensi microservice for detecting and mitigating prompt injection attacks, PII exposure, and other security threats in LLM-based systems. PromptSentinel provides real-time protection using multi-layered detection strategies with sub-100ms response times.

🚀 Key Features

Multi-Layer Detection: Combines heuristic patterns, LLM classification, and PII detection

Intelligent Routing: Automatically routes prompts to optimal detection strategy based on complexity

Flexible Authentication: Multiple deployment modes - no auth (sidecar), optional (mixed), or required (SaaS)

ML Pattern Discovery: Self-learning system that discovers new attack patterns from real threats

API Usage Monitoring: Track costs, usage, and performance with budget controls and alerts

Multi-Provider LLM Support: Anthropic (Claude), OpenAI (GPT), and Google (Gemini) with automatic failover

PII Protection: Detects and redacts 15+ built-in PII types plus custom organization-specific patterns

High Performance: 98% faster responses with Redis caching (12ms )

WebSocket Support: Real-time streaming detection for continuous monitoring

A/B Testing: Built-in experimentation framework for optimizing detection strategies

ML Pattern Discovery: Automated discovery of new attack patterns using clustering and machine learning

Rate Limiting: Token bucket algorithm with per-client and global limits

Flexible Deployment: Works with or without Redis, Docker, Kubernetes ready

Production Ready: OpenTelemetry monitoring, structured logging, health checks

Configurable Detection Modes: Strict, moderate, or permissi based on your needs

Format : Encourages secure prompt design with role separation

Enterprise Compliant: SOC 2, FINRA, GDPR/CPRA ready with audit logging

📋 Table of Contents

Quick Start

Installation

API Documentation

Understanding Confidence Scores

Configuration

WebSocket Support

Redis Caching

PII Detection

Custom PII Rules

Detection Strategies

De

Deployment

Performance

Security

Roadmap

- Stubs and Experimental Modules

🚀 Quick Start

Using Docker Hub (Fastest)

Using Docker (Build Locally)

Using Makefile

We provide a comprehensi with 40+ commands:

🔧 Installation

Prerequisites

Python 3.11+ or Docker

At least one LLM provider API key (Anthropic, OpenAI, or Google)

Optional: Redis for caching

Local De Setup

📚 API Documentation

Core Endpoints

#### POST /api/v1/detect - Simple Detection

Response:

#### POST /api/v1/detect - Ad Detection with Role Support

#### POST /api/v1/analyze - Comprehensi
Provides detailed analysis including PII detection, format , and security recommendations.

#### POST /api/v1/detect/intelligent - Intelligent Routing Detection (NEW)
Automatically analyzes prompt complexity and routes to optimal detection strategy:

Features:

Automatic complexity analysis (trivial → critical)

Adapti strategy selection

Performance optimization for simple prompts

Comprehensi for complex/risky prompts

Risk indicator detection (encoding, role manipulation, etc.)

#### GET /api/v1/routing/complexity - Complexity Analysis
Analyze prompt complexity without detection:

#### GET /api/v1/routing/metrics - Routing Metrics
Get intelligent routing performance statistics.

#### GET /api/v1/metrics/complexity - Complexity Metrics
Comprehensi complexity metrics and distribution:

Monitoring & Budget Control

Configuration: Budget and rate limits are now configurable :

#### GET /api/v1/monitoring/usage - API Usage Metrics
Track API usage, costs, and performance:

Returns:

Request counts and success rates

Token usage and costs

Provider breakdown

Performance metrics

#### GET /api/v1/monitoring/budget - Budget Status
Monitor spending against configured limits:

Features:

Hourly, daily, and monthly budget tracking

Automatic alerts at 75% and 90% thresholds

Cost projections

Optimization recommendations

#### GET /api/v1/monitoring/rate-limits - Rate Limit Status
Check current rate limiting status:

#### GET /api/v1/monitoring/usage/trend - Usage Trends
Historical usage data for analysis:

#### POST /api/v1/monitoring/budget/configure - Configure Budget
Dynamically update budget limits:

#### GET /api/v1/cache/stats - Cache Statistics

API Documentation UI

When running, visit:

Swagger UI:

ReDoc:

Understanding Confidence Scores

PromptSentinel uses confidence scores to indicate the certainty of its detection . Understanding these scores is crucial for implementing appropriate response strategies.

Confidence Score Scale

Confidence scores range from 0.0 to 1.0:

0.90 - 1.00: Very high confidence in the

0.70 - 0.89: High confidence in the

0.50 - 0.69: Moderate confidence in the

0.30 - 0.49: Low confidence in the

0.00 - 0.29: Very low confidence in the

What Confidence Means

The confidence score represents how certain PromptSentinel is about its , NOT the se of the threat. The same confidence value (e.g., 0.95) can appear with different :

#### Example 1: High Confidence ALLOW (Safe Content)
Request:

Response:

Interpretation: The system is 95% confident this content is SAFE.

#### Example 2: High Confidence BLOCK (Malicious Content)
Request:

Response:

Interpretation: The system is 92% confident this content is MALICIOUS.

#### Example 3: Low Confidence FLAG (Uncertain)
Request:

Response:

Interpretation: The system is only 45% confident about this . The content mentions "system prompts" which could be innocent or malicious depending on context. Manual review recommended.

Quick Reference: Confidence

| Confidence | | Meaning | Recommended Action |
|------------|---------|---------|-------------------|
| 0.95 | allow | 95% certain content is safe | Process normally |
| 0.95 | block | 95% certain content is malicious | Block immediately |
| 0.45 | allow | Uncertain, but leaning toward safe | Monitor/log for analysis |
| 0.45 | flag | Uncertain, possibly suspicious | Manual review recommended |
| 0.45 | block | Uncertain, but concerning patterns | Consider blocking or review |

Confidence by Detection Method

Different detection methods contribute different confidence levels:

1. Heuristic Detection:
- Pattern matches: 0.7 - 0.95 confidence based on pattern strength
- No patterns matched: 0.95 confidence for benign content

2. LLM Classification:
- Typically provides 0.6 - 0.9 confidence based on model certainty
- Lower confidence for ambiguous or context-dependent content

3. Combined Detection:
- When both methods agree: Confidence boosted by up to 0.1
- When methods disagree: Lower confidence, typically 0.4 - 0.6

Using Confidence in Your Application

Confidence in Detection Modes

Detection mode affects confidence thresholds for :

Strict Mode: Lower thresholds, more likely to flag/block at lower confidence

Moderate Mode: Balanced thresholds

Permissive Mode: Higher thresholds, requires higher confidence to block

🔐 Authentication & API Keys

PromptSentinel supports flexible authentication to fit different deployment scenarios:

Recommended Authentication Modes

| Mode | Use Case | Description |
|------|----------|-------------|
| none | Sidecar/Internal | No authentication required - for trusted en |
| optional | De/Mixed | API keys improve rate limits but aren't required |
| required | Public Access/Sensitive Info | API keys mandatory for all requests |

Deployment Scenarios

#### 1. Sidecar Container (No Auth)

#### 2. Internal Microservice (Optional Auth)

#### 3. Public Deployment (Required Auth)

API Key Management

#### Generate API Key (Admin)

#### Using API Keys

Rate Limiting

PromptSentinel includes configurable rate limiting to prevent abuse and manage resource usage. These limits are fully configurable based on your deployment needs:

Rate limits are enforced using a token bucket algorithm and can be adjusted based on your infrastructure capacity and security requirements.

⚙️ Configuration

Essential En

Detection Modes

| Mode | Description | False Positives | False Negatives | Use Case |
|------|-------------|-----------------|-----------------|----------|
| Strict | Maximum security | Higher | Lower | Financial, Healthcare |
| Moderate | Balanced approach | Medium | Medium | General applications |
| Permissive | Minimize false positives | Lower | Higher | Content creation tools |

🤖 ML Pattern Discovery

PromptSentinel includes an ad ML-based pattern discovery system that automatically learns from detected threats:

Features

Automatic Clustering: Uses DBSCAN/HDBSCAN to find groups of similar attacks

Pattern Extraction: Extracts regex patterns from clustered threats

Self-Learning: Continuously impro based on real-world attacks

Performance Tracking: Monitors pattern effecti with precision/recall metrics

Lifecycle Management: Automatically promotes high-performing patterns and retires ineffective ones

API Endpoints

How It Works

1. Event Collection: Detection events are collected in a circular buffer
2. Feature Extraction: Multi-dimensional features extracted from prompts
3. Clustering: Similar attacks grouped using density-based clustering
4. Pattern Mining: Common patterns extracted using multiple techniques:
- Common substring analysis
- N-gram pattern matching
- Template-based extraction
- Differential analysis
5. E: Patterns tested and scored for precision/recall
6. Promotion: High-performing patterns automatically activated
7. Integration: Discovered patterns seamlessly integrated with heuristic detection

Installation with ML Features

🤖 ML Pattern Discovery

PromptSentinel includes an ad machine learning system that automatically discovers new attack patterns from blocked requests:

Features

Automatic Clustering: Uses DBSCAN/HDBSCAN to find groups of similar attacks

Pattern Extraction: Generates regex patterns from clustered samples

Performance Tracking: Monitors pattern precision/recall and auto-promotes successful patterns

Feedback Loop: Learn from false positi to impro

Self-Learning: Continuously adapts to new attack techniques

API Endpoints

Installation with ML Features

🔌 WebSocket Support

PromptSentinel provides WebSocket support for real-time streaming detection and continuous monitoring:

WebSocket Features

Real-time Detection: Stream prompts for instant analysis

Bidirectional Communication: Recei and updates

Batch Processing: Process multiple prompts with progress updates

Connection Management: Automatic reconnection and heartbeat

Low Latency: Direct streaming without HTTP o

WebSocket Client Example

Python WebSocket Client

WebSocket Message Types

| Type | Direction | Description |
|------|-----------|-------------|
| detection | Client→Server | Single prompt detection |
| analysis | Client→Server | Comprehensi |
| batch_detection | Client→Server | Multiple prompts |
| ping | Client→Server | Heartbeat check |
| stats | Client→Server | Get connection stats |
| detection_response | Server→Client | Detection result |
| analysis_response | Server→Client | Analysis result |
| system_message | Server→Client | System notifications |

💾 Redis Caching (Optional)

PromptSentinel includes optional Redis caching that provides:

98% faster responses: 12ms (cached) (uncached)

60-70% reduction in LLM API calls and costs

Ephemeral security: Memory-only, no disk persistence

Automatic failover: System works perfectly without Redis

Running with Redis

Redis Security Features

Password protected with AUTH

Memory-only operation (no disk writes)

Dangerous commands disabled (FLUSHDB, CONFIG, etc.)

Read-only filesystem with tmpfs

Auto-expiring entries (max 1 hour TTL)

Hashed cache keys (no sensitive data exposed)

🔍 PII Detection

PromptSentinel detects and redacts 15+ PII types:

Supported PII Types

Financial: Credit cards (with Luhn ), bank accounts, IBAN

Identity: SSN, passport numbers, driver's licenses

Contact: Email addresses, phone numbers, physical addresses

Credentials: API keys, passwords, AWS credentials, JWT tokens

Network: IP addresses (IPv4/IPv6), MAC addresses

Crypto: Bitcoin addresses, Ethereum addresses

Other: URLs, dates of birth, license plates

Redaction Modes

| Mode | Example Input | Example Output | Use Case |
|------|---------------|----------------|----------|
| mask | SSN: 123-45-6789 | SSN: XXX-XX-6789 | Default - partial |
| remove | My SSN is 123-45-6789 | My SSN is [SSN_REMOVED] | Complete removal |
| hash | | a3f5b2c8... | Consistent anonymization |
| reject | (any PII detected) | (request blocked) | Zero tolerance |

🛠️ Custom PII Rules

PromptSentinel supports defining custom PII detection patterns specific to your organization through YAML configuration files. This allows you to detect domain-specific sensitive data like employee IDs, internal project codes, or proprietary identifiers.

Quick Example

Create a config/custom_pii_rules.yaml file:

Key Features

YAML Configuration: Easy-to-maintain rule definitions

ReDoS Pre: Automatic complexity checking prevents regex denial-of-service

Custom Redaction: Define custom masking formats for each PII type

API Testing: Test rules before deployment via API endpoints

Security-First: Rules are immutable after startup, pre runtime tampering

Seamless Integration: Works alongside built-in PII detection

API Endpoints

Configuration

📚 Full Documentation - Comprehensi with security considerations, best practices, and industry-specific examples.

🛡️ Detection Strategies

1. Heuristic Detection (Pattern-Based)

Speed: <10ms

Accuracy: 85-90% for known patterns

Patterns: 50+ injection techniques including:

- Instruction o ("ignore previous", "disregard")
- Role manipulation attempts
- Encoding attacks (base64, hex, unicode)
- Context switching
- Jailbreak attempts

2. LLM Classification (AI-Based)

Speed: 500-2000ms (without cache)

Accuracy: 95-98% with multi-provider consensus

Pro: Anthropic → OpenAI → Gemini (with failover)

Features:

- Semantic understanding
- Context-aware analysis
- No detection

3. Combined

The system combines both methods for maximum accuracy:

Both detect → High confidence block

One detects → Medium confidence flag

Neither detects → Allow with monitoring

🔨 De

Test Suite Status

Total Tests: 1,653

Pass Rate: 100%

Code Co: 61%

Test Categories:

- Unit Tests: Comprehensi of all modules
- Integration Tests: End-to-end workflow
- E2E Tests: Full system tests with WebSocket support
- Performance Tests: Benchmark and optimization

📚 Full testing documentation: See docs/TESTING.md

Running Tests

Code Quality

Adding Detection Patterns

Edit src/prompt_sentinel/detection/heuristics.py:

🚢 Deployment

Docker

Kubernetes

Production Checklist

[ ] Set API_EN and DEBUG=false

[ ] Configure strong Redis password

[ ] Set appropriate rate limits

[ ] Enable OpenTelemetry monitoring

[ ] Configure log aggregation

[ ] Set up health check monitoring

[ ] Implement secret rotation

[ ] Configure auto-scaling

📊 Performance

Benchmarks

| Scenario | Latency (P50) | Latency (P95) | Throughput |
|----------|---------------|---------------|------------|
| Heuristic only | 8ms | 15ms | 5000 req/s |
| LLM only (no cache) | 700ms | 1500ms | 50 req/s |
| LLM with cache | 12ms | 25ms | 2000 req/s |
| Full detection + cache | 15ms | 35ms | 1500 req/s |

Optimization Tips

1. Enable Redis caching for 98% latency reduction
2. Use heuristic-only mode for high-, low-risk scenarios
3. Implement rate limiting to prevent abuse
4. Use connection pooling for database connections
5. Deploy multiple replicas for high a

🔒 Security

Best Practices

🔑 Ne API keys - Use en

🔐 Rotate secrets regularly - Implement key rotation

📊 Monitor detection logs - Track attack patterns

🛡️ Keep patterns updated - Regular security updates

🧪 Test before production - detection accuracy

🚨 Set up alerting - Get notified of attacks

Responsible Disclosure

This is a defensi tool. Please:

Never use it to create or test attacks

Report to:

Follow responsible disclosure practices

📈 Monitoring

Health Checks

PromptSentinel provides comprehensi check endpoints for monitoring and orchestration:

#### Basic Health Check

Returns:

O service status (healthy/degraded/unhealthy)

LLM provider health status

Redis connection status and latency

Cache statistics (hits, misses, hit rate)

System metrics (memory, CPU, connections)

Detection methods status

Service uptime

#### Detailed Component Health

Returns detailed status for:

Detector (heuristic, LLM, PII detection methods)

Cache (Redis connection, statistics)

LLM Pro (indi provider health)

Authentication (mode, bypass rules)

Rate Limiter (global limits)

ML Patterns (enabled/disabled)

WebSocket (acti, message count)

Monitoring (budget status)

#### Kubernetes Probes

The readiness probe checks:

Detector initialization

At least one healthy LLM provider (if classification enabled)

Critical service dependencies

Metrics

Request rate and latency per endpoint

Detection distribution

PII detection statistics

Cache hit/miss rates

LLM provider usage and failo

Error rates and types

OpenTelemetry Integration

Configure OTLP endpoint for traces and metrics:

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Write tests for your changes
4. Ensure all tests pass (make test)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

📄 License

This project is licensed under the Elastic License 2.0 (ELv2) - see the LICENSE file for details.

What this means for you:

✅ You CAN:

Use PromptSentinel commercially in your products and services

Modify and distribute the code

Integrate it into your applications (SaaS, on-premise, etc.)

Use it internally in your organization

Create plugins and extensions

❌ You CANNOT:

Offer PromptSentinel as a hosted/managed service to third parties

Remove or circumvent license key functionality

Remo and license notices

Why ELv2?

We chose the Elastic License 2.0 to ensure PromptSentinel remains free for companies to use while pre cloud pro from offering it as a competing service without contributing back to the project.

🙏 Acknowledgments

OpenAI, Anthropic, and Google for LLM APIs

FastAPI team for the excellent framework

UV team for the blazing-fast package manager

The security research community for prompt injection research

📊 Project Status

✅ Production Ready Features

Core Detection Engine: Multi-layer detection with heuristic and LLM-based classification

Multi-Provider Support: Anthropic, OpenAI, and Gemini with automatic failover

Intelligent Routing: Complexity-based optimization with 98% performance gain

WebSocket Support: Real-time streaming detection for continuous monitoring

ML Pattern Discovery: Self-learning system that discovers new attack patterns

API Monitoring: Usage tracking, budget controls, and rate limiting

PII Detection: 15+ PII types with multiple redaction modes

Redis Caching: 98% performance impro with optional Redis support

Comprehensi: 1,653 tests with 100% pass rate, 61+% code co

API Documentation: Full OpenAPI/Swagger support with interactive UI

Docker Support: Official images a at promptsentinelai/prompt-sentinel

Kubernetes Ready: Helm charts and deployment configurations included

SDK Libraries: Python, Ja, and Go SDKs (implemented, pending package registry publication)

🚧 In De

A/B Testing Framework: Experimentation system for optimizing detection strategies (stubbed endpoints present; see docs/ROADMAP.md)

Grafana Dashboards: Monitoring dashboards (templates a, pending refinement)

Obser Modules: Structured logging/tracing/test-only metrics are experimental stubs; prefer prompt_sentinel.monitoring in production

Performance Middleware: A but not wired by default; experimental

📋 Roadmap

See docs/ROADMAP.md for the full, regularly updated roadmap including:

- SDK publishing (PyPI, npm, pkg.go.dev)
- Enhanced ML features (ad clustering, embeddings)
- Multi-language support (i18n patterns, translation-assisted detection)
- Obser (tracing, logging, alerting) maturation
- Cloud-nati (Lambda, Cloud Functions, Azure Functions)
- Enterprise features (SAML/SSO, audit logging, compliance)

🧪 Stubs and Experimental Modules

Some modules are provided as scaffolding for future features and for tests. They are not wired into production flows by default:

prompt_sentinel.obser.* (logging, tracing, pipeline): in-memory/test stubs. Prefer prompt_sentinel.monitoring.metrics (Prometheus-backed) in production. See docs/ROADMAP.md.

prompt_sentinel.api.performance_middleware: experimental; disabled by default.

prompt_sentinel.i18n.* (localization, translator, detector helpers): experimental; minimal functionality intended for tests/examples.

Batch async status endpoint in api/batch_endpoint.py: stubbed (NotImplementedError) until async orchestration is introduced.

Refer to docs/ROADMAP.md for the status and planned milestones of these modules.

---

Built with ❤️ + ☕️ && 🥃 for securing all the things

*: 1.0.0 | Status: Production Ready*

项目信息

开发语言：Python

Star 数：2

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕A defensive security microservice for detecting and mitigating prompt injection attacks in LLM-based systems. PromptSentinel provides real-time protection using multi-layered detection strategies.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。