static_file_analysis - Analysis of file (doc, pdf, exe, ...) in deep (emm

项目说明

Static analysis malicious files

*Analysis malicious files in deep with clamscan and yara rules.*

This tool written in python langage makes the link between clam and yara. It can help you to score suspect file, can build tree graph for fast display embeded files (parent type, type, suspect or dangerous content), and can compute indicator of compromission. It uses clamav in order to extract embeded files and make json tree, then it sends all embbeded files to yara with context (in externs ) in order to check rules. If a rules matched, it gi of this rule. The max rule score is added to top of tree, you can add globale score that use all score found for make coefficient score. Extra feature, the tool can extract specific pattern (URL, HOST, IP, ...).

Features

Easy to use: docker pull lprat/sfa

Web ihm integrated in API

Clamscan extracts embedded files and makes json report

Clamscan check password on zip encrypted (ref: )

Extract file from URL with THUG () and analysis files extracted

Analyse json report and make json trees to consolidate informations

Extract patterns (pattern.db) with the ability to use the yara rules

Scan embedded files and root file with yara rules (+context informations in externs : type, parent type, pattern extract, ...)

- 2 level of yara rules (order), for gain fast and avoid multi rules (same) for each extension
- First level: format-specific rule
- check file type (reg, chm, exe, dll, ...) and potential risk according by extension (script, autopen, ...).Then push (by external ) check only element linked with extension for level 2
- check file origin: embed file
- Second level: global rules same for multi format
- check if unknown file type (extension): entropy, ...
- check suspect content file: obfuscate, cypher, packed, ...
- check dangerous elements (Mitre Attack): registry, command, ...
- check IOC familly malware (MISP import)

Compute risk score

- Put max score on top of tree
- Add global score with coefficient mechanism (coef.conf) to max score

Extract IOC on yara rules match

Extract text from image by OCR

Decompile JAR & CLASS java with procyon

Check & INTEZER & Hybrid Analysis & APPANY.RUN & OTX & XFORCE & MISP

Create PNG graph for fast analysis

Output result tree json in a file

Interresting tools

My docker contener contains Static analysis tools and others tools for deep analysis when match to yara rules:

Special tools

- Flash: ffdec
- Office document & rtf: oletools ()
- pdf: peepdf ()
- sdb: python-sdb ()

Decompiler

- Ja
- Exe python py2exe: unpy2exe ()
- Exe python PyInstaller: PyInstaller Extractor () or binwalk!
- Python bytecode (pyc): uncompyle6
- Autoit: clamav auto extract script
- & dotnet: () use with wine

Emulator/sandbox

- : ()
- ja:
- box-js ()
- JsJaws ()
- capacity PE/ELF/shellcode:
- elf:
- mbox ()
- zelos ()
- PE/Shellcode:
- Cmulator ()
- wine ( => WINEDEBUG=+all wine malware.exe)
- Speakeasy: ()
- Php:
-
-

Debugger/DBI

- bash: "bash -x script.sh"
- python: "python -m pdb script.py"
- strace: syscall trace
- ltrace: lib trace
- frida: frida ()

Others

- web analyse: thug ()
- shellcode extract: scanr ()
- IDX parser: Ja ()
- Beautifie js: js-beautify ()
- String Solver: floss ()
- firemware analysis: binwalk ()
- Samples redteam: atomic-red-team ()
- reverse engineering framework: ghidra ()

You can use other tools not include in my docker contener:

reverse engineering framework:

- radare2 ( )
- IHM Radare2 cutter ()

decompiler dotnet: ilspy ()

decompiler based on LL () - retdec-fileinfo identify "Original language"

sandbox powershell: PSDecode () - you can use in powershell docker ()

powershell capacities:

ja:

sandbox ruby: safe_ruby ()

sandbox python: pysandbox ()

sandbox generic: cuckoo ()

API call trace: drltrace ()

DBI: pin ()

Reverse Android: androidre ()

Reverse: radare2 ()

Usage

~~~
Static analysis by clamav and yara rules -- Contact:
Usage: analysis.py [-c /usr/local/bin/clamscan] [-d /tmp/extract_emmbedded] [-p pattern.db] [-s /tmp/graph.png] [-j /tmp/result.json] [-m coef_path] [-g] [-v] [-b password.pwdb] [-i /usr/bin/tesseract] [-l fra] [-] [-J] [-O] -f/-u path_filename/URL -y yara_rules_path1/ -a yara_rules_path2/

-h/--help : for help to use

-f/--filename= : path of filename to analysis

-u/--url= : url analysis use thug

-y/--yara_rules_path= : path of rules yara level 1

-a/--yara_rules_path2= : path of rules yara level 2

-p/--pattern= : path of pattern filename for data miner

-b/--password= : path of password clamav (.pwdb see: )

-c/--clamscan_path= : path of binary clamscan [>=0.99.3]

-m/--coef_path= : path of coef config file

-d/--directory_tmp= : path of directory to extract emmbedded file(s)

-j/--json_save= : path filename where save json result (JSON)

-i/--image= : path of 'tesseract' for analysis on potential social engenering by image

-J/--ja : Ja class/jar with procyon (apt-get install procyon-decompiler)

-l/--lang_image= : 'tesseract' lang ocr extratc (eng, fra, ...)

-g/--graph : generate graphe of analyz

-s/--sa= : path filename where sa (PNG)

-O/--osint : acti (hash, filename, domaine, url)
OSINT hybridanalisys env key: HYBRID_KEY
OTX env key: OTX_KEY
XFORCE env key: XFORCE_KEY & env pass: XFORCE_PASS
env key:
MISP env key: MISP_KEY & MISP env host: MISP_HOST
INTEZER env key: INTEZER_KEY

-r/--remove= : remo files

-= : API Key

-= : mode

example: analysis.py -c ./clama/clamscan/clamscan -f /home/analyz/strange/invoice.rtf -y /home/analyz/yara_rules1/ -a /home/analyz/yara_rules2/ -b /home/analyz/password.pwdb -i /usr/bin/tesseract -l fra -g -O

example: analysis.py -c ./clama/clamscan/clamscan -u www.exploitkit.top/id?000 -y /home/analyz/yara_rules1/ -a /home/analyz/yara_rules2/ -b /home/analyz/password.pwdb -i /usr/bin/tesseract -l fra -g -O

lionel:~/static_analysis$ python3 analysis.py -c clama/clamscan/clamscan -y yara_rules1/ -a yara_rules2/ -j /tmp/log.json -p pattern.db -g -f tests/pdf/jaff.pdf
Static analysis by clamav and yara rules -- Contact:
Create directory temp for emmbedded file: /tmp/tmpUee2rj

Extract emmbedded file(s) with clamav...
Analyz result...
Find resultat in json file:/tmp/tmpUee2rj/clama83886b9286c2.tmp...
Phase one finish!

~~~

PNG report example for jaff

!alt text

JSON report example for jaff

Requirements

clamav

python3: see requirements.txt [use docker]

For Image OCR: tesseract-ocr-all (deb)

For decompil ja (deb)

Install

~~~
Recompile clamav with json options and HARDENING compilation
./remake_clamav.sh
~~~

Docker install

~~~
git clone
cd static_file_analysis/docker
mkdir /tmp/samples && cp file_to_analyz.pdf /tmp/samples
docker-compose run sfa
$python3 analysis.py -c ./clama/clamscan/clamscan -f samples/file_to_analyz.pdf -y yara_rules1/ -a yara_rules2/ -b password.pwdb -i /usr/bin/tesseract -l fra -g -O -v &> /tmp/log
~~~

Docker install API REST

~~~
git clone
cd static_file_analysis/docker
#edit file docker-compose_api.yaml and change EN & UPDATE PROXY (if need)
docker-compose -f docker-compose_api.yml run sfa
~~~

Configure

coef.conf : file configuration for e coefficient score

pattern.db : file configuration with extracting pattern

yara_rules1/ : directory which contains yara rules level 1

yara_rules2/ : directory which contains yara rules level 2

password.pwdb : file contains password database to try open zip with password

Make your own yara rules

To create yara rules with this tool, you must use meta field:

description: description of the rule

weight: the score of the rule

: optionnal, you can add extern var if rule match for subsequent check ( global - on all files)

check_level2: optionnal, you can add extern var used to choice level 2 check (value: "check_command_bool,check_registry_bool") ( local - only on current files)

ids: extract ioc from YARA rules match "strings" () and output in json result on 'ioc' and 'globalIOC'. You choice category IOC, example: ids = "win_api" then create ioc{'win_api': ['first string found by yara match', 'second string found by yara match', ...]}

You can use extern build with clama and send them to yara with python script (analysis.py):

PathFile: filename and path

FileParentType: parent type of file, it's written as clama

FileType: Type of current file, it's written as clama

FileSize: Size of current fuke

FileMD5: MD5 of current file

CDBNAME: Original name of current file (exemple in MACRO file, or CHM file...)

zip_crypt_bool: Zip file with password (crypted)

EMBED_FILES: if zip file with password, contains filenames in zip file

image2text: if image file you can extract text with ocr (tesseract => !! attention Leptonica ha..., on debian, tesseract compiled with hardening option security)

serr: Debug flux of clamav

///: Virus total result

now_7_int: timstamp of now-7j

All make in json report of clamav

All informations extracted by pattern match

Check in path yara_rules for view samples!

Use tool in CRITS

I added this tool in CRITS services. I created pull request in CRITS service but it's not yet , but you can use my github repository so far.

Collaborati Into Threats - CRITS

Github CRITS services

My Github account of modified CRITS services

Use IHM WEB

Run docker compose or docker run for launch api
(docker lprat\sfa on cloud)
~~~
docker-compose -f ./docker-compose_api.yml up -d
or
docker run -ti -e "API_KEY=myapikey" -p 8000:8000 docker_sfa
~~~

With your fa browser go to

Use API REST

Run docker compose or docker run for launch api
(docker lprat\sfa on cloud)
~~~
docker-compose -f ./docker-compose_api.yml up -d
or
docker run -ti -e "API_KEY=myapikey" -p 8000:8000 docker_sfa
~~~

Request on port 8000:

~~~
Check File:
curl -k -F 'file=@/home/lionel/malwares/calc.xll' -H "x-api-key: mykeyapi"
Check URL:
curl -k --header "Content-Type: application/json" --request POST --data '{"url":" -H "x-api-key: mykeyapi"

Return JSON:
{"graph.png":"/download/700c4644ec40bfdada4502ffd5cb1411","result.json":"/download/9b9c453dc45b665c596b0f58c1c272b1","risk_score":4,"trace-serr.debug":"/download/d41d8cd98f00b204e9800998ecf8427e","trace-sout.debug":"/download/ef59eb8e65035a1064c1c32565bc0e74","ef59eb8e65035a1064c1c32565bc0000":"/download/ef59eb8e65035a1064c1c32565bc000"}
"ef59eb8e65035a1064c1c32565bc0000": for download embed file md5

Download file embed/json result/graph/...
curl -k -X 'POST' -H "x-api-key: mykeyapi"
~~~

Use reverse proxy for API or IHM web

Config exemple for nginx:
~~~
server {
listen $IP:443 ssl;
ser sfa.$yourdomain;
location / {

Use certificate auth

if ($ssl_client_ != SUCCESS) {

return 403;

}

if ($ssl_client_s_dn_cn = "NAME-On-Cert") {

return 403;

}

Use login/password auth

auth_basic "Authentification";

auth_basic_user_file /etc/nginx/.passwdweb;

proxy_pass_request_headers on;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;

Docker IP

proxy_pass
}
}
~~~

Extra

In Sigma_rules, you can find rule format SIGMA for detect files to analyse.

Greetz

- clama
- yara community
- Stéphane L.

Contact

项目信息

开发语言：YARA

Star 数：51

技术要点

该项目属于防护思路方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Analysis of file (doc, pdf, exe, ...) in deep (emmbedded file(s)) with clamscan and yara rules展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。