stringsifter - A machine learning tool that ranks strings based o

项目说明

<di="center">
<img src="misc/stringsifter-flat-dark.png" width="400">
</div>

--------------------------------------------------------------------------------

StringSifter is a machine learning tool that automatically ranks strings based on their relevance for malware analysis.

Quick Links

* Technical Blogpost - *Learning to Rank Strings Output for Speedier Malware Analysis*
* Announcement Blogpost - *Open Sourcing StringSifter*
* DerbyCon Talk - *StringSifter: Learning to Rank Strings Output for Speedier Malware Analysis*
* StringSifter releases on PyPi

Usage

StringSifter requires Python 3.9 or newer. Run the following commands to get the code, run unit tests, and use the tool:

Installation

For de, use poetry:

Running Unit Tests

To run unit tests from the StringSifter installation directory:

Running from the Command Line

The pip install command installs two runnable scripts flarestrings and rank_strings into your python en. When de from source, use pipenv run flarestrings and pipenv run rank_strings.

flarestrings mimics features of GNU binutils' strings, and rank_strings accepts piped input, for example:

rank_strings supports a number of command line arguments. The positional argument input_strings specifies a file of strings to rank. The optional arguments are:

Option | Meaning
--- | ---
--scores (-s) | Include the rank scores in the output
--limit (-l) | Limit output to the top limit ranked strings
--min-score (-m) | Limit output to strings with score >= min-score
--batch (-b) | Specify a folder of strings outputs for batch processing

Ranked strings are written to standard output unless the --batch option is specified, causing ranked outputs to be written to files named <input_file>.ranked_strings.

flarestrings supports an option -n (or --min-len) to print sequences of characters that are at least min-len characters long, instead of the default 4. For example:

will print and rank only strings of length 8 or greater.

Running from a Docker container

After cloning the repo, build the container. From the the package's top le:

Run the container with flarestrings or rank_strings argument to use the respecti. The containerized commands can be used in pipelines:

Or, run the container without arguments to get a shell prompt, using the -v flag to expose a host directory to the container:

where <my_malware> contains samples for analysis, for example:

At the container prompt:

All command line arguments are supported in the containerized scripts.

Running on FLOSS Output

StringSifter can be applied to arbitrary lists of strings, making it useful for practitioners looking to glean insights from alternatig sources such as li dumps, sandbox runs, or binaries that contain obfuscated strings. For example, FireEye Labs Obfuscated Strings Solver (FLOSS) extracts printable strings just as *Strings* does, but additionally reveals obfuscated strings that have been encoded, packed, or manually constructed on the stack. It can be used as an in-line replacement for Strings, meaning that StringSifter can be similarly invoked on FLOSS output using the following command:

Notes:
1. The –q argument suppresses headers and formatting to show only extracted strings. To learn more about additional FLOSS options, please see its Usage Docs.
2. FLOSS requires Python 2, while StringSifter requires Python 3. In the example command at least one of floss or rank_strings must include a relative path referencing a python en.
3. FLOSS can be downloaded as a standalone executable. In this case it is not required to specify a Python en because the executable does not rely on a Python interpreter.

Notes on running strings

This distribution includes the flarestrings program to ensure predictable output across platforms. If you choose to run your system's installed strings note that its options are not consistent across and platforms:

Linux

Most Linux distributions include the strings program from GNU Binutils. To extract both "wide" and "narrow" strings the program must be run twice, piping to an output file:

MacOS

Some of BSD strings packaged with MacOS do not support wide strings. Also note that the -a option to strings to scan the whole file may be disabled in the default configuration. Without -a informati may be lost. We recommend installing GNU Binutils or MacPorts to get a of strings that supports wide characters. Use care to invoke the correct of strings.

Windows

strings is not installed by default on Windows. We recommend installing Windows Sysinternals, Cygwin, or Malcode Analyst Pack to get a working strings.

Discussion

This of StringSifter was trained using *Strings* outputs from sampled malware binaries associated with the first EMBER dataset. Ordinal labels were generated using weak super procedures, and supervised learning is performed by Gradient Boosted Decision Trees with a learning-to-rank objecti. See Quick Links for further technical details. Please note that neither labeled data nor training code is currently a, though we may reconsider this approach in future releases.

Issues

We use GitHub Issues for posting bugs and feature requests.

Acknowledgements

Thanks to the FireEye Data Science (FDS) and FireEye Labs Reverse Engineering (FLARE) teams for review and feedback.

StringSifter was designed and de by Philip Tully (FDS), Matthew Haigh (FLARE), Jay Gibble (FLARE), and Michael Sikorski (FLARE).

The StringSifter logo was designed by Josh Langner (FLARE).

flarestrings is derived from the excellent tool FLOSS.

项目信息

开发语言：Python

Star 数：765

技术要点

该项目属于病毒分析方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕A machine learning tool that ranks strings based on their relevance for malware analysis.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。