DC3-MWCP - DC3 Malware Configuration Parser (DC3-MWCP) is a f

项目说明

DC3-MWCP

Changelog | Releases

DC3 Malware Configuration Parser (DC3-MWCP) is a framework for parsing configuration information from malware.
The information extracted from malware includes items such as addresses, passwords, filenames, and
mutex names. A parser module is usually created per malware family.
DC3-MWCP is designed to help ensure consistency in parser function and output, ease parser de,
and facilitate parser sharing. DC3-MWCP supports both analyst directed analysis and
large-scale automated execution, utilizing either the nati API, a REST API, or a provided
command line tool. DC3-MWCP is authored by the Defense Cyber Crime Center (DC3).

Install

Builtin Parsers

Dragodis Support

DC3-Kordesii Support

Usage

- CLI Tool
- REST API
- Python API

Schema

STIX Output

YARA Matching

Helper Utilities

Guides

Parser De

Parser Components

Parser Installation

Parser Testing

Python Style Guide

Malstruct Tutorial

Style Guide

Testing

Install

By default, only the dependencies needed to run the base framework are installed.
To use the builtin parsers or specific features the optional dependencies may need to be installed:

Builtin Parsers

DC3-MWCP includes a handful of builtin parsers to get you started.
These can be used as-is, subclassed, or included in your own parser groups.

To use the builtin parsers, you must ensure the parsers extra is installed:

To view the a parsers:

Parsers are installed under the dc3 source name. To include them in a group simply add them with
the dc3: prefix.

Dragodis Support

DC3-MWCP optionally supports Dragodis
if it is installed. This allows you to obtain a disassembler agnostic interface for parsing
the file's disassembly from the mwcp.FileObject object with the .disassembly() function.

You can install Dragodis along with DC3-MWCP by adding [dragodis] to your appropriate install command:

After installation make sure to follow Dragodis's installation instructions to setup
a backend disassembler.

*It is recommended to also install Rugosa
for emulation and regex/yara matching capabilities using Dragodis.*

DC3-Kordesii Support

DC3-MWCP optionally supports DC3-Kordesii
if it is installed. This will allow you to run any DC3-Kordesii decoder from the
mwcp.FileObject object with the run_kordesii_decoder function.

You can install DC3-Kordesii along with DC3-MWCP by adding [kordesii] to your appropriate install command:

Usage

DC3-MWCP is designed to allow easy de and use of malware config parsers. DC3-MWCP is also designed to ensure
that these parsers are scalable and that DC3-MWCP can be integrated in other systems.

Most automated processing systems will use a condition, such as a yara signature match, to trigger execution
of an DC3-MWCP parser.

There are 3 options for integration of DC3-MWCP:

CLI: mwcp

REST API: mwcp serve

Python API

DC3-MWCP also includes a utility for test case generation and execution.

CLI tool

DC3-MWCP can be used directly from the command line using the mwcp command.

see for full set of options

REST API

DC3-MWCP can be used as a web service. The web service provides a web application as
well as a REST API for some commonly used functions:

-- executes a parser on uploaded file

-- executes a parser on uploaded file using YARA matching to determine the parser.

-- provides list of a parsers

-- provides the schema for report output

The ser mwcp to be installed with the server extra:

To use, first start the server by running:

Then you can either use an HTTP client to create REST requests.

#### Arguments

The REST API for /run_parser will accept a number of request parameters for customizing the processing and output results.

data -- The input file data.

legacy -- If this argument is set to True, the legacy JSON schema output will be produced. Defaults to the new schema.

- *NOTE: Legacy output will e be removed in a 4.0 release.*

output -- Sets the output format for parsing results.

- json -- JSON format (this is the default)
- zip -- Generates a ZIP file containing results and extracted residual files.
- stix -- STIX 2.1 JSON format

include_logs -- Whether to include logs in the report. Defaults to True.

no_file_data -- If this argument is set to True, binary data for extracted residual files won't be included in the report.

recursive - Whether to recursively process unidentified files with YARA matched parsers. (YARA_REPO must be setup for this option to be active.)

external_strings -- Whether to create external string reports for reported decoded strings found in each file. Defaults to False.

- These reports will be returned in the same manner as residual files.
- When enabled, the strings in the main report will be removed.

param/parameter -- Provides external parameters which will be injected into the knowledge_base before parsing starts.

- should be a key/value pair split by a :. (e.g. param="key_name:secret")
- This can be provided multiple times for multiple parameters.

#### Example Using cURL

#### Example Using Python

#### Example Output

The default parsing results will be in JSON format following the standardized schema.

A simple HTML interface is also a at the same address. By default this
is . Indi samples can be submitted and results
saved as JSON, plain text, or ZIP archives.

Python API

DC3-MWCP can be run directly from Python.

Configuration

All options are configurable through a settings.toml file.

DC3-MWCP looks for a user defined configuration file at either ~/.config/mwcp/settings.toml or %LOCALAPPDATA%\dc3\mwcp\settings.toml
to o the default settings.

To view the current configuration run the following:

To edit the configuration run the following to open the file in a text editor.
(This will copy the default configuration into a user directory)

To create a new user configuration file without editing:

An alternati file can also be used by creating a mwcp.toml file under the current directory.
This file has higher priority then other settings

We use Dynaconf which provides con like setting configuration using en
prefixed with MWCP_.

For example, to set the location of the malware repo:

Logging

DC3-MWCP uses Python's builtin in logging module to log all messages.
By default, logging is configured using the log_config.yml configuration
file. Which is currently set to log all messages to the console and error messages to %LOCALAPPDATA%/mwcp/errors.log.

You can provide your own custom log configuration file by adding the path
to the configuration parameter LOG_CONFIG_PATH.
(Please see Python's documentation for more information on how to write your own configuration file.)

You may also use the -- or --debug flags to adjust the logging level when using the mwcp tool.

Schema

One of the major goals of DC3-MWCP is to standardize output for malware configuration parsers, making the data
from one parser comparable with that of other parsers. This is achieved by establishing a schema of
standardized metadata elements that represent the common malware configuration items seen across malware families.

A formal JSON Schema can be found at schema.json, by calling mwcp schema in the command line, or programmatically by calling mwcp.schema().
This schema is the same as DC3-MWCP. A change in the may not necessarily
reflect a change in the actual schema. However, any major or minor changes to the schema will
be reflected in an appropriate change to the and will be noted in the changelog.
Please ensure you pin DC3-MWCP appropriately.

It is acknowledged that a set of generic elements will often not be adequate to capture the nuances of
indi malware families. To ensure that malware family specific attributes are appropriately captured
in parser output, the schema includes an "Other" element which supports arbitrary key-value pairs.
The keys and are arbitrary to permit flexibility in describing the peculiarities of indi malware families.
Information
not captured in the abstract standardized elements is captured through this mechanism.

The use of tags is encouraged to provide additional context for the configuration items.
For example, if a specific url is used to download a second stage component, a tag of "download"
could be added to the reported URL element. Alternatively, if the URL is used for a proxy,
a tag of "proxy" could be included.
There is no standard on what tags are a or when they should be included.
This should be determined by your organization.

Extending the Schema

It is possible to extend the schema to include your own custom metadata elements.
This can be accomplished by creating a class that inherits from mwcp.metadata.Metadata.
This class must be decorated with attr using the custom configuration mwcp.metadata.config.

*NOTE: The class name must be unique from other metadata elements.*

Please note, that extending the schema will ob cause the schema.json file to be incorrect.
To regenerate the schema to also include the custom element run mwcp.schema() afterwards.

STIX Output

MWCP can generate a STIX 2.1 JSON output that is suitable for integration into many
systems that support the STIX standard. This output format makes use of three SCO
extensions and one property extension in addition to the currently defined STIX
objects order to accurately convey MWCP's scan results.

Some tools may not support these extensions yet which can result in the following data
being omitted when ingesting MWCP's STIX output. The following provides a list of STIX
objects and extensions are used and what MWCP classes these are associated with:

1. artifact (SCO)
1. File -- only used if the original binary is requested
2. crypto-currency-address (SCO Extension)
1. CryptoAddress
3. directory (SCO)
1. File
2. Path
3. Service
4. domain-name (SCO)
1. Socket
2. URL
5. email-address (SCO)
1. EmailAddress
6. file (SCO)
1. File
2. Path
3. Service
7. ip (SCO)
1. Socket
2. URL
8. ip (SCO)
1. Socket
2. URL
9. malware-analysis (SDO)
1. MWCP's scan results are tied together via a malware-analysis object showing the input object and the outputs
10. mutex (SCO)
1. Mutex
11. network-traffic (SCO)
1. Socket
2. URL
12. note (SDO)
1. Boolean and Integer for Other. These are added to the description of the Note.
2. Descriptions and other narrative text tied to SCOs
3. Tags for SCOs excluding files
13. obser (SCO Extension)
1. DecodedString
2. MissionID
3. Other
4. Pipe
5. User Agent
6. UUID
14. process (SCO)
1. Command
2. Service
15. relationship (SRO)
1. DecodedString
2. URL
16. RSA Private Key (Property Extension for x509-certificate)
1. RSAPri
17. symmetric-encryption (SCO)
1. EncryptionKey
18. user-account (SCO)
1. Credential
19. url (SCO)
1. URL
20. x509-certificate (SCO)
1. RSAPri
2. RSAPublicKey
3. SSLCertSHA1
21. windows-registry-key (SCO)
1. Registry2

YARA Matching

MWCP includes a runner that can use YARA match results to determine which parser(s) to run on a given file.

This will be used whenever you use - instead of specifying a parser on the command line,
when a parser isn't specified in mwcp.run(), or when a parser isn't specified in a ser.

As well, YARA matching will be recursively used on unidentified residual files.
If you want to disable this, either set --no-recursive on the command line or set recursi on mwcp.run().

Setup

To enable YARA matching you'll need to specify a directory containing YARA signatures which use the mwcp
meta field to map a signature to a comma delimited list of parsers. Parsers can be specified in the same
way as on the command line or Python API. That is, parser group names, . notation for specific parser components,
and the use of : for specifying a parser source are all valid.

Any signatures that don't have the mwcp meta field will be ignored.

To setup a YARA repo, set the YARA_REPO field to point to a directory containing YARA signatures (subdirectories allowed)
in the configuration file that appears when you call mwcp config.
If you ha from an older of MWCP, you may need to first backup and remove the original configuration file and
then run mwcp config again to have MWCP recreate the file.

Alternatively, the yara repo can be specified in the command line with --yara-repo. But the former method
is necessary to use YARA matching with the server.

Testing

Recursive YARA matching for unidentified files can be done when creating test cases. Simply include the --recursive flag
when adding a new or updating an existing test case.

Once a test case is created with recursion turned on, anybody running your test case must also have a YARA repo setup
with the same YARA signatures so the test will pass for them.
For this reason, it is recommended to turn on recursion for a test only if the parser's full functionality depends on it.

Helper Utilities

MWCP comes with a few helper utilities (located in mwcp.utils) that may become useful for parsing malware files.

pefileutils - Provides helper functions for common routines done with the pefile library. (obtaining or checking for exports, imports, resources, sections, etc.)

elffileutils - Provides helper functions for common routines done with the elftools library. Provides a consistent interface similar to pefileutils.

custombase64 - Provides functions for base64 encoding/decoding data with a custom alphabet.

poshdeob - An experimental powershell deobfuscator utility used to statically deobfuscate code and extract strings.

项目信息

开发语言：Python

Star 数：353

技术要点

该项目属于病毒分析方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕DC3 Malware Configuration Parser (DC3-MWCP) is a framework for parsing configuration information from malware. The information extracted from malware includes items such as addresses, passwords, filenames, and mutex names.展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。