midas - MiDas: Multi-granularity Detector for Vulnerabilit

项目说明

⚙️ MiDas 🔎

*by Truong Giang Nguyen, Thanh Le-Cong, Hong Jin Kang, Ratnadira Widyasari, Chengran Yang, Zhipeng Zhao, Bowen Xu, Jiayuan Zhou, Xin Xia, Ahmed E. Hassan, Xuan-Bach D. Le, David Lo*

<p align="center">
<a href=" src=" TSE 49 (2023)-green?style=for-the-badge">
<a href=" src="
<a href=" src="
</p>

Welcome to the source code repo of MiDas, a commit detection tool introduced in our paper "Multi-granularity Detector for Fixes"!

Please cite the following article if you find Midas to be useful:

Truong Giang Nguyen, Thanh Le-Cong, Hong Jin Kang, Ratnadira Widyasari, Chengran Yang, Zhipeng
Zhao, Bowen Xu, Jiayuan Zhou, Xin Xia, Ahmed E. Hassan, Xuan-Bach D. Le, and David Lo

📃 Introduction

MiDas is a transformer-based no for detecting commits. MiDas extract information of commit in respect to multiple levels of granularity (i.e. commit level, file level, hunk level, line level)

MiDas consists of se extractors, regard the combination of granularity and CodeBERT representation:

| Feature extractor index | Granularity | CodeBERT representation |
|------------------|-------------|-------------------------|
| 1 | Commit | Context-dependant |
| 2 | File | Context-dependant |
| 3 | Hunk | Context-dependant |
| 5 | Commit | Context-free |
| 6 | File | Context-free |
| 7 | Hunk | Context-free |
| 8 | Line | Context-free |

To replicate the training process of MiDas, please follow the below steps:

1. Finetune CodeBERT for each feature extractor
2. Sa embedding represented by CodeBERT
3. Train feature extractors
4. Infer feature extractors to extract commit's features
5. Train neural classifier
6. Apply adjustment function
7. E MiDas

🔧 Prerequisites

Make sure you create a directory to store embedding , a folder "model" to store sa, and a "features" folder to store extractor features following this hierarchy:

Note: If you run MiDas on a Docker container, please run docker with parameters: "LANG=C.UTF-8 -e LC_ALL=C.UTF-8" to avoid error when writing to file, "--shm-size 16G" to avoid memory problem, "--gpus all" in case you use multiple GPUs

📁 Dataset

The dataset is a at:
Please download and put dataset inside the MiDas folder

🚀 Replication

Note: The current code base requires two GPUs to run. We will try to make it more flexible.

#### Finetune CodeBERT
Corresponding to se extractors, we ha python scripts to finetune them.

| Feature extractor index | Finetuning script |
|------------------|---------------------------------------|
| 1 | python .py |
| 2 | python .py |
| 3 | python ate.py |
| 5 | python .py |
| 6 | python .py |
| 7 | python ate.py |
| 8 | python ate.py |

#### Saving embedding
After finetuning, run the following scripts to sa corresponding to each feature extractor:

| Feature extractor index | Saving embeddings script |
|------------------|------------------------------------------|
| 1 | python preprocess_finetuned_.py |
| 2 | python preprocess_finetuned_.py |
| 3 | python preprocess_finetuned_.py |
| 5 | python preprocess_finetuned_.py |
| 6 | python preprocess_finetuned_.py |
| 7 | python preprocess_finetuned_.py |
| 8 | python preprocess_finetuned_.py |

#### Saving embedding
Next, we need to train se extractors

| Feature extractor index | Extractor training script |
|------------------|------------------------------------------|
| 1 | python .py |
| 2 | python .py |
| 3 | python .py |
| 5 | python .py |
| 6 | python .py |
| 7 | python .py |
| 8 | python .py |

#### Infer feature extractors and train neural classifier

Simply use the following two commands:

#### Apply adjustment function

Simply run:

#### E MiDas

The script for e is placed in e.py

Run e.py with parameter "--rq <rq_number>" to e MiDas with the corresponding research questions:

RQ1: Performance of MiDas on Java and Python project

RQ2: Performance of MiDas with/without adjustment function

RQ3: Performance of MiDas when continuously adding granularities

To obtain performance of MiDas using only line level, run:

To obtain performance of MiDas using only line level + hunk level, run two commands:

To obtain performance of MiDas using only line level + hunk level + file level, run two commands:

The performance of MiDas using all granularities is obtained in RQ1 (we hope you run it successfully).

项目信息

开发语言：Python

Star 数：14

技术要点

该项目属于安全软件方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕MiDas: Multi-granularity Detector for Vulnerability Fixes (IEEE TSE)展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。