achilles-benchmark-depscanners - Achilles - Benchmark for assessing OSS-Vulnerabili

项目说明

Achilles - Test Suite for assessing OSS- Scanners

Achilles is an open test suite for e the performance of open-source scanners specifically for *Java* and *Maven* applications.

Achilles was created by Andreas Dann (1), Henrik Plate (2), Ben Hermann (3), Serena Elisa (2) Ponta, and Eric Bodden (1) of the security research group at Paderborn Uni (1), SAP Security Research Mougins, France (2), and the secure software engineering group at Technical Uni Dortmund (3).

Achilles is the right benchmark for you, if you are

* Using open-source libraries and frameworks,
* Using the build-automation system *Maven* to manage project dependencies,
* Worried about (potential) in the libraries you use,
* Comparing the performance of depenendy scanners,
* Deciding what tools to use for checking your open-source libraries for ,

What is included

Test Cases

The folder detection contains 2505 test fixtures to measure the performance of open-source scanner for detecting the inclusion of known dependencies.
The test fixtures are in the form of json files following the naming scheme: <CVE>_<GAV>.json..

The test fixtures contain true and false positi raised by the tools Eclipse Steady, OWASP Dependency-Check, and a commercial open-source dependency scanner.

#### Stats
The test cases cover *251* distinct CVEs, and *534* distinct

An o of the included artifacts is given in the table below

| GroupId:ArtifactId | #Num|
|:---------------------------------------------------------------|----:|
| ('axis', 'axis') | 3 |
| ('bouncycastle', 'bcpro') | 13 |
| ('com.fasterxml.jackson.core', 'jackson-databind') | 606 |
| ('com.fasterxml.jackson.dataformat', 'jackson-dataformat-xml') | 77 |
| ('com.google.guava', 'guava') | 25 |
| ('com.google.guava', 'gua') | 2 |
| ('com.google.protobuf', 'protobuf-java') | 1 |
| ('com.itextpdf', 'itextpdf') | 1 |
| ('com.squareup.okhttp', 'okhttp') | 2 |
| ('commons-beanutils', 'commons-beanutils') | 6 |
| ('commons-collections', 'commons-collections') | 11 |
| ('commons-fileupload', 'commons-fileupload') | 10 |
| ('commons-httpclient', 'commons-httpclient') | 1 |
| ('dom4j', 'dom4j') | 1 |
| ('io.netty', 'netty-all') | 12 |
| ('io.undertow', 'undertow-core') | 9 |
| ('jasperreports', 'jasperreports') | 4 |
| ('javax.el', 'el-api') | 1 |
| ('javax.el', 'ja') | 2 |
| ('ja', 'ja') | 2 |
| ('ja.jsp', 'jsp-api') | 1 |
| ('net.sf.jasperreports', 'jasperreports') | 14 |
| ('ognl', 'ognl') | 1 |
| ('org.apache.axis', 'axis-saaj') | 3 |
| ('org.apache.commons', 'commons-compress') | 1 |
| ('org.apache.httpcomponents', 'httpasyncclient') | 1 |
| ('org.apache.httpcomponents', 'httpclient') | 32 |
| ('org.apache.lucene', 'lucene-queryparser') | 1 |
| ('org.apache.poi', 'poi') | 37 |
| ('org.apache.poi', 'poi-ooxml') | 3 |
| ('org.apache.tomcat.embed', 'tomcat-embed-core') | 29 |
| ('org.beanshell', 'bsh') | 2 |
| ('org.bouncycastle', 'bcpro') | 5 |
| ('org.bouncycastle', 'bcpro') | 3 |
| ('org.codehaus.castor', 'castor') | 1 |
| ('org.codehaus.groovy', 'groovy') | 2 |
| ('org.codehaus.groovy', 'groo') | 17 |
| ('org.codehaus.groovy', 'groo') | 1 |
| ('org.eclipse.jetty', 'jetty-http') | 97 |
| ('org.eclipse.jetty', 'jetty-security') | 15 |
| ('org.eclipse.jetty', 'jetty-server') | 71 |
| ('org.eclipse.jetty', 'jetty-servlet') | 19 |
| ('org.eclipse.jetty', 'jetty-util') | 38 |
| ('org.hibernate', 'com.springsource.org.hibernate.') | 2 |
| ('org.hibernate', 'hibernate-') | 7 |
| ('org.jruby', 'jruby') | 9 |
| ('org.jruby', 'jruby-core') | 11 |
| ('org.jruby', 'openssl') | 94 |
| ('org.jruby', 'readline') | 3 |
| ('org.jruby', 'ripper') | 11 |
| ('org.jruby', 'yecht') | 6 |
| ('org.jruby.ext.posix', 'jnr-posix') | 3 |
| ('org.jruby.extras', 'bytelist') | 12 |
| ('org.jruby.extras', 'jffi') | 3 |
| ('org.jruby.util', 'bytelist') | 3 |
| ('org.jsoup', 'jsoup') | 1 |
| ('org.slf4j', 'slf4j-api') | 1 |
| ('org.springframework', 'spring-asm') | 62 |
| ('org.springframework', 'spring-core') | 413 |
| ('org.springframework', 'spring-expression') | 92 |
| ('org.springframework', 'spring-jcl') | 36 |
| ('org.springframework', 'spring-oxm') | 27 |
| ('org.springframework', 'spring-tx') | 1 |
| ('org.springframework', 'spring-web') | 104 |
| ('org.springframework', 'spring-webmvc') | 33 |
| ('p2.eclipse-plugin', 'org.apache.axis') | 1 |
| ('rubygems', 'jruby-openssl') | 438 |
| ('stax', 'stax') | 1 |
| ('taglibs', 'standard') | 1 |

Test Generator

The generator creates, based on the test fixtures, Maven/Ja that can be used as an input for open-source scanner.
Each test fixtures specifies in the element if the warnings is a true or false positive.

<p align="center">
<img width="75%" src="achilles_o.png">
</p>

Publications

If you want to read the details on how Achilles works, and the basis on which the test suite was created, our IEEE TSE paper Identifying Challenges for OSS Scanners - A Study & Test Suite
is a good place to start.

We welcome your contributions!

You are most welcome to contribute additional test cases to Achilles.
To do so, please fork the project, commit an appropriate test fixture, update this README and then send us a pull request.

License

Achilles is licensed under the LGPLv3 license, see LICENSE file. This basically means that you are free to use the tool (even in commercial, closed-source projects). However, if you extend or modify the tool, you must make your changes a under the LGPLv3 as well. This ensures that we can continue to improve the tool as a community effort.

Acknowledgement

This work was partially supported by the German Research Foundation (DFG) within the Collaborati Centre ”On-The-Fly Computing“ (GZ: SFB 901/3) under the project number 160364472.

Contact

If you experience any issues, you can ask for help on GitHub issue board. You can also contact us at

项目信息

开发语言：Java

Star 数：11

技术要点

该项目属于安全软件方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Achilles - Benchmark for assessing OSS-Vulnerability Scanners 59展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。