URLextractor - Information gathering & website reconnaissance | h

项目说明

URLextractor

Information gathering & website reconnaissance
------

Usage:
./extractor

![]()

Tips:
* Colorex: put colors to the ouput pip install colorex and use it like ./extractor | colorex -g "INFO" -r "ALERT"
* Tldextract: is used by dnsenumeration function pip install tldextract

Features:
------

* IP and hosting info like city and country (using FreegeoIP)
* DNS servers (using dig)
* ASN, Network range, ISP name (using RISwhois)
* Load balancer test
* Whois for abuse mail (using Spamcop)
* PAC (Proxy Auto Configuration) file
* Compares hashes to diff code
* robots.txt (recursively looking for hidden stuff)
* Source code (looking for passwords and users)
* External links (frames from other websites)
* Directory FUZZ (like Dirbuster and Wfuzz - using Dirbuster) directory list)
* URLvoid API - checks Google page rank, Alexa rank and possible blacklists
* Provides useful links at other websites to correlate with IP/ASN
* Option to open ALL results in browser at the end

Changelog to 0.2.0:
------

* [Fix] Changed GeoIP from freegeoip to ip-api
* [Fix/Impro] Remo from robots.txt
* [Impro] Better whois abuse contacts (abuse.net)
* [Impro] Top passwords collection added to sourcecode checking
* [New feature] Firt run to install dependencies if need
* [New feature] Log file
* [New feature] Check for hostname on log file
* [New feature] Check if hostname is listed on Spamaus Domain Blacklist
* [New feature] Run a quick dnsenumeration with common ser

Changelog to 0.1.9:
------

* Abuse mail using lynx istead of ~~curl~~
* Target server name parsing fixed
* More about HTTP codes and directory discovery
* MD5 collection for IP fixed
* Links found now show unique URLs from array
* [New feature] Google results
* [New feature] Bing IP check for other hosts/
* [New feature] Opened ports from Shodan
* [New feature] information about IP
* [New feature] Alexa Rank information about $TARGET_HOST

Requirements:
------

Tested on Kali light mini AND OSX 10.11.3 with brew

Configuration file:

Todo list:
------

* [x] Upload to github :)
* [x] Check for installed packages
* [ ] Integration with other APIs
* [ ] Export to CSV
* [ ] Integration with CipherScan

Stargazers over time

![Stargazers over time]()

项目信息

开发语言：Shell

Star 数：455

技术要点

该项目属于病毒分析方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕Information gathering & website reconnaissance | https://phishstats.info/展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。