SpiderFoot v2.6.1 - Open Source Intelligence Automation

SpiderFoot is an open source intelligence (OSINT) automation tool. Its goal is to automate the process of gathering intelligence about a given target.

Purpose 

There are three main areas where SpiderFoot can be useful:

• If you are a pen-tester, SpiderFoot will automate the reconnaisance stage of the test, giving you a rich set of data to help you pin-point areas of focus for the test.
• Understand what your network/organisation is openly exposing to the outside world. Such information in the wrong hands could be a significant risk.
• SpiderFoot can also be used to gather threat intelligence about suspected malicious IPs you might be seeing in your logs or have obtained via threat intelligence data feeds.

Features

• Utilises a shedload of data sources; over 40 so far and counting, including SHODAN, RIPE, Whois, PasteBin, Google, SANS and more.
• Designed for maximum data extraction; every piece of data is passed on to modules that may be interested, so that they can extract valuable information. No piece of discovered data is saved from analysis.
• Runs on Linux and Windows. And fully open-source so you can fork it on GitHub and do whatever you want with it.
• Visualisations. Built-in JavaScript-based visualisations or export to GEXF/CSV for use in other tools, like Gephi for instance.
• Web-based UI. No cumbersome CLI or Java to mess with. Easy to use, easy to navigate. Take a look through the gallery for screenshots.
• Highly configurable. Almost every module is configurable so you can define the level of intrusiveness and functionality.
• Modular. Each major piece of functionality is a module, written in Python. Feel free to write your own and submit them to be incorporated!
• SQLite back-end. All scan results are stored in a local SQLite database, so you can play with your data to your heart’s content.
• Simultaneous scans. Each footprint scan runs as its own thread, so you can perform footprinting of many different targets simultaneously.
• So much more.. check out the documentation for more information.

Data Sources

This is an ever-growing list of data sources SpiderFoot uses to gather intelligence about your target. A few require API keys but they are freely available.

Source	Location	Notes
abuse.ch	http://www.abuse.ch	Various malware trackers.
AdBlock	https://easylist-downloads.adblockplus.org/easylist.txt	AdBlock pattern matches
AlienVault	https://reputation.alienvault.com	AlienVault’s IP reputation database.
Autoshun.org	http://www.autoshun.org	Blacklists.
AVG Site Safety Report	http://www.avgthreatlabas.com	Site safety checker.
Bing	http://www.bing.com	Scraping but future version to also use API.
Blocklist.de	http://lists.blocklist.de	Blacklists.
Checkusernames.com	http://www.checkusernames.com	Look up username availability on popular sites.
DNS	Your configured DNS server.	Defaults to your local DNS but can be configured to whatever IP address you supply SpiderFoot.
DomainTools	http://www.domaintools.com
DroneBL	http://www.dronebl.org
DuckDuckGo	http://www.duckduckgo.com
Facebook	http://www.facebook.com	Scraping but future version to also use API.
FreeGeoIP	http://freegeoip.net
Github	http://www.github.com
Google	http://www.google.com	Scraping but future version to also use API.
Google+	http://plus.google.com	Scraping but future version to also use API.
Google Safe Browsing	http://www.google.com/safebrowsing	Site safety checker.
IPCat	https://raw.githubusercontent.com/client9/ipcat/master/datacenters.csv	IP Categorisation.
LinkedIn	http://www.linkedin.com	Scraping but future version to also use API.
malc0de.com	http://malc0de.com	Blacklists.
malwaredomainlist.com	http://www.malwaredomainlist.com	Blacklists.
malwaredomains.com	http://www.malwaredomains.com	Blacklists.
McAfee SiteAdvisor	http://www.siteadvisor.com	Site safety checker.
NameDroppers	http://www.namedroppers.org
Notepad.cc	http://www.notepad.cc
Nothink.org	http://www.nothink.org	Blacklists.
Onion.City	http://onion.city	Search engine for the dark web.
OpenBL	http://www.openbl.org	Blacklists.
PasteBin	http://www.pastebin.com	Achieved through Google scraping.
Pastie	http://www.pastie.org
PGP Servers	http://pgp.mit.edu/pks/	PGP public keys.
PhishTank	http://www.phishtank.org	Identified phishing sites.
Project Honeypot	http://www.projecthoneypot.org	Blacklists. API key needed.
PunkSPIDER	http://www.punkspider.org
RIPE/ARIN	http://stat.ripe.net/
Robtex	http://www.robtex.com
SANS ISC	http://isc.sans.edu	Internet Storm Center IP reputation database.
SHODAN	http://www.shodanhq.com	API key needed.
SORBS	http://www.sorbs.net	Blacklists.
SpamHaus	http://www.spamhaus.org	Blacklists.
ThreatExpert	http://www.threatexpert.com	Blacklists.
TOR Node List	http://torstatus.blutmagie.de
TotalHash.com	http://www.totalhash.com	Domains/IPs used by malware.
UCEPROTECT	http://www.uceprotect.net	Blacklists.
VirusTotal	http://www.virustotal.com	Domains/IPs used by malware. API key needed.
WayBack Machine	http://www.archive.org
Whois	Various	Whois servers for different TLDs.
XSSposed	http://www.xssposed.org
Yahoo	http://www.yahoo.com	Scraping but future version to also use API.
Zone-H	http://www.zone-h.org	Easy to get black-listed. Log onto the site in a browser from the IP you’re scanning from first and enter the CAPTCHA, then it should be fine.

Download SpiderFoot v2.6.1