FestInis a tool for discovering open S3 Buckets starting from a domains.
It perform a lot of test and collects information from:
- Web Pages (Crawler)
- S3 bucket itself (like S3 redirections)
There's a lot of S3 tools for enumeration and discover S3 bucket. Some of them are great but anyone have a complete list of features that
Main features that does
- Various techniques for finding buckets: crawling, dns crawling and S3 responses analysis.
- Proxy support for tunneling requests.
- AWS credentials are not needed.
- Works with any S3 compatible provider, not only with AWS.
- Allows to configure custom DNS servers.
- Integrated high performance HTTP crawler.
- Recursively search and feedback from the 3 engines: a domain found by dns crawler is send to S3 and Http Crawlers analyzer and the same for the S3 and Crawler.
- Works as 'watching' mode, listening for new domains in real time.
- Save all of the domains discovered in a separate file for further analysis.
- Allow to download bucket objects and put then in a FullText Search Engine (Redis Search) automatically, indexing the objects content allowing powerful search further.
- Limit the search for specific domain/s.
Python 3.8 of above needed!
$ pip install festin
$ festin -h
$ docker run --rm -it cr0hn/festin -h
$ festin -h
usage: __main__.py [-h] [--version] [-f FILE_DOMAINS] [-w] [-c CONCURRENCY] [--no-links] [-T HTTP_TIMEOUT] [-M HTTP_MAX_RECURSION] [-dr DOMAIN_REGEX] [-rr RESULT_FILE] [-rd DISCOVERED_DOMAINS] [-ra RAW_DISCOVERED_DOMAINS]
[--tor] [--debug] [--no-print] [-q] [--index] [--index-server INDEX_SERVER] [-dn] [-ds DNS_RESOLVER]
[domains [domains ...]]
Festin - the powered S3 bucket finder and content discover
-h, --help show this help message and exit
--version show version
-f FILE_DOMAINS, --file-domains FILE_DOMAINS
file with domains
-w, --watch watch for new domains in file domains '-f' option
-c CONCURRENCY, --concurrency CONCURRENCY
--no-links extract web s ite links
-T HTTP_TIMEOUT, --http-timeout HTTP_TIMEOUT
set timeout for http connections
-M HTTP_MAX_RECURSION, --http-max-recursion HTTP_MAX_RECURSION
maximum recursison when follow links
-dr DOMAIN_REGEX, --domain-regex DOMAIN_REGEX
only follow domains that matches this regex
-rr RESULT_FILE, --result-file RESULT_FILE
-rd DISCOVERED_DOMAINS, --discovered-domains DISCOVERED_DOMAINS
file name for storing new discovered after apply filters
-ra RAW_DISCOVERED_DOMAINS, --raw-discovered-domains RAW_DISCOVERED_DOMAINS
file name for storing any domain without filters
--tor Use Tor as proxy
--debug enable debug mode
--no-print doesn't print results in sc reen
-q, --quiet Use quiet mode
--index Download and index documents into Redis
Redis Search ServerDefault: redis://localhost:6379
not follow dns cnames
-ds DNS_RESOLVER, --dns-resolver DNS_RESOLVER
comma separated custom domain name servers
Configure search domains
FestInaccepts a start domain as command line parameter:
> festin mydomain.com
> cat domains.txt
> festin -f domains.txt
FestInperforms a lot of test for a domain. Each test was made concurrently. By default concurrency is set to 5. If you want to increase the number of concurrency tests you must set the option
> festin -c 10 mydomain.com
Be carefull with the number of concurrency test or "alarms" could raises in some web sites.
HTTP Crawling configuration
FestInembed a small crawler to discover links to S3 buckets. Crawler accepts these options:
- Timeout (
--http-timeout): configure a timeout for HTTP connections. If website of the domain you want to analyze is slow, we recommend to increase this value. By default timeout is 5 seconds.
- Maximum recursion (
--http-max-recursion): this value setup a limit for crawling recursion. Otherwise
FestInwill scan all internet. By default this value is 3. It means that only will follow: domain1.com -> [link] -> domain2.com -> [link] -> domain3.com -> [link] -> Maximum recursion reached. Stop
- Limit domains (
--domain-regex): set this option to limit crawler to these domains that matches with this regex.
> festin -T 20 -H 8 -dr *mydomain* mydomain.com
FestInruns it discover a lot of useful information. Not only about S3 buckets, also for other probes we could do. For example:
After we use
FestInwe can use discovered information (domains, links, resources, other buckets...) as input of other tools, like nmap.
For above reason
FestInhas 3 different modes to store discovered information and we can combine them:
FestInresult file (
--result-file): this file contains one JSON per line with buckets found by them. Each JSON includes: origin domain, bucket name and the list of objects for the bucket.
- Filtered discovered domains file (
--discovered-domains): this file contains one domain per line. These domains are discovered by the crawler, dns or S3 probes but only are stored these domains that matches with user and internal filters.
- Raw discovered domains file (
--raw-discovered-domains): this file contains all domains, one per line, discovered by
FestInwithout any filter. This option is useful for post-processing and analyzing.
> festin -rr festin.results -rd discovered-domains.txt -ra raw-domains.txt mydomain.txt
> festin -rd domains.txt && nmap -Pn -A -iL domains.txt -oN nmap-domains.txt
FestInembeds the option
--tor. By using this parameter you need local Tor proxy running at port 9050 at 127.0.0.1.
> tor &
> festin --tor mydomain.com
Some tests made by
FestIninvolves DNS. It support these options:
- Disable DNS discovery (
- Custom DNS server (
--dns-resolver): setup custom DNS server. If you plan to perform a lot of tests you should use a different DNS server like you use to your browser.
> festin -ds 220.127.116.11 mydomain.com
Full Text Support
FestInnot only can discover open S3 buckets. It also can download all content and store them in a Full Text Search Engine. This means that you can perform Full Text Queries to the content of the bucket!
FestInuses as Full Text Engine the Open Source project Redis Search.
This feature has two options:
- Enable indexing (
--index): to enable the indexing to the search engine you must setup this flag.
- Redis Search config (
--index-server): you only need to setup this option if your server is running in a different IP/Port that: localhost:6379.
> docker run --rm -p 6700:6379 redislabs/redisearch:latest -d
> festin --index --index-server redis://127.0.0.1:6700 mydomain.com
Pay attention to option `--index-server` is must has the prefix **redis://**
Running as a service (or watching mode)
Some times we don't want to stop
FestInand launch them some times when we have a new domain to inspect or any external tool discovered new domains we want to check.
FestInsupports watching mode. This means that
FestInwill start and listen for new domains. The way to "send" new domains to
FestInis by domains file. It monitor this file for changes.
This feature is useful to combine
FestInwith other tools, like dnsrecon
> festin --watch -f domains.txt
> echo "my-second-domain.com" >> domains.txt
> echo "another-domain.com" >> domains.txt
Example: Mixing FesTin + DnsRecon
The domain chosen for this example is target.com.
Step 1 - Run dnsrecon with desired options against target domain and save the output
> dnsrecon -d target.com -t crt -c target.com.csv
Step 2 - Prepare the previous generated file to feed
> tail -n +2 target.com.csv | sort -u | cut -d "," -f 2 >> target.com.domains
Step 3 - Run FestIn with desired options and save output
> festin -f target.com.domains -c 5 -rr target.com.result.json --tor -ds 18.104.22.168 >target.com.stdout 2>target.com.stderr
- target.com.result.json - Main result file with one line per bucket found. Each line is a JSON object.
- target.com.stdout - The standard output of festin command execution
- target.com.stderr - The standard error of festin command execution
Using FestIn with DnsRecon results
Run against target.com with default options and leaving result to target.com.result file:
> festin target.com -rr target.com.result.json
Run against target.com using tor proxy, with concurrency of 5, using DNS 22.214.171.124 for resolving CNAMEs and leaving result to target.com.result file:
> festin target.com -c 5 -rr target.com.result.json --tor -ds 126.96.36.199
Q: AWS bans my IP A:
When you perform a lot of test against AWS S3, AWS includes your IP in a black list. Then each time you want to access to any S3 bucket with
FestInof with your browser will be blocked.
We recommend to setup a proxy when you use
Who uses FestIn
They analyze and assess your company risk exposure in real time. Website