Add documentation and some minor improvements
This commit is contained in:
Cag
Родитель
07ce02ddf0
Коммит
7902049b9f
|
|
@ -1,4 +1,4 @@
|
|||
FROM ruby:latest
|
||||
FROM ruby
|
||||
MAINTAINER Cag
|
||||
|
||||
# This is to be able to talk to Tenable API
|
||||
|
|
|
|||
84
Readme.md
84
Readme.md
|
|
@ -1 +1,85 @@
|
|||
# vautomator-standalone
|
||||
Iterative automation of common VA tasks using OOP.
|
||||
|
||||
If you'd like to contribute, please reach out to [me](https://mozillians.org/en-US/u/Cag/) and I'd be happy to add you as a contributor.
|
||||
|
||||
## Install & Running
|
||||
|
||||
1. First, download the repo: `git clone https://github.com/caggle/vautomator-standalone.git && cd vautomator-standalone`
|
||||
2. Build the Docker image: `docker-compose build vautomator`
|
||||
3. Run it!: `docker run -v ${PWD}/results:/app/results -it vautomator:latest ./run.py <target>`
|
||||
|
||||
Example run:
|
||||
```
|
||||
$ docker run -v ${PWD}/results:/app/results -it vautomator:latest ./run.py http://192.168.0.1
|
||||
[f2769b83b62b] 2019-01-21 06:23:51 AM UTC INFO [+] Running all the scans now. This may take a while...
|
||||
[f2769b83b62b] 2019-01-21 06:24:23 AM UTC WARNING [!] The target has recently been scanned by Tenable.io, retrieving results...
|
||||
[f2769b83b62b] 2019-01-21 06:24:30 AM UTC INFO [+] Running nmap port scans...
|
||||
[f2769b83b62b] 2019-01-21 06:26:54 AM UTC INFO [+] Nmap port scan(s) successfully ran.
|
||||
[f2769b83b62b] 2019-01-21 06:26:54 AM UTC INFO [+] Running ssh_scan...
|
||||
[f2769b83b62b] 2019-01-21 06:26:58 AM UTC INFO [+] SSH scan successfully ran.
|
||||
[f2769b83b62b] 2019-01-21 06:26:58 AM UTC INFO [+] Running TLS Observatory scan...
|
||||
[f2769b83b62b] 2019-01-21 06:27:19 AM UTC INFO [+] TLS Observatory scan successfully ran.
|
||||
[f2769b83b62b] 2019-01-21 06:27:19 AM UTC INFO [+] Running dirb scan...
|
||||
[f2769b83b62b] 2019-01-21 06:31:48 AM UTC INFO [+] Directory brute scan successfully ran.
|
||||
[f2769b83b62b] 2019-01-21 06:31:49 AM UTC INFO [+] All done. Tool output from the scan can be found at /app/results/192.168.0.1/
|
||||
|
||||
====== SCAN SUMMARY ======
|
||||
INFO [+] [\o/] nmap scan completed successfully!
|
||||
INFO [+] [\o/] dirbrute scan completed successfully!
|
||||
INFO [+] [\o/] sshscan scan completed successfully!
|
||||
INFO [+] [\o/] tlsobs scan completed successfully!
|
||||
INFO [+] [\o/] nessus scan completed successfully!
|
||||
WARNING [!] [ :| ] httpobs scan skipped as not applicable to the target.
|
||||
====== END OF SCAN =======
|
||||
```
|
||||
|
||||
## What it does
|
||||
|
||||
Using **Python 3**, it runs a bunch of tools against a URL/FQDN/IPv4 address on a Docker image of its own, and saves tool outputs for later analysis, as a part of a vulnerability assessment.
|
||||
|
||||
### What it actually does
|
||||
|
||||
* Determines if the the target is a URL, an IPv4 address or a hostname/FQDN
|
||||
* If URL *(note: it could be a URL with FQDN or IPv4 address)* it will run:
|
||||
* An nmap UDP scan for about 25 selected UDP services
|
||||
* An nmap TCP scan for top 1000 services
|
||||
* ssh_scan (if an SSH service is identified)
|
||||
* A Nessus (Tenable.io) "Basic Network Scan" (provided if you have valid Tenable.io API keys)
|
||||
* HTTP Observatory scan
|
||||
* TLS Observatory scan
|
||||
* Directory bruteforcing against a wordlist
|
||||
|
||||
* If IP address, it will only run:
|
||||
* An nmap UDP scan for about 25 selected UDP services
|
||||
* An nmap TCP scan for top 1000 services
|
||||
* ssh_scan (if an SSH service is identified)
|
||||
* A Nessus (Tenable.io) "Basic Network Scan" (provided if you have valid Tenable.io API keys)
|
||||
|
||||
In the current implementation these tasks are performed sequentially with the intent being "run and forget" for a couple of hours, while you are doing other important work.
|
||||
|
||||
#### Port scans
|
||||
|
||||
For TCP and UDP port scans, [python-nmap](https://pypi.org/project/python-nmap/) is used.
|
||||
|
||||
##### SSH scan
|
||||
|
||||
For SSH scan, [ssh_scan](https://github.com/mozilla/ssh_scan) is used.
|
||||
|
||||
#### Nessus scan
|
||||
|
||||
Nessus scans will fail unless you have a pair of valid Tenable.io API keys *with administrative permissions*. If you do, populate the .env file with them in the below form building the Docker image:
|
||||
|
||||
```
|
||||
TENABLEIO_ACCESS_KEY=<ACCESS_KEY>
|
||||
TENABLEIO_SECRET_KEY=<SECRET_KEY>
|
||||
```
|
||||
|
||||
#### Web App scans
|
||||
|
||||
If you are running the tool against a URL, a number of additional external tools will be utilised. These will be installed in the Docker container when you build it.
|
||||
* [HTTP Observatory](https://github.com/mozilla/http-observatory) is used as a Python module.
|
||||
* [TLS Observatory](https://github.com/mozilla/tls-observatory), by means of `tlsobs` client.
|
||||
* For directory brute-forcing:
|
||||
* By default, `dirb` will be used with the common wordlist.
|
||||
* `gobuster` will also be installed in the Docker container, however a command line switch to use it instead is not available yet (you would have to modify the code).
|
||||
|
|
|
|||
|
|
@ -1,6 +1,9 @@
|
|||
version: '3'
|
||||
services:
|
||||
vautomator:
|
||||
# This does not work for some reason
|
||||
# sysctls:
|
||||
# - net.ipv6.conf.all.disable_ipv6=0
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./Dockerfile
|
||||
|
|
@ -8,4 +11,3 @@ services:
|
|||
- TENABLEIO_ACCESS_KEY=${TENABLEIO_ACCESS_KEY}
|
||||
- TENABLEIO_SECRET_KEY=${TENABLEIO_SECRET_KEY}
|
||||
image: vautomator
|
||||
|
||||
|
|
|
|||
|
|
@ -6,9 +6,11 @@ from netaddr import valid_ipv4
|
|||
from urllib.parse import urlparse
|
||||
from lib import task
|
||||
|
||||
# Logging in UTC
|
||||
logger = logging.getLogger(__name__)
|
||||
coloredlogs.install(level='INFO', logger=logger, reconfigure=True,
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s')
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s',
|
||||
datefmt="%Y-%m-%d %I:%M:%S %p %Z")
|
||||
|
||||
|
||||
class Target:
|
||||
|
|
@ -89,7 +91,8 @@ class Target:
|
|||
return False
|
||||
|
||||
def addTask(self, new_task):
|
||||
# self.tasklist.append(new_task)
|
||||
# This is a hacky way pf running ssh_scan
|
||||
# right after nmap port scan
|
||||
if isinstance(new_task, task.SSHScanTask):
|
||||
self.tasklist.insert(2, new_task)
|
||||
else:
|
||||
|
|
@ -110,6 +113,7 @@ class Target:
|
|||
elif isinstance(one_task, task.NessusTask):
|
||||
nessus_results = one_task.runNessusScan()
|
||||
if (nessus_results):
|
||||
# TODO: Need to be more precise about this time check
|
||||
epoch_cdate = nessus_results.histories()[0].creation_date
|
||||
cdate = datetime.datetime.fromtimestamp(float(epoch_cdate))
|
||||
if(cdate.date() < datetime.date.today()):
|
||||
|
|
|
|||
24
lib/task.py
24
lib/task.py
|
|
@ -11,9 +11,11 @@ from tenable_io.api.scans import ScanExportRequest
|
|||
from tenable_io.api.models import Scan
|
||||
from distutils.spawn import find_executable
|
||||
|
||||
# Logging in UTC
|
||||
logger = logging.getLogger(__name__)
|
||||
coloredlogs.install(level='INFO', logger=logger, reconfigure=True,
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s')
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s',
|
||||
datefmt="%Y-%m-%d %I:%M:%S %p %Z")
|
||||
|
||||
|
||||
class Task:
|
||||
|
|
@ -34,16 +36,16 @@ class NmapTask(Task):
|
|||
# We need to check if SSH service is available within port scan results
|
||||
if (port_scan_results["".join(port_scan_results.all_hosts())].has_tcp(22)):
|
||||
# Port 22/tcp is open, perform ssh_scan scan
|
||||
# SSHScanTask(self.tasktarget).runSSHScan(22)
|
||||
self.tasktarget.addTask(SSHScanTask(self.tasktarget, 22))
|
||||
|
||||
else:
|
||||
# Need to find the actual SSH port, in case it is not 22
|
||||
# Magic happens here...
|
||||
# Ref: https://bitbucket.org/xael/python-nmap/src/2b493f71a26f63a01c155c073fbf0211a3219ff2/nmap/nmap.py?at=default&fileviewer=file-view-default#nmap.py-436:465
|
||||
for ssh_port in port_scan_results["".join(port_scan_results.all_hosts())].all_tcp():
|
||||
if 'script' in port_scan_results["".join(port_scan_results.all_hosts())]['tcp'][ssh_port].keys():
|
||||
if 'ssh' in "".join(port_scan_results["".join(port_scan_results.all_hosts())]['tcp'][ssh_port]['script'].values()).lower():
|
||||
# We have SSH service on a non-standard port, perform scan
|
||||
# SSHScanTask(self.tasktarget).runSSHScan(ssh_port)
|
||||
self.tasktarget.addTask(SSHScanTask(self.tasktarget, ssh_port))
|
||||
return
|
||||
|
||||
|
|
@ -230,7 +232,7 @@ class NessusTask(Task):
|
|||
logger.error("[-] Tenable.io scan failed: ".format(TIOException))
|
||||
return False
|
||||
|
||||
def downloadReport(self, nscan, reportformat="html", style="vulns"):
|
||||
def downloadReport(self, nscan, reportformat="html", style="assets"):
|
||||
report_path = "/app/results/" + self.tasktarget.targetdomain + "/Scan_for_" + self.tasktarget.targetdomain
|
||||
|
||||
if reportformat == "html":
|
||||
|
|
@ -246,7 +248,7 @@ class NessusTask(Task):
|
|||
else:
|
||||
return False
|
||||
|
||||
if style == "vulns":
|
||||
if style == "assets":
|
||||
reportoutline = ScanExportRequest.CHAPTER_CUSTOM_VULN_BY_HOST
|
||||
elif style == "exec":
|
||||
reportoutline = ScanExportRequest.CHAPTER_EXECUTIVE_SUMMARY
|
||||
|
|
@ -257,9 +259,9 @@ class NessusTask(Task):
|
|||
|
||||
nscan.download(report_path, format=fmt, chapter=reportoutline)
|
||||
|
||||
def checkScanStatus(self, nscan, scan_history=None):
|
||||
def checkScanStatus(self, nscan):
|
||||
# Query Tenable API to check if the scan is finished
|
||||
status = nscan.status(nscan.id, scan_history)
|
||||
status = nscan.status(nscan.id)
|
||||
|
||||
if status == nscan.STATUS_COMPLETED:
|
||||
return "COMPLETE"
|
||||
|
|
@ -336,12 +338,12 @@ class DirectoryBruteTask(Task):
|
|||
logger.info("[+] Running dirb scan...")
|
||||
if "URL" in self.tasktarget.getType():
|
||||
cmd = "/app/vendor/dirb222/dirb " + self.tasktarget.targetname \
|
||||
+ "/ /app/vendor/dirb222/wordlists/common.txt -o /app/results/" \
|
||||
+ self.tasktarget.targetdomain + "/https_dirb_common.txt -f"
|
||||
+ "/ /app/vendor/dirb222/wordlists/small.txt -o /app/results/" \
|
||||
+ self.tasktarget.targetdomain + "/https_dirb_common.txt -f -w -S -r"
|
||||
else:
|
||||
cmd = "/app/vendor/dirb222/dirb https://" + self.tasktarget.targetdomain \
|
||||
+ "/ /app/vendor/dirb222/wordlists/common.txt -o /app/results/" \
|
||||
+ self.tasktarget.targetdomain + "/https_dirb_common.txt -f"
|
||||
+ "/ /app/vendor/dirb222/wordlists/small.txt -o /app/results/" \
|
||||
+ self.tasktarget.targetdomain + "/https_dirb_common.txt -f -w -S -r"
|
||||
|
||||
dirbscan_cmd = utils.sanitise_shell_command(cmd)
|
||||
p = subprocess.Popen(dirbscan_cmd, stdout=subprocess.PIPE, shell=True)
|
||||
|
|
|
|||
|
|
@ -12,8 +12,8 @@ def sanitise_shell_command(command):
|
|||
|
||||
|
||||
def package_results(output_dir):
|
||||
# Do reporting (take all the output from the prior runs,
|
||||
# zip it up, and attach to BMO)
|
||||
# Do reporting (take all the output from
|
||||
# the prior runs, zip it up
|
||||
tarfile = output_dir.split('/')
|
||||
cmd = "tar --warning=no-all -zcf " + output_dir + tarfile[3] + ".tar.gz -C " \
|
||||
+ output_dir + " . --exclude=" + output_dir + tarfile[3] + ".tar.gz"
|
||||
|
|
|
|||
11
run.py
11
run.py
|
|
@ -8,10 +8,11 @@ import coloredlogs
|
|||
from urllib.parse import urlparse
|
||||
from lib import target, task, utils
|
||||
|
||||
# Logging in UTC
|
||||
logger = logging.getLogger(__name__)
|
||||
# Default logging level is INFO
|
||||
coloredlogs.install(level='INFO', logger=logger, reconfigure=True,
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s')
|
||||
fmt='[%(hostname)s] %(asctime)s %(levelname)-8s %(message)s',
|
||||
datefmt="%Y-%m-%d %I:%M:%S %p %Z")
|
||||
|
||||
|
||||
def setupVA(va_target):
|
||||
|
|
@ -33,7 +34,7 @@ def setupVA(va_target):
|
|||
va_target.addTask(task.DirectoryBruteTask(va_target, tool="dirb"))
|
||||
else:
|
||||
va_target.addTask(task.MozillaTLSObservatoryTask(va_target))
|
||||
# va_target.addTask(task.DirectoryBruteTask(va_target, tool="dirb"))
|
||||
va_target.addTask(task.DirectoryBruteTask(va_target, tool="dirb"))
|
||||
# HTTP Observatory does not like IPs as a target, skipping
|
||||
va_target.resultsdict.update({'httpobs': "PASS"})
|
||||
elif va_target.getType() == "IPv4":
|
||||
|
|
@ -73,9 +74,10 @@ def runVA(scan_with_tasks, outpath):
|
|||
results = scan_with_tasks.runTasks()
|
||||
# results here is a dict
|
||||
time.sleep(1)
|
||||
# Return code check is a bit hacky,
|
||||
# basically we are ignoring warnings from tar
|
||||
if utils.package_results(outpath).returncode is not 127:
|
||||
logger.info("[+] All done. Tool output from the scan can be found at " + outpath)
|
||||
# return results
|
||||
else:
|
||||
logger.warning("[!] There was a problem compressing tool output. Check " + outpath + " manually.")
|
||||
time.sleep(1)
|
||||
|
|
@ -85,7 +87,6 @@ def runVA(scan_with_tasks, outpath):
|
|||
def main():
|
||||
|
||||
results = {'nmap': False, 'nessus': False, 'tlsobs': False, 'httpobs': False, 'sshscan': False, 'dirbrute': False}
|
||||
# Get targeting info
|
||||
destination = sys.argv[1]
|
||||
output_path = "/app/results/" + destination + "/"
|
||||
va_target = target.Target(destination, results)
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче