mirror of
https://github.com/aljazceru/recon-pipeline.git
synced 2025-12-24 09:44:25 +01:00
6eb3bd8cb0
Co-authored-by: Ryan Good <usafaryangood@gmail.com> * added initial skeleton; restructured project directories * removed workers directive from luigi; changed input to tko-subs * changed masscan command to use config.tool_paths * linted __init__ files and updated docstring for get_scans * added per-file-ignores for linting * recon-pipeline linted * PoC working for amass results -> db; rudimentary db mgmt commands also * more linting * added database management commands to the shell * db_location passes through to all tasks; masscan results added to db * removed unused imports from masscan.py * added ParseNmapOutput class to handle parsing for database storage * cleaned up repeat code * searchsploit results stored in db * lint/format * gobuster scans now stored in database * fixed test_recon tests to use db_location * fixed web tests * tkosub entries recorded in db * subjack scan results stored in database * webanalyze results stored in db * refactored older commits to use newer helper functions * refactored older commits to use newer helper functions * aquatone results stored in database refactored a few scans to use dbmanager helper functions refactored db structure wrt headers/screenshots added 80/443 to web_ports in config.py * fixed a few queries and re-added webanalyze to FullScan * view targets/endpoints done * overhauled nmap parsing * print all nmap_results good, next to focus on filtering * complex nmap filters complete * nmap printing done * updated pipfile * view web-technologies complete * view searchsploit results complete * removed filesystem code from amass * targetlist moved to db only * targets,amass,masscan all cutover to full database; added view ports * nmap fully db compliant * aquatone and webtargets db compliant * gobuster uses db now * webanalyze db compliant * all scans except corscanner are db compliant * recon tests passing * web tests passing * linted files * added tests for helpers.py and parsers.py * refactored some redundant code * added tests to pre-commit * updated amass tests and pre-commit version * updated recon.targets tests * updated nmap tests * updated masscan tests * updated config tests * updated web targets tests * added gobuster tests * added aquatone tests * added subdomain takeover and webanalyze tests; updated test data * removed homegrown sqlite target in favor of the sqla implementation * added tests for recon-pipeline.py * fixed cluge function to set __package__ globally * updated amass tests * updated targets tests * updated nmap tests * updated masscan tests * updated aquatone tests * updated nmap tests to account for no searchsploit * updated nmap tests to account for no searchsploit * updated masscan tests * updated subjack/tkosub tests * updated web targets tests * updated webanalyze tests * added corscanner tests * linted DBManager a bit * fixed weird cyclic import issue that only happened during docs build; housekeeping * added models tests, removed test_install dir * updated docs a bit; sidenav is wonky * fixed readthedocs requirements.txt * fixed issue where view results werent populated directly after scan * added new tests to pipeline; working on docs * updated a few overlooked view command items * updated tests to reflect changes to shell * incremental push of docs update * documentation done * updated exploitdb install * updated exploitdb install * updated seclists install * parseamass updates db in the event of no amass output * removed corscanner * added pipenv shell to install instructions per @GreaterGoodest * added pipenv shell to install instructions per @GreaterGoodest * added check for chromium-browser during aquatone install; closes #26 * added check for old recon-tools dir; updated Path.resolve calls to Path.expanduser.resolve; fixed very specific import bug due to filesystem location * added CONTIBUTING.md; updated pre-commit hooks/README * added .gitattributes for linguist reporting * updated tests * fixed a few weird bugs found during test * updated README * updated asciinema links in README * updated README with view command video * updated other location for url scheme /status * add ability to specify single target using --target (#31) * updated a few items in docs and moved tool-dict to tools-dir * fixed issue where removing tempfile without --verbose caused scan to fail
91 lines
2.9 KiB
ReStructuredText
91 lines
2.9 KiB
ReStructuredText
.. _scan-ref-label:
|
|
|
|
Running Scans
|
|
=============
|
|
|
|
All scans are run from within ``recon-pipeline``'s shell. There are a number of individual scans, however to execute
|
|
multiple scans at once, ``recon-pipeline`` includes wrappers around multiple commands. As of version |version|, the
|
|
following individual scans are available
|
|
|
|
- :class:`pipeline.recon.amass.AmassScan`
|
|
- :class:`pipeline.recon.web.aquatone.AquatoneScan`
|
|
- :class:`pipeline.recon.web.gobuster.GobusterScan`
|
|
- :class:`pipeline.recon.masscan.MasscanScan`
|
|
- :class:`pipeline.recon.nmap.SearchsploitScan`
|
|
- :class:`pipeline.recon.web.subdomain_takeover.SubjackScan`
|
|
- :class:`pipeline.recon.nmap.ThreadedNmapScan`
|
|
- :class:`pipeline.recon.web.subdomain_takeover.TKOSubsScan`
|
|
- :class:`pipeline.recon.web.webanalyze.WebanalyzeScan`
|
|
|
|
Additionally, two wrapper scans are made available. These execute multiple scans in a pipeline.
|
|
|
|
- :class:`pipeline.recon.wrappers.FullScan` - runs the entire pipeline
|
|
- :class:`pipeline.recon.wrappers.HTBScan` - nicety for hackthebox players (myself included) that omits the scans in FullScan that don't make sense for HTB
|
|
|
|
Example Scan
|
|
############
|
|
|
|
Here are the steps the video below takes to scan tesla[.]com.
|
|
|
|
Create a targetfile
|
|
|
|
.. code-block:: console
|
|
|
|
# use virtual environment
|
|
pipenv shell
|
|
|
|
# create targetfile; a targetfile is required for all scans
|
|
mkdir /root/bugcrowd/tesla
|
|
cd /root/bugcrowd/tesla
|
|
echo tesla.com > tesla-targetfile
|
|
|
|
# create a blacklist (if necessary based on target's scope)
|
|
echo energysupport.tesla.com > tesla-blacklist
|
|
echo feedback.tesla.com >> tesla-blacklist
|
|
echo employeefeedback.tesla.com >> tesla-blacklist
|
|
echo ir.tesla.com >> tesla-blacklist
|
|
|
|
# drop into the interactive shell
|
|
/root/PycharmProjects/recon-pipeline/pipeline/recon-pipeline.py
|
|
recon-pipeline>
|
|
|
|
|
|
Create a new database to store scan results
|
|
|
|
.. code-block:: console
|
|
|
|
recon-pipeline> database attach
|
|
1. create new database
|
|
Your choice? 1
|
|
new database name? (recommend something unique for this target)
|
|
-> tesla-scan
|
|
[*] created database @ /home/epi/.local/recon-pipeline/databases/tesla-scan
|
|
[+] attached to sqlite database @ /home/epi/.local/recon-pipeline/databases/tesla-scan
|
|
[db-1] recon-pipeline>
|
|
|
|
|
|
Scan the target
|
|
|
|
.. code-block:: console
|
|
|
|
[db-1] recon-pipeline> scan FullScan --exempt-list tesla-blacklist --target-file tesla-targetfile --interface eno1 --top-ports 2000 --rate 1200
|
|
[-] FullScan queued
|
|
[-] TKOSubsScan queued
|
|
[-] GatherWebTargets queued
|
|
[-] ParseAmassOutput queued
|
|
[-] AmassScan queued
|
|
[-] ParseMasscanOutput queued
|
|
[-] MasscanScan queued
|
|
[-] WebanalyzeScan queued
|
|
[-] SearchsploitScan queued
|
|
[-] ThreadedNmapScan queued
|
|
[-] SubjackScan queued
|
|
[-] AquatoneScan queued
|
|
[-] GobusterScan queued
|
|
[db-1] recon-pipeline>
|
|
|
|
.. raw:: html
|
|
|
|
<script id="asciicast-318397" src="https://asciinema.org/a/318397.js" async></script>
|
|
|