13 Commits

Author SHA1 Message Date
TheAuditorTool
4f6729d64c Fix: Handle null line/column values in ESLint and Bandit parsers
- Fix TypeError when linters return null instead of missing fields
  - Use 'or' operator to safely convert null to 0
  - Affects only JSON-based parsers (ESLint, Bandit)
  - Line 0 indicates file-level or configuration issues

  Fixes GitHub issue: "TypeError: '<' not supported between instances of 'str' and 'NoneType'"
2025-09-09 17:46:15 +07:00
TheAuditorTool
2520ad8698 Critical Fix: Implement adaptive timeouts to prevent long-running analysis tools from being killed
- Replace hardcoded 300s (5 min) timeout that was killing taint-analyze and detect-patterns
  - Add COMMAND_TIMEOUTS dictionary with appropriate timeouts per command type:
    * taint-analyze, detect-patterns: 2 hours (can take 10-60 min on large codebases)
    * lint: 15 minutes (ESLint/ruff on monorepos)
    * fce: 30 minutes (correlation analysis)
    * graph operations: 10 minutes
    * Quick operations (deps, docs, workset): 5 minutes
  - Add get_command_timeout() function to determine timeout based on command name
  - Support environment variable overrides:
    * Global: THEAUDITOR_TIMEOUT_SECONDS (default 30 min)
    * Per-command: THEAUDITOR_TIMEOUT_TAINT_ANALYZE_SECONDS, etc.
  - Update all 3 run_subprocess_with_interrupt calls to use adaptive timeouts
2025-09-09 17:22:12 +07:00
TheAuditorTool
c7b1f54ab8 Fix: Monorepo dependency parsing and docs fetch pipeline integration
- Add automatic detection of package.json in common monorepo patterns (backend/, frontend/, packages/*/, apps/*/,
  services/*/)
  - Fix pipeline deps.json path mismatch - deps writes to .pf/raw/deps.json but docs fetch was looking in
  .pf/deps.json
  - Add _parse_standalone_package_json() to handle subdirectory packages without workspace detection
  - Properly track workspace_package field with relative paths for all discovered package.json files

  Fixes PlantPro and similar monorepos where deps were not being detected (only finding 1 Docker dep instead of 100+
   npm deps)
2025-09-09 16:48:11 +07:00
TheAuditorTool
c7a59e420b Fix: Critical Windows ProcessPoolExecutor hang and documentation drift
Fixed critical Windows compatibility issues and updated outdated documentation.

  CRITICAL WINDOWS HANG FIXES:
  1. ProcessPoolExecutor → ThreadPoolExecutor
     - Fixes PowerShell/terminal hang where Ctrl+C wouldn't work
     - Prevents .pf directory lock requiring Task Manager kill
     - Root cause: Nested ProcessPool + ThreadPool on Windows creates kernel deadlock

  2. Ctrl+C Interruption Support
     - Replaced subprocess.run with Popen+poll pattern (industry standard)
     - Poll subprocess every 100ms for interruption checking
     - Added global stop_event and signal handlers for graceful shutdown
     - Root cause: subprocess.run blocks threads with no signal propagation

  DOCUMENTATION DRIFT FIX:
  - Removed hardcoded "14 phases" references (actual is 19+ commands)
  - Updated to "multiple analysis phases" throughout all docs
  - Fixed CLI help text to be version-agnostic
  - Added missing "Summary generation" step in HOWTOUSE.md

  Changes:
  - pipelines.py: ProcessPoolExecutor → ThreadPoolExecutor, added Popen+poll pattern
  - Added signal handling and run_subprocess_with_interrupt() function
  - commands/full.py: Updated docstring to remove specific phase count
  - README.md: Changed "14 distinct phases" to "multiple analysis phases"
  - HOWTOUSE.md: Updated phase references, added missing summary step
  - CLAUDE.md & ARCHITECTURE.md: Removed hardcoded phase counts

  Impact: Critical UX fixes - Windows compatibility restored, pipeline interruptible
  Testing: Ctrl+C works, no PowerShell hangs, .pf directory deletable
2025-09-09 14:26:18 +07:00
TheAuditorTool
e89c898c91 Fix: Complete monorepo support and versioned backups for deps command
Major fixes to deps command for proper monorepo and mixed-stack support:

  MONOREPO SUPPORT ISSUES FIXED:
  1. JavaScript/TypeScript monorepos - was only upgrading root package.json
     - Now handles: backend/frontend/packages/*/package.json
     - Supports npm workspaces, Lerna, pnpm workspaces

  2. Python monorepos - completely missed Python deps in subdirectories
     - Now scans: backend/requirements.txt, services/*/requirements.txt
     - Now scans: apps/*/pyproject.toml, backend/pyproject.toml
     - Common pattern fixed: Python backend + JS frontend mixed stacks

  3. Source tracking - requirements.txt always showed just filename
     - Now shows: backend/requirements.txt, services/api/requirements.txt
     - Enables proper grouping for upgrade operations

  BACKUP SAFETY ISSUE FIXED:
  4. Backup overwrite bug - would destroy original .bak on second run
     - Now creates: .bak, .bak.1, .bak.2 (versioned backups)
     - Users can safely run upgrade multiple times

  CHANGES:
  - Added subdirectory scanning for Python files (*/requirements*.txt, */pyproject.toml)
  - Fixed _parse_requirements_txt() and _parse_pyproject_toml() to track relative paths
  - Modified upgrade_all_deps() to handle workspace_package field for npm
  - Added proper source key handling for Python workspace files
  - Implemented _create_versioned_backup() for incremental backups
  - Updated all upgrade functions to use versioned backups

  TESTED WITH:
  ✓ "Private project" monorepo (3 package.json files detected correctly)
  ✓ Mixed Python/JS stacks (backend/requirements.txt + frontend/package.json)
  ✓ Versioned backups (creates .bak, .bak.1, .bak.2 without overwriting)
  ✓ Python subdirectory parsing (services/api/requirements.txt detected)

  Impact: Full monorepo support for real-world project structures
2025-09-09 13:20:23 +07:00
TheAuditorTool
576b3916da Fix: Increase pip install timeouts for slower systems and C compilation
Increased timeouts from 120s to 300s (5 minutes) for pip installations
  in venv_install.py to handle:
  - Tree-sitter C extension compilation on Linux
  - Slower internet connections downloading packages
  - Lower-spec machines taking longer to compile
  - Multiple large packages in [linters,ast] groups

  Affected operations:
  - deps --upgrade-all: Now 5 minutes (was 2 minutes)
  - pip install [linters,ast]: Now 5 minutes (was 2 minutes)

  Fixes timeout errors on Linux systems where tree-sitter compilation
  exceeds the previous 2-minute limit.
2025-09-08 14:44:34 +07:00
TheAuditorTool
fcaf983fc4 Subtle fallback bug treesitter(python/nix)
User reports are starting to come in which is amazing and invaluable feedback for me to fix errors outside "it works on my machine"

Include tree-sitter and AST tools in default setup

  Modified venv_install.py to install both [linters] and [ast] groups
  Tree-sitter is critical for accurate pattern detection but was missing
  Users on Linux/Mac were getting regex-only fallback detection
  Now all users get full AST-based vulnerability scanning
2025-09-08 14:24:49 +07:00
TheAuditorTool
8ffacca419 Critical bug in aud init.
Fix: Create .pf directory before writing files in indexer

  Commit Description:

  Fixed critical bug where `aud init` would fail on clean projects with:
  "Failed to write manifest: [Errno 2] No such file or directory"

  Cause:
  - The indexer tried to write to .pf/manifest.json without creating parent directory
  - `aud init` calls indexer directly, which had no directory creation logic
  - `aud full` worked because pipelines.py creates .pf/ before calling indexer

  Effect:
  - Users running `aud init` on fresh projects would get immediate failure
  - Only affected first-time users following README instructions
  - Did not affect `aud full` users or existing projects with .pf/ directory

  Solution:
  - Added Path().parent.mkdir(parents=True, exist_ok=True) before writing manifest
  - Added same logic before creating database file
  - Ensures .pf directory structure is created regardless of entry point

  This fix ensures both `aud init` and `aud full` work on clean projects.
2025-09-08 14:15:45 +07:00
TheAuditorTool
ca866e188a UX improvement for deps
Fixed UX for deeps, wasnt reporting what packages it was updating, was just silent...
Changed readme to remove the personal note.
2025-09-08 01:25:01 +07:00
TheAuditorTool
83e7dbd537 Fixed progressbar and init logic/command seperation
Progress bar was just shown at the end rather then running as steps...
also removed logic from the command file to clean it up.
2025-09-08 00:24:32 +07:00
TheAuditorTool
5ef25486a6 Pipeline order mismatch
aud graph analyze was running to late, causing no initial analysis to be found, causing simpler output data.
2025-09-07 23:54:48 +07:00
TheAuditorTool
f77173a551 Wrong checksums
Only windows packages for node portable had correct checksums. Rest was hallunicated by the AI during implementation. Sorry.
Updated and taken from https://nodejs.org/download/release/v20.11.1/SHASUMS256.txt
2025-09-07 23:02:46 +07:00
TheAuditorTool
ba5c287b02 Initial commit: TheAuditor v1.0.1 - AI-centric SAST and Code Intelligence Platform 2025-09-07 20:39:47 +07:00