Files
hackingtool/tests/test_cli.py
T
Hardik Zinzuvadiya 2482cc9fd0 feat: AI operator console — 215 curated tools, AI layer, /find discovery (v2.0.0 rework)
Squashed rework of hackingtool from a tool launcher into an AI-guided operator
console for authorized security testing. 93 commits collapsed into this one;
the pre-rework tree is tagged v2.0.0.

CATALOG & ENGINE
- Data-driven YAML catalog (21 categories, 215 live tools + 59 archived) with a
  registry/overlay loader and a fixed 66-tag taxonomy (63 in use). Adding a tool
  is one YAML entry, not edits across the codebase.
- Engine honesty: real exit codes, truthful install success/failure, reuse-first
  skip, EOF-safe prompts, command audit logging.
- Safe installs: sha256-required safe-fetch (killed `curl | bash` in feroxbuster,
  Caido and Sliver), list-form subprocess only, no forced sudo.

AI LAYER (bring-your-own-key or local model; degrades offline, never fabricates)
- AI1 intent -> tools; AI2 tool+goal -> command, curated-first with a grounded
  fallback; AI3 findings summary and engagement report; AI4 per-finding impact
  and remediation. Prompt-injection hardened per OWASP LLM01.
- /goal plans an objective and runs it one step at a time, showing every command
  before it runs, with a plan.json + run.log audit trail.

/find TOOL DISCOVERY (this branch's headline feature)
- Suggests real GitHub projects when the catalog has no tool for a need.
  Deterministic: zero model calls, structured API fields only, suggest-only —
  it never clones, installs or runs anything.
- A charter filter refuses destructive/DoS/jamming/mass-targeting/evasion asks
  before any network I/O, while a defensive-intent guard keeps blue-team and
  DFIR phrasing ("detect a SYN flood in a pcap") from being false-refused.
- Query rewriting proved to be the dominant quality lever (the first design
  measured 29% precision with no results on 5 of 8 needs): a curated 41-row
  intent table maps plain English to canonical jargon plus a GitHub topic, and
  a two-arm search unions topic coverage with jargon precision.
- Explainable additive ranking: log-flattened stars, license/age/language,
  trusted-author bonus derived from owners we already ship, docs-repo demotion
  by name, staleness as a soft demotion rather than a filter (a hard cutoff
  would delete THC-Hydan and John the Ripper), and a relevance term weighting
  curated topics above free-text description.
- Optional no-scope GitHub token purely as a rate-limit lever (10 -> 30 req/min);
  it reaches only an Authorization header, never a cache key, log or output.
- `[a]` saves a pick to ~/.hackingtool/found.yaml as a structurally inert entry,
  and the loader strips executable keys from user catalogs at read time so a
  hand-edited file cannot become a runnable command.

CONSOLE & PACKAGING
- REPL with a / command palette, @ tool mentions, tag filters, history and
  completion; background tmux panes; settings and first-run scaffolding.
- src-layout package with catalog and pipelines as package data, console entry
  point, Docker image, signed releases with SBOM and build provenance.
- Health docs (SECURITY, CONTRIBUTING, CHANGELOG, CODE_OF_CONDUCT), a CI gate
  (ruff + pytest + catalog/taxonomy conformance) and a pre-push hook.
- README rewritten with a section index, the tool catalog split into
  docs/TOOLS.md and a step-by-step docs/HOW-TO-USE.md.

278 tests passing; scripts/check.sh green.
2026-07-27 00:18:34 +05:30

103 lines
4.2 KiB
Python

import hackingtool.cli as hackingtool
from hackingtool.tags import TAXONOMY
def test_recommendation_tags_are_taxonomy_valid():
# Every curated shortcut must map onto the one canonical vocabulary — guards
# against typos like the old "port-scanner" (should be "port-scan").
bad = {phrase: [t for t in tags if t not in TAXONOMY]
for phrase, tags in hackingtool._RECOMMENDATIONS.items()
if any(t not in TAXONOMY for t in tags)}
assert not bad, bad
def test_tag_index_speaks_one_vocabulary_from_real_tags():
idx = hackingtool._get_all_tags()
# The whole index is taxonomy-valid (real tags + taxonomy-valid regex fallback).
assert set(idx) <= set(TAXONOMY), sorted(set(idx) - set(TAXONOMY))
# A tagged tool contributes exactly its real TAGS — no regex bleed.
for tool, cat in hackingtool._collect_all_tools():
if getattr(tool, "TAGS", None):
here = {tg for tg in idx if (tool, cat) in idx[tg]}
assert here == set(tool.TAGS), (tool.TITLE, here, tool.TAGS)
break
def test_legacy_overlay_titles_match_real_tools():
# Overlays apply by exact title — a typo would silently no-op. Guard it.
import yaml
import hackingtool.registry as registry
f = registry.CATALOG_DIR / "legacy_overlays.yaml"
entries = (yaml.safe_load(f.read_text()) or {}).get("overlay", [])
titles = {t.TITLE for t, _ in hackingtool._collect_all_tools()}
missing = [e["title"] for e in entries if e["title"] not in titles]
assert not missing, missing
assert len(entries) >= 90 # the curated legacy batch landed
def test_free_text_routes_to_recommend(monkeypatch):
# NL-first: a plain intent string goes to the AI1 free-text path.
seen = {}
monkeypatch.setattr(hackingtool, "_recommend_freetext",
lambda intent: seen.setdefault("intent", intent))
hackingtool.recommend_tools("crack a wifi handshake")
assert seen["intent"] == "crack a wifi handshake"
def test_arg_parser_flags():
p = hackingtool._build_arg_parser()
ns = p.parse_args(["--engagement", "acme", "--targets", "example.com", "--ai-summary"])
assert ns.engagement == "acme"
assert ns.targets == "example.com"
assert ns.pipeline is None # default
assert ns.ai_summary is True
assert ns.report is False
def test_pipeline_runs_against_stored_targets(tmp_path, monkeypatch):
import hackingtool.engagement as engagement
monkeypatch.setattr(engagement, "ENGAGEMENTS_ROOT", tmp_path)
engagement.create("acme", targets=["example.com"]) # targets already stored
called = {}
def fake_run(e, name):
called["run"] = name
return []
monkeypatch.setattr(hackingtool.orchestrator, "run_pipeline", fake_run)
monkeypatch.setattr(hackingtool.report, "generate_report", lambda e: e.report_file)
args = hackingtool._build_arg_parser().parse_args(["--engagement", "acme", "--pipeline", "recon"])
hackingtool._run_headless(args)
assert called["run"] == "recon" # ran with NO --targets
def test_load_targets_inline_vs_file(tmp_path):
assert hackingtool._load_targets("example.com") == ["example.com"]
f = tmp_path / "t.txt"
f.write_text("a.com\nb.com\n\n")
assert hackingtool._load_targets(str(f)) == ["a.com", "b.com"]
def test_run_headless_dispatches(tmp_path, monkeypatch):
import hackingtool.engagement as engagement
monkeypatch.setattr(engagement, "ENGAGEMENTS_ROOT", tmp_path)
called = {}
def fake_run(e, name):
called["run"] = name
return []
monkeypatch.setattr(hackingtool.orchestrator, "run_pipeline", fake_run)
def fake_report(e):
called["report"] = True
return e.report_file
monkeypatch.setattr(hackingtool.report, "generate_report", fake_report)
def fake_summary(e):
called["summary"] = True
return "ok"
monkeypatch.setattr(hackingtool.ai_summary, "summarize", fake_summary)
args = hackingtool._build_arg_parser().parse_args(
["--engagement", "acme", "--targets", "example.com", "--ai-summary"])
hackingtool._run_headless(args)
assert called["run"] == "recon"
assert called["report"] is True
assert called["summary"] is True