Skip to content

Update fingerprintx to 1.1.19 - #3672

Open
blsaccess wants to merge 1 commit into
devfrom
update-fingerprintx
Open

blsaccess wants to merge 1 commit into
devfrom
update-fingerprintx

Conversation

@blsaccess

Copy link
Copy Markdown
Contributor

This PR uses https://api.github.com/repos/praetorian-inc/fingerprintx/releases/latest to obtain the latest version of fingerprintx and update the version in bbot/modules/fingerprintx.py.

Release notes:

What's Changed

FTP Plugin Enhancements

  • Fix false positives on port 25 - FTP plugin no longer incorrectly detects SMTP services (fixed wildcard detection bug, added helpful log message #55)
  • Whitelist-based detection - Uses FTP keyword matching instead of blacklist approach
  • Confidence scoring - Returns high/medium/low confidence based on port and keywords
  • Version extraction - Extracts version from vsftpd, ProFTPD, Pure-FTPd, FileZilla, IIS FTP, wu-ftpd
  • CPE generation - Generates CPE strings for vulnerability tracking (uses * for unknown versions)
  • 78 unit tests - Comprehensive test coverage for detection logic

Detection Strategy

Scenario Result
Port 21 + FTP keyword HIGH confidence FTP
Non-standard port + FTP keyword MEDIUM confidence FTP
Port 25 + No FTP keyword REJECTED (not FTP)

Full Changelog: praetorian-inc/fingerprintx@v1.1.18...v1.1.19

@blsaccess
blsaccess requested a review from a team as a code owner November 21, 2026 00:20
@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Benchmark Report

Comparing dev (2628c9ef5) (baseline) vs update-fingerprintx (a584810ad) (current)

📈 Detailed Results (All Benchmarks)

📋 Complete results for all benchmarks - includes both significant and insignificant changes

🧪 Test Name 📏 Base 📏 Current 📈 Change 🎯 Status
Bloom Filter Dns Mutation Tracking Performance 2.22ms 2.25ms +1.7% ⚪ ✅
Bloom Filter Large Scale Dns Brute Force 9.30ms 9.47ms +1.9% ⚪ ✅
Large Closest Match Lookup 216.91ms 218.35ms +0.7% ⚪ ✅
Realistic Closest Match Workload 117.94ms 118.59ms +0.5% ⚪ ✅
Event Memory Medium Scan 1389 B/event 1389 B/event +0.0% ⚪ ✅
Event Memory Large Scan 1514 B/event 1514 B/event +0.0% ⚪ ✅
Event Validation Full Scan Startup Small Batch 21.01ms 21.25ms +1.1% ⚪ ✅
Event Validation Full Scan Startup Large Batch 111.74ms 104.88ms -6.1% ⚪ ✅
Make Event Autodetection Small 16.40ms 16.38ms -0.2% ⚪ ✅
Make Event Autodetection Large 165.02ms 164.35ms -0.4% ⚪ ✅
Make Event Explicit Types 6.93ms 6.91ms -0.4% ⚪ ✅
Excavate Single Thread Small 2.907s 2.569s -11.6% 🟢🟢 🚀
Excavate Single Thread Large 7.029s 6.817s -3.0% ⚪ ✅
Excavate Parallel Tasks Small 3.095s 2.782s -10.1% 🟢🟢 🚀
Excavate Parallel Tasks Large 4.602s 4.418s -4.0% ⚪ ✅
Intercept Throughput Small 352.45ms 354.58ms +0.6% ⚪ ✅
Intercept Throughput Medium 391.40ms 351.77ms -10.1% 🟢🟢 🚀
Dns Throughput Quiet 1.229s 1.251s +1.8% ⚪ ✅
Dns Throughput Loaded 710.42ms 685.13ms -3.6% ⚪ ✅
Dns Throughput Inherited 1.016s 1.063s +4.6% ⚪ ✅
Is Ip Performance 1.37ms 1.40ms +2.2% ⚪ ✅
Make Ip Type Performance 112.14µs 114.32µs +1.9% ⚪ ✅
Mixed Ip Operations 1.42ms 1.43ms +0.5% ⚪ ✅
Memory Use Web Crawl 1.1 MB 1.1 MB +0.0% ⚪ ✅
Memory Use Subdomain Enum 27.8 MB 27.8 MB +0.0% ⚪ ✅
Memory Use Deep Chain 5.1 MB 5.1 MB +0.0% ⚪ ✅
Memory Use Parallel Chains 11.6 MB 12.1 MB +4.0% ⚪ ✅
Scan Throughput 100 1.606s 1.752s +9.1% ⚪ ✅
Scan Throughput 1000 12.898s 13.045s +1.1% ⚪ ✅
Typical Queue Shuffle 3.57µs 3.55µs -0.7% ⚪ ✅
Priority Queue Shuffle 17.46µs 17.46µs -0.0% ⚪ ✅

🎯 Performance Summary

+ 3 improvements 🚀
  28 unchanged ✅

🔍 Significant Changes (>10%)

  • Excavate Single Thread Small: 11.6% 🚀 faster
  • Excavate Parallel Tasks Small: 10.1% 🚀 faster
  • Intercept Throughput Medium: 10.1% 🚀 faster

🐍 Python Version 3.12.3

@en0f en0f left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 305b43e.

🔴 fingerprintx 1.1.19 ships no release binaries, so the module can't install

deps_ansible downloads fingerprintx_<ver>_<os>_<arch>.tar.gz from the release this pin names, and upstream has attached no assets to any release since v1.1.14, so every platform gets a 404. That is the red CI on all five Python jobs: the download 404s, both fingerprintx module tests error on missing deps, and test_cli_args fails on --install-all-deps. Merged, any scan that includes fingerprintx (-m fingerprintx, tech-detect, -f service-enum) aborts at setup with a ScanError unless --force is given, including for existing users once they upgrade bbot, since the deps cache is keyed on bbot's version. AGENTS.md § Testing Principle asks that a changed module's test still pass.

Fix: keep 1.1.4, or pin 1.1.13 (the newest release that still ships binaries), or build from source with go install github.com/praetorian-inc/fingerprintx/cmd/fingerprintx@v1.1.19, which needs a Go toolchain dependency bbot doesn't have yet. version-updater.yml reads releases/latest without checking for assets, so it will reopen this PR daily until it skips releases that have no matching asset.

Reproduce this review
# save the script below as verify-3672.sh, then:
bash verify-3672.sh blacklanternsecurity/bbot 305b43e8c84b398bede65065d7defda56afe9401
# expected: SUMMARY 0 PASS, 5 FAIL, 0 SKIP, 0 BROKEN
# control:  bash verify-3672.sh blacklanternsecurity/bbot 2628c9ef505b326f45914903db3dbd2f7b06b18a   -> V01..V05 PASS (SUMMARY 5 PASS, 0 FAIL, 0 SKIP, 0 BROKEN)
#!/usr/bin/env bash
# verify-3672.sh: reproducible checks for blacklanternsecurity/bbot#3672
# Usage: bash verify-3672.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
#
# The claim: the fingerprintx version pinned in bbot/modules/fingerprintx.py must
# name an upstream release whose binary archives exist, because deps_ansible
# downloads one of them. The base pins 1.1.4 (ships binaries); the head pins 1.1.19.
#
# CONTROL: sha 2628c9ef505b326f45914903db3dbd2f7b06b18a
# CONTROL-EXPECT: V01=PASS V02=PASS V03=PASS V04=PASS V05=PASS
# AUTHOR-CODE: none
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "curl $(curl --version | head -1 | cut -d' ' -f2)"
echo "gh $(gh --version | head -1 | cut -d' ' -f3)"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi

# ---- checks: each line states the desired property --------------------------
MOD=bbot/modules/fingerprintx.py
VER=$(sed -nE 's/^[[:space:]]+version: VersionStr = Field\("([^"]+)".*/\1/p' "$MOD" | head -1)
TPL=$(sed -nE 's/^[[:space:]]+"src": "(https:\/\/github\.com\/praetorian-inc\/fingerprintx\/releases\/download\/[^"]+)",.*/\1/p' "$MOD" | head -1)
if [ -z "$VER" ] || [ -z "$TPL" ]; then
  broken V00 "read version and download URL from $MOD" "version='$VER' template='$TPL'"; summary; exit 1
fi
echo "INFO pinned version $VER"
echo "INFO url template $TPL"

# V01: the pinned upstream release publishes binary assets at all
ASSETS=$(gh api --method GET "repos/praetorian-inc/fingerprintx/releases/tags/v$VER" --jq '.assets | length' 2>/dev/null)
if [ -z "$ASSETS" ]; then
  skip V01 "release v$VER publishes binary assets" "release lookup failed"
elif [ "$ASSETS" -gt 0 ]; then
  pass V01 "release v$VER publishes binary assets ($ASSETS)"
else
  fail V01 "release v$VER publishes binary assets" "release has 0 assets"
fi

# V02..V05: the exact URL deps_ansible builds downloads, for each platform bbot supports
n=2
for plat in linux_amd64 linux_arm64 darwin_amd64 darwin_arm64; do
  os=${plat%_*}; arch=${plat#*_}
  url=${TPL//\#\{BBOT_MODULES_FINGERPRINTX_VERSION\}/$VER}
  url=${url//\#\{BBOT_OS_PLATFORM\}/$os}
  url=${url//\#\{BBOT_CPU_ARCH_GOLANG\}/$arch}
  id=$(printf 'V%02d' "$n"); n=$((n+1))
  code=$(curl -s -o /dev/null -L -I -w '%{http_code}' --max-time 30 "$url")
  if [ "$code" = "200" ]; then
    pass "$id" "deps_ansible download for $plat resolves (HTTP 200): $url"
  elif [ "$code" = "000" ]; then
    skip "$id" "deps_ansible download for $plat resolves" "network error for $url"
  else
    fail "$id" "deps_ansible download for $plat resolves" "HTTP $code for $url"
  fi
done
# ------------------------------------------------------------------------------
summary


class Config(BaseModuleConfig):
version: VersionStr = Field("1.1.4", description="fingerprintx version")
version: VersionStr = Field("1.1.19", description="fingerprintx version")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 v1.1.19 has no release assets (none since v1.1.14), so the download on line 28 404s on every platform and the module can't install. Keep 1.1.4 or pin 1.1.13, the newest release with binaries.

@singlerider singlerider added the enhancement New feature or request label Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants