feat: auth_format parameter for Level 2 authenticated scanning + report dir fix#4
Open
jordi-murgo wants to merge 2 commits into
Open
feat: auth_format parameter for Level 2 authenticated scanning + report dir fix#4jordi-murgo wants to merge 2 commits into
jordi-murgo wants to merge 2 commits into
Conversation
added 2 commits
May 22, 2026 10:48
When a scan has no target record in the database (e.g. deleted or orphaned), _find_report_dir would bail out early. Now falls through to a filesystem scan that matches report directories by scan_id in their validated_findings.json or raw_findings.json metadata. - Handle missing scan/target gracefully (was returning None) - Add filesystem fallback scanning all report dirs for scan_id - Check validated_findings.json and raw_findings.json for metadata match
Level 2 authentication now supports explicit auth_format to control how login credentials are sent: - "json" (default): sends credentials as JSON body via POST Works with REST APIs (e.g. Juice Shop /rest/user/login) - "form": extracts CSRF tokens from login page HTML, then sends credentials as application/x-www-form-urlencoded via POST Works with traditional web apps (e.g. DVWA /login.php) Changes: - scan_service.py: _setup_auth_tokens now accepts auth_format from ScanOptions, uses json= or data= accordingly, captures both JWT tokens and session cookies via client.cookies jar - gospider_agent.py: accepts scan_ctx_id, retrieves auth headers (cookies + JWT) from scan context, passes --cookie and -H Authorization to GoSpider - team.py: passes scan_context to GoSpiderAgent constructor - external.py: run_gospider accepts extra_headers, passes them as -H flags to the GoSpider CLI - scan_context.py: added auth_format field to ScanOptions - api/routes/scans.py: passes auth_format from request to ScanOptions - api/schemas.py: added auth_format field to CreateScanRequest Fixes authentication bypass in GoSpider where Level 2 sessions were started but cookies/JWT tokens were never forwarded to the crawler.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes for authenticated scanning and report lookups:
1.
auth_formatparameter for Level 2 login (feat)Level 2 authentication now supports an explicit
auth_formatparameter to control how login credentials are sent:"json"(default): sends credentials as JSON body viaPOST— works with REST APIs (e.g. Juice Shop/rest/user/login)"form": extracts CSRF tokens from the login page HTML, then sends credentials asapplication/x-www-form-urlencodedviaPOST— works with traditional web apps (e.g. DVWA/login.php)Files changed:
scan_service.py:_setup_auth_tokensusesauth_formatto choosejson=vsdata=, captures both JWT tokens and session cookies viaclient.cookiesgospider_agent.py: acceptsscan_ctx_id, retrieves auth headers (cookies + JWT) from scan context, passes--cookieand `-H Authorization to GoSpiderteam.py: passesscan_contexttoGoSpiderAgentconstructorexternal.py:run_gospideracceptsextra_headers, passes them as-Hflags to GoSpider CLIscan_context.py: addedauth_formatfield toScanOptionsapi/routes/scans.py: passesauth_formatfrom request toScanOptionsapi/schemas.py: addedauth_formatfield toCreateScanRequestFixes: GoSpider was starting authenticated sessions via Level 2 but cookies/JWT tokens were never forwarded to the crawler, making authenticated crawling ineffective.
Tested end-to-end:
auth_format="form"): 2 session cookies captured, CSRF token auto-extracted, GoSpider finds 5+ authenticated URLsauth_format="json"): JWT extracted from login response, passed asAuthorization: Bearerheader to GoSpider2. Report directory lookup fix (fix)
When a scan has no target record in the database (e.g. deleted or orphaned),
_find_report_dirwould bail out early. Now falls through to a filesystem scan that matches report directories byscan_idin theirvalidated_findings.jsonorraw_findings.jsonmetadata.Files changed:
reports.py: handle missing scan/target gracefully, add filesystem fallback scanning all report dirs for scan_idRelated: #3 (GoSpider auth headers issue)