feat: add --auto-wildcard (-aw) flag for per-domain wildcard detection#959
feat: add --auto-wildcard (-aw) flag for per-domain wildcard detection#959CharlesWong wants to merge 1 commit intoprojectdiscovery:mainfrom
Conversation
…tion (projectdiscovery#924) - Add -aw/--auto-wildcard flag that auto-detects wildcard DNS per root domain and filters matching results without requiring manual -wd specification - Use 3 random probes per domain (reduces false negatives vs single probe) - Detect CNAME wildcards in addition to A/AAAA records - Thread-safe wildcard cache with RWMutex and double-check locking pattern - Proper eTLD+1 extraction via golang.org/x/net/publicsuffix (handles co.uk etc) - IP address rejection to avoid false wildcard matches on reverse-DNS targets - Inline filtering during resolution (not post-processing) for efficiency - Add QueryType() method to dnsx lib for explicit record type probing - Validation: -aw and -wd cannot be used together; -aw not supported in stream mode - Integrates seamlessly with existing -wd logic (IsWildcard now takes explicit domain param) - Full unit test coverage: extractRootDomain, isAutoWildcardMatch, cache thread-safety - README documentation with usage examples and comparison to -wd Closes projectdiscovery#924
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Thanks for the summary! Happy to address any specific feedback. The key improvements over other PRs: 3 probes per domain (reduces false negatives), CNAME wildcard detection, thread-safe RWMutex cache, and mutual exclusion with -wd flag. |
Problem
dnsx requires manually specifying
-wdper domain. When scanning mixed input (multiple root domains), this is impractical — you'd need to know which domains are wildcards ahead of time.Solution
Add
--auto-wildcard(-aw) flag that automatically:Key improvements over competing PRs
golang.org/x/net/publicsuffixHow it works
For each resolved host, dnsx:
api.dev.example.com→example.com)Compatibility
-awand-wdare mutually exclusive (error if both specified)-awis disabled in stream mode (same as-wd)-re,-ro,-joutput modesgolang.org/x/netalready in go.mod)Files changed
libs/dnsx/dnsx.go: addQueryType()for explicit record type probinginternal/runner/wildcard.go: all auto-wildcard logic (fingerprinting, caching, matching)internal/runner/runner.go:autoWildcardMu/autoWildcardCachefields, inline filter inworker(), pass wildcardDomain toIsWildcard()internal/runner/options.go:AutoWildcardfield,-awflag, validationinternal/runner/wildcard_auto_test.go: comprehensive unit testsREADME.md: auto-wildcard documentation with examplesCloses #924