Skip to content

feat: add --auto-wildcard (-aw) flag for per-domain wildcard detection#959

Open
CharlesWong wants to merge 1 commit intoprojectdiscovery:mainfrom
CharlesWong:feat/auto-wildcard-detection
Open

feat: add --auto-wildcard (-aw) flag for per-domain wildcard detection#959
CharlesWong wants to merge 1 commit intoprojectdiscovery:mainfrom
CharlesWong:feat/auto-wildcard-detection

Conversation

@CharlesWong
Copy link

Problem

dnsx requires manually specifying -wd per domain. When scanning mixed input (multiple root domains), this is impractical — you'd need to know which domains are wildcards ahead of time.

Solution

Add --auto-wildcard (-aw) flag that automatically:

  1. Probes each unique root domain (eTLD+1) with 3 random subdomains for accuracy
  2. Detects wildcard DNS responses for A, AAAA, and CNAME record types
  3. Filters matching results inline during resolution (not post-processing)
  4. Caches results per domain for efficiency across concurrent workers

Key improvements over competing PRs

Feature This PR Others
Probes per domain 3 (reduces false negatives) 1
CNAME wildcard support Most: ❌
Inline filtering (not post-processing) Mixed
Thread-safe cache (RWMutex + double-check lock) Mixed
eTLD+1 via golang.org/x/net/publicsuffix ✅ (already in go.mod) Some
IP address rejection Some
Unit tests ✅ (extractRootDomain, match logic, cache thread-safety) Mixed
-aw and -wd mutual exclusion validation Most
stream mode guard Most
README documentation Some

How it works

dnsx -l subdomain_list.txt -aw
[INF] [auto-wildcard] Detected wildcard domain: *.dev.example.com
api.example.com
www.example.com

For each resolved host, dnsx:

  1. Extracts the eTLD+1 root domain (e.g. api.dev.example.comexample.com)
  2. If not yet probed: fires 3 random subdomain queries (A + CNAME) and stores the fingerprint
  3. Compares the resolved A/AAAA/CNAME records against the fingerprint
  4. Filters if any record matches; passes through if records differ (real override)

Compatibility

  • -aw and -wd are mutually exclusive (error if both specified)
  • -aw is disabled in stream mode (same as -wd)
  • Works with -re, -ro, -j output modes
  • No new dependencies (uses golang.org/x/net already in go.mod)

Files changed

  • libs/dnsx/dnsx.go: add QueryType() for explicit record type probing
  • internal/runner/wildcard.go: all auto-wildcard logic (fingerprinting, caching, matching)
  • internal/runner/runner.go: autoWildcardMu/autoWildcardCache fields, inline filter in worker(), pass wildcardDomain to IsWildcard()
  • internal/runner/options.go: AutoWildcard field, -aw flag, validation
  • internal/runner/wildcard_auto_test.go: comprehensive unit tests
  • README.md: auto-wildcard documentation with examples

Closes #924

…tion (projectdiscovery#924)

- Add -aw/--auto-wildcard flag that auto-detects wildcard DNS per root domain
  and filters matching results without requiring manual -wd specification
- Use 3 random probes per domain (reduces false negatives vs single probe)
- Detect CNAME wildcards in addition to A/AAAA records
- Thread-safe wildcard cache with RWMutex and double-check locking pattern
- Proper eTLD+1 extraction via golang.org/x/net/publicsuffix (handles co.uk etc)
- IP address rejection to avoid false wildcard matches on reverse-DNS targets
- Inline filtering during resolution (not post-processing) for efficiency
- Add QueryType() method to dnsx lib for explicit record type probing
- Validation: -aw and -wd cannot be used together; -aw not supported in stream mode
- Integrates seamlessly with existing -wd logic (IsWildcard now takes explicit domain param)
- Full unit test coverage: extractRootDomain, isAutoWildcardMatch, cache thread-safety
- README documentation with usage examples and comparison to -wd

Closes projectdiscovery#924
@coderabbitai
Copy link

coderabbitai bot commented Mar 11, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 908cd900-c760-4523-9a16-7c52580c31f7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@CharlesWong
Copy link
Author

Thanks for the summary! Happy to address any specific feedback. The key improvements over other PRs: 3 probes per domain (reduces false negatives), CNAME wildcard detection, thread-safe RWMutex cache, and mutual exclusion with -wd flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant