Skip to content

feat: honor HTTP_PROXY/HTTPS_PROXY env vars when -proxy is unset #2493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mzack9999 merged 1 commit into from
May 13, 2026
+11 −1
Merged

feat: honor HTTP_PROXY/HTTPS_PROXY env vars when -proxy is unset #2493

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions common/httpx/httpx.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,8 @@ func New(options *Options) (*HTTPX, error) {
return nil, parseErr
}
transport.Proxy = http.ProxyURL(proxyURL)
} else {
transport.Proxy = http.ProxyFromEnvironment
}

httpx.client = retryablehttp.NewWithHTTPClient(&http.Client{
Expand Down
8 changes: 8 additions & 0 deletions runner/headless.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ func NewBrowser(proxy string, useLocal bool, optionalArgs map[string]string) (*B
}
}

if proxy == "" {
for _, k := range []string{"HTTPS_PROXY", "https_proxy", "HTTP_PROXY", "http_proxy"} {
if v := os.Getenv(k); v != "" {
proxy = v
break
}
}
}
Comment on lines +87 to +94
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

`For go-rod v0.116.2 / Chromium launch flags, what is the correct way to:

  1. apply different HTTP_PROXY and HTTPS_PROXY values, and
  2. honor NO_PROXY/no_proxy via proxy bypass settings
    when launching Chrome?`

💡 Result:

To launch Chromium with go-rod v0.116.2 using different HTTP_PROXY and HTTPS_PROXY values, and honoring NO_PROXY/no_proxy bypasses: Use the launcher.Set method with Chromium's --proxy-server flag for scheme-specific proxies, and --proxy-bypass-list for bypass rules matching NO_PROXY format. Example code: import ( \"github.com/go-rod/rod/lib/launcher\" \"github.com/go-rod/rod/lib/launcher/flags\" \"github.com/go-rod/rod\" ) httpProxy := \"http://http-proxy-host:8080\" httpsProxy := \"http://https-proxy-host:8443\" noProxy := \"localhost,127.0.0.1,.example.com\" // matches NO_PROXY env var format, comma-separated l := launcher.New(). Set(\"--proxy-server\", \"http=\"+httpProxy+\";https=\"+httpsProxy). Set(\"--proxy-bypass-list\", noProxy). Headless(true) // or false controlURL := l.MustLaunch() defer l.Cleanup() browser := rod.New().ControlURL(controlURL).MustConnect() defer browser.MustClose() // Test page := browser.MustPage(\"https://httpbin.org/ip\") println(page.MustElement(\"body\").MustText()) Key points: - --proxy-server="http=PROXY;https=PROXY" sets different proxies for HTTP vs HTTPS schemes [1][2][3][4]. - --proxy-bypass-list="host1,host2,domain.com" bypasses proxy for matching hosts/domains (semicolon or comma separated), directly honors NO_PROXY format [5][1][6][7]. - Use launcher.Proxy(PROXY) or launcher.Set(flags.ProxyServer, PROXY) for single proxy [3][8][6]. - Environment variables like HTTP_PROXY are not directly used by Chromium command line; flags override [9][10]. - For auth, embed in proxy URL like "http://user:pass@host:port\" [8][11], or use browser.MustHandleAuth [3]. - Confirmed in go-rod examples and issues for v0.116.x [6][3][12].

Citations:

🏁 Script executed:

# View the actual code at lines 87-94 in runner/headless.go
head -95 runner/headless.go | tail -20

Repository: projectdiscovery/httpx

Length of output: 641

🏁 Script executed:

# Search for NO_PROXY handling in the file
rg -n "NO_PROXY|no_proxy" runner/headless.go

Repository: projectdiscovery/httpx

Length of output: 48

🏁 Script executed:

# Search for any proxy-related environment variable handling in the repository
rg -n "HTTP_PROXY|HTTPS_PROXY|proxy" runner/headless.go -A 2 -B 2

Repository: projectdiscovery/httpx

Length of output: 542

🏁 Script executed:

# Search for chromeLauncher.Proxy usage and context in the file
rg -n "Proxy" runner/headless.go -B 2 -A 2

Repository: projectdiscovery/httpx

Length of output: 153

🏁 Script executed:

# Check if there's any TODO or FIXME related to proxy handling
rg -n "TODO|FIXME|proxy|PROXY" runner/headless.go

Repository: projectdiscovery/httpx

Length of output: 446

🏁 Script executed:

# Check the go.mod or imports to understand which go-rod version is being used
fd "go.mod" | head -5 | xargs cat

Repository: projectdiscovery/httpx

Length of output: 8886

🏁 Script executed:

# Search for .Set usage with chromeLauncher in the file
rg -n "chromeLauncher\." runner/headless.go -A 1

Repository: projectdiscovery/httpx

Length of output: 364

🏁 Script executed:

# Check for launcher package imports and usage patterns
rg -n "launcher\." runner/headless.go | head -20

Repository: projectdiscovery/httpx

Length of output: 170

Headless env-proxy resolution drops NO_PROXY and collapses HTTP/HTTPS into one proxy.

Lines 87–94 pick a single env proxy and never consider NO_PROXY/no_proxy. This can route hosts through the proxy that should bypass it, and mishandles setups where HTTP_PROXY and HTTPS_PROXY differ.

Chromium requires --proxy-server="http=PROXY;https=PROXY" (scheme-specific) and --proxy-bypass-list="..." to honor NO_PROXY. The .Set() mechanism already exists in the code (line 100); use it to construct and pass these flags from the environment variables instead of collapsing them into a single .Proxy() call.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@runner/headless.go` around lines 87 - 94, The current logic collapses proxies
into a single proxy value and ignores NO_PROXY; update the proxy resolution to
read HTTP_PROXY/https_proxy and HTTPS_PROXY/https_proxy separately along with
NO_PROXY/no_proxy, and when any are present use the existing Set() path (not
.Proxy()) to add Chromium flags: construct
--proxy-server="http=HTTP_PROXY;https=HTTPS_PROXY" using only the schemes that
exist (omit missing schemes) and add --proxy-bypass-list="NO_PROXY_VALUE" when
NO_PROXY is set; ensure you still respect an already-set proxy variable (proxy)
if provided but prefer explicit per-scheme env vars when present and remove the
.Proxy() call in favor of setting these flags via Set().

if proxy != "" {
chromeLauncher = chromeLauncher.Proxy(proxy)
}
Expand Down
2 changes: 1 addition & 1 deletion runner/options.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,7 @@ func ParseOptions() *Options {
flagSet.BoolVar(&options.RandomAgent, "random-agent", true, "enable Random User-Agent to use"),
flagSet.BoolVar(&options.AutoReferer, "auto-referer", false, "set the Referer header to the current URL"),
flagSet.VarP(&options.CustomHeaders, "header", "H", "custom http headers to send with request"),
flagSet.StringVarP(&options.Proxy, "proxy", "http-proxy", "", "proxy (http|socks) to use (eg http://127.0.0.1:8080)"),
flagSet.StringVarP(&options.Proxy, "proxy", "http-proxy", "", "proxy (http|socks) to use (eg http://127.0.0.1:8080); falls back to HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars"),
flagSet.BoolVar(&options.Unsafe, "unsafe", false, "send raw requests skipping golang normalization"),
flagSet.BoolVar(&options.Resume, "resume", false, "resume scan using resume.cfg"),
flagSet.BoolVarP(&options.FollowRedirects, "follow-redirects", "fr", false, "follow http redirects"),
Expand Down
Loading