Skip to content

Context canceled for JS loading site #1214

@suntong

Description

@suntong

And here's an example for chromedp:

// The CSS selector for the element you must wait for
const selectorToWaitFor = "a" 

var htmlContent string
const targetURL = "https://example.com"

log.Printf("Navigating to %s, waiting for '%s', and retrieving source...", targetURL, selectorToWaitFor)

err := chromedp.Run(ctx,
    // 1. Navigate to the target URL
    chromedp.Navigate(targetURL),
    
    // 2. Add the waiting condition here
    // This pauses execution until the element is visible in the DOM
    chromedp.WaitVisible(selectorToWaitFor, chromedp.ByQuery),
    
    // 3. Retrieve the OuterHTML of the entire 'html' element
    chromedp.OuterHTML(`html`, &htmlContent, chromedp.ByQuery),
)

It works for example.com, but not another site (with the same code) which uses JS to load its content (targetURL = "https://so.gitee.com/?q=Vue%20go", selectorToWaitFor = "a[aria-label=Next]") --

I'm using the latest github.com/chromedp/chromedp (v0.14.2), and

I set a generous overall timeout using context.WithTimeout (300 seconds) for the entire operation. However, during execution of specific actions (chromedp.WaitVisible), the program consistently fails with a "context canceled" error after approximately 7 seconds. The actual body returned is (for sure the element is not found):

 <div class="d-flex justify-content-center align-middle" style="height: 80vh;"><div class="m-auto"><div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div> <div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div> <div class="spinner-grow spinner-grow-sm text-success" role="status"><span class="visually-hidden">Loading...</span></div></div></div></body></html>

The site is heavy JS based, the initial load takes 6~7 seconds. then the browser should run JS to finish loading.

Using go-rod, and waiting for the same element, it can retrieve the whole JS loaded content.

The issue might be a lightpanda cloud browser issue, as I am using Google Chrome browser using browser=chrome parameter in the url for go-rod, yet when adding browser=chrome parameter in the url for chromedp, it failed with the following error (so I'm not sure where exactly the problem is):

Visiting 'https://example.com' (& wait for 'a') ...
2025/11/14 09:26:50 -> {"id":1,"method":"Target.createTarget","params":{"url":"about:blank","enableBeginFrameControl":false,"newWindow":false,"background":false,"forTab":false,"hidden":false}}
2025/11/14 09:26:51 <- {"id":1,"error":{"code":-32000,"message":"Failed to open new tab - no browser is open"}}
...
exit status 1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions