Skip to content

Add headers to .render() #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bwacek wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add headers to .render() #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion requests_html.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,7 @@ def __next__(self):
def add_next_symbol(self, next_symbol):
self.next_symbol.append(next_symbol)

def render(self, retries: int = 8, script: str = None, wait: float = 0.2, scrolldown=False, sleep: int = 0, reload: bool = True, timeout: Union[float, int] = 8.0, keep_page: bool = False):
def render(self, retries: int = 8, script: str = None, wait: float = 0.2, scrolldown=False, sleep: int = 0, reload: bool = True, timeout: Union[float, int] = 8.0, keep_page: bool = False, headers: dict = {}):
"""Reloads the response in Chromium, and replaces HTML content
with an updated version, with JavaScript executed.

Expand All @@ -497,6 +497,7 @@ def render(self, retries: int = 8, script: str = None, wait: float = 0.2, scroll
:param sleep: Integer, if provided, of how many long to sleep after initial render.
:param reload: If ``False``, content will not be loaded from the browser, but will be provided from memory.
:param keep_page: If ``True`` will allow you to interact with the browser page through ``r.html.page``.
:param headers: The additional headers added to the page.

If ``scrolldown`` is specified, the page will scrolldown the specified
number of times, after sleeping the specified amount of time
Expand Down Expand Up @@ -537,6 +538,10 @@ async def _async_render(*, url: str, script: str = None, scrolldown, sleep: int,
try:
page = await self.session.browser.newPage()

# Add additional headers to the page
if headers:
await page.setExtraHTTPHeaders(headers)

# Wait before rendering the page, to prevent timeouts.
await asyncio.sleep(wait)

Expand Down