Skip to content

Related to [SCR-45]: Add support for multiple ScrapingBee APIs (Google, Amazon, Walmart, YouTube, ChatGPT) + Usage endpoint#20

Merged
sahilsunny merged 7 commits intomasterfrom
add-other-apis
Jan 19, 2026
Merged

Related to [SCR-45]: Add support for multiple ScrapingBee APIs (Google, Amazon, Walmart, YouTube, ChatGPT) + Usage endpoint#20
sahilsunny merged 7 commits intomasterfrom
add-other-apis

Conversation

@sahilsunny
Copy link
Contributor

  • Added htmlApi() method for HTML API
  • Added googleSearch() method for Google Search API
  • Added amazonSearch() method for Amazon Search API
  • Added amazonProduct() method for Amazon Product API
  • Added walmartSearch() method for Walmart Search API
  • Added walmartProduct() method for Walmart Product API
  • Added youtubeSearch() method for YouTube Search API
  • Added youtubeMetadata() method for YouTube Metadata API
  • Added youtubeTranscript() method for YouTube Transcript API
  • Added youtubeTrainability() method for YouTube Trainability API
  • Added chatGPT() method for ChatGPT API
  • Added usage() method for Usage endpoint
  • Added new HTML API parameters: ai_query, ai_selector, return_page_markdown, return_page_text, scraping_config
  • Refactored internal request() method to be API-agnostic

… Amazon Search API

Changed the request method such that it can work with all of our APIs and the test was successful. The get, post, googleSearch, amazonSearch methods are working correctly. So now we just need to add the rest of the APIs.
- Added all APIs. Now it supports HTML API, Google Search API, Amazon Search API, Amazon Product API, Walmart Search API, Walmart Product API, ChatGPT API, YouTube Search API, YouTube Metadata API, YouTube Trainability API, YouTube Transcript API
- Added a method to call our usage endpoint
- Create a new method for accessing our HTML API. So that in future, we can deprecate the get and post methods.
Added more tests to both client and utils test files
- Updated README.md
- Improved manual-test.js
- Added information to CHANGELOG.md
- Updated version in version.ts
@sahilsunny sahilsunny requested review from daolf and kirby81 December 19, 2025 15:27
@sahilsunny sahilsunny requested review from NBens and kostas-jakeliunas-sb and removed request for daolf January 6, 2026 12:29
Copy link
Collaborator

@kirby81 kirby81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the breaking changes introduced, the implementation looks solid, nice work!

- Used util.deprecate for legacy get and post methods and added a warning that it will be removed on version 2.0.0
- Updated changelog to reflect the correct date
- Removed DELETE, PATCH, HEAD, and OPTIONS from request methods in HtmlApiConfig as they are not supported by the API.
@sahilsunny sahilsunny requested a review from kirby81 January 16, 2026 11:15
Copy link
Collaborator

@kirby81 kirby81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect !

@sahilsunny sahilsunny merged commit 868a8e4 into master Jan 19, 2026
3 checks passed
@sahilsunny sahilsunny deleted the add-other-apis branch January 19, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants