Stop relying on overpriced APIs for popular domains! 🤚🏻
This repository contains scraping samples for the most scraped domains on the web, with full-length tutorials and functioning scrapers mainly built with Python and NodeJS (NodeJS files included in separate folders).
Scrapers in this repository are programmed to help you understand the basics of scraping these popular websites and serving as a base for you to customize and use in your own projects.
For Python, you'll be using:
- Python 3.8+,
- Requests for handling HTTP requests,
- Beautiful Soup for parsing information,
- and urllib.parse for URL encoding.
This repository also uses Scrape.do's web scraping API, to bypass Web Application Firewalls (WAFs) of the domains you're scraping and also interacting with the browser.
| Domain | Status | Scraped Data | Tutorial | Last Updated |
|---|---|---|---|---|
| Amazon (External) | 🟢 Working | • Product data from single pages • Product variants with different sizes • Search results and ads |
Scrape Product Data from Amazon | Dec 18, 2025 |
| eBay | 🟢 Working | • Basic product listings • Product reviews with pagination • Search results with multiple layouts • Product variants and hidden data |
Scrape Products, Reviews, and Variants from eBay | Aug 1, 2025 |
| Etsy | 🟢 Working | • Category and search result listings • Single product details via JSON-LD • Product reviews via internal API • Pricing, ratings, and seller information |
Scrape Products and Reviews from Etsy | Oct 30, 2025 |
| Google AI Overview | 🟢 Working | • AI Overview text blocks and references • Deferred state handling via SERP API • Playwright browser extraction |
Scrape AI Overview Summaries from Google | Mar 19, 2026 |
| Google Maps | 🟢 Working | • Business listings via protobuf API • Place details, hours, and contact info • Customer reviews with sorting and pagination |
Scrape Business Data and Reviews from Google Maps | Mar 19, 2026 |
| Google News | 🟢 Working | • News tab headlines and metadata • Full article content as markdown • Top stories via SERP API |
Scrape Headlines and Articles from Google News | Mar 19, 2026 |
| Google Scholar | 🟢 Working | • Search results with paper metadata • Article download as markdown • Author profiles and citation stats |
Scrape Papers and Author Profiles from Google Scholar | Mar 19, 2026 |
| Google Search | 🟢 Working | • First page organic results • All paginated organic results • Paid search ads and FAQs • Related search terms • SERP API structured search |
Scrape Organic Results, Ads, and FAQs from Google | Mar 19, 2026 |
| Google Shopping | 🟢 Working | • Search results with async pagination • Product details via hidden OAPV endpoint • Seller offers, reviews, and forums • Full pipeline with JSON export • AI Mode shopping recommendations |
Scrape Products, Sellers, and Reviews from Google Shopping | Mar 19, 2026 |
| Google Trends | 🟢 Working | • Interest over time and by region • Related queries (top and rising) • Trending Now topics with news articles |
Scrape Trends, Regional Data, and Related Queries from Google | Mar 19, 2026 |
| Bing | 🟢 Working | • Organic web search results with pagination • Image search results with download option • Shopping/product listings |
Scrape Search Results, Images, and Products from Bing | Oct 21, 2025 |
| G2 | 🟢 Working | • Company and product data • Product reviews • Search results and categories |
Scrape Company Data and Reviews from G2 | Aug 5, 2025 |
| DoorDash | 🟢 Working | • Consumer address registration • Restaurant and store listings • Category-specific products • Full menu catalogs |
Scrape Restaurants, Products, and Menus from DoorDash | Jul 17, 2025 |
| Best Buy | 🟢 Working | • Product listings with Selenium • Product data with cloud rendering • Brand, price, rating, SKU, and URL extraction |
Scrape Products from Best Buy | Feb 18, 2026 |
| Uber Eats | 🟢 Working | • Store listings via API • Store listings via frontend • Restaurant menus • Chain store products |
Scrape Stores, Menus, and Products from Uber Eats | Aug 1, 2025 |
| Zillow | 🟢 Working | • Property listing details • Pricing and market data |
Scrape Property Data from Zillow | Aug 5, 2025 |
| Naver | 🟢 Working | • E-commerce product pages • Product data via API • Paid search advertisements • Organic search results • Image search results |
Scrape Products and Search Results from Naver | Sep 1, 2025 |
| Chewy | 🟢 Working | • Pet product information • Names and pricing |
Scrape Pet Products from Chewy | Aug 5, 2025 |
| Mouser | 🟢 Working | • Electronic component data • Product names and pricing |
Scrape Electronic Components from Mouser | Aug 5, 2025 |
| Fnac | 🟢 Working | • French retail product data • Names and pricing |
Scrape Products from Fnac | Aug 5, 2025 |
| Ubersuggest | 🟢 Working | • Bearer token generation • Keyword overview data • SERP results • Automated keyword analysis |
Scrape Keyword Data from Ubersuggest | Jun 23, 2025 |
| Zomato | 🟢 Working | • Restaurant information • Delivery menu data |
Scrape Restaurant Data from Zomato | Jul 2, 2025 |
| Allegro | 🟢 Working | • Polish e-commerce products • Names, pricing, and ratings |
Scrape Products from Allegro | Aug 5, 2025 |
| AUTODOC | 🟢 Working | • Automotive parts data • Product details and pricing |
Scrape Auto Parts from AUTODOC | Aug 5, 2025 |
| AutoScout24 | 🟢 Working | • Car listing information • Vehicle names and pricing |
Scrape Car Listings from AutoScout24 | Aug 5, 2025 |
| Idealista | 🟢 Working | • Spanish real estate listings • Property details, pricing, and location data • Search results with pagination |
Scrape Property Data from Idealista | Aug 6, 2025 |
| FastPeopleSearch | 🟢 Working | • Person information lookup • Names, ages, addresses, and ZIP codes • Phone numbers and email addresses • Aliases and relatives |
Scrape People Data from FastPeopleSearch | Feb 24, 2026 |
| HungerStation | 🟢 Working | • Restaurant listings • Store menu items |
Scrape Restaurants and Menus from HungerStation | Jul 17, 2025 |
| Imovelweb | 🟢 Working | • Brazilian real estate data • Property details and pricing |
Scrape Properties from Imovelweb | Aug 5, 2025 |
| Klium | 🟢 Working | • Tool and equipment data • Product names, pricing, and stock |
Scrape Tools from Klium | Aug 5, 2025 |
| MSC Direct | 🟢 Working | • Industrial supply products • Brand, names, and pricing |
Scrape Industrial Supplies from MSC Direct | Aug 5, 2025 |
| TicketMaster | 🟢 Working | • Event details and schedules • Dates, venues, and locations |
Scrape Events from TicketMaster | Aug 5, 2025 |
| TruePeopleSearch | 🟢 Working | • Person information lookup • Names, ages, addresses, and ZIP codes • Phone numbers and email addresses • Aliases and relatives |
Scrape People Data from TruePeopleSearch | Feb 24, 2026 |
| Walmart | 🟢 Working | • Individual product details with store-specific pricing • Product variations (sizes, colors, packs) • Category pages with pagination • Price and stock change tracking |
Scrape Complete Grocery Data from Walmart | Sep 2, 2025 |
| Zoro | 🟢 Working | • Industrial business supplies • Product names and manufacturer data |
Scrape Business Supplies from Zoro | Aug 5, 2025 |
| Cineworld | 🟢 Working | • Movie screenings and showtimes • Ticket prices for all screening types |
Scrape Screenings and Ticket Prices from Cineworld | Aug 27, 2025 |
| Regmovies | 🟢 Working | • Regal Cinemas movie screenings • Ticket prices and availability |
Scrape Screenings and Ticket Prices from Regmovies | Aug 27, 2025 |
| Redfin | 🟢 Working | • Property details • Local listings and details |
Scrape Property Details and Search Results from Redfin | Oct 03, 2025 |
| PeopleSearchNow | 🟢 Working | • Person information lookup • Names, ages, and addresses |
- | Feb 19, 2026 |
| SearchPeopleFree | 🟢 Working | • Person information lookup • Names, ages, addresses, and ZIP codes • Phone numbers and email addresses • Spouse and family members |
Scrape Person Data from SearchPeopleFree | Feb 24, 2026 |
| WhitePages | 🟢 Working | • Person information lookup • Names, ages, and addresses • Phone numbers and ZIP codes |
Scrape Person Data from WhitePages | Feb 24, 2026 |
| 🟢 Working | • Subreddit posts with pagination • Search results with cursor-based pagination • Comments matching search queries |
Scrape Posts, Search Results, and Comments from Reddit | Mar 11, 2026 | |
| Rightmove | 🟢 Working | • Paginated property listing results • Address, price, type, bedrooms, agent, and phone • Full property details via embedded PAGE_MODEL JSON |
Scrape Property Listings and Details from Rightmove | Apr 6, 2026 |
| Wayfair | 🟢 Working | • Single product details with images and reviews • Category pages with pagination and deduplication • Full variation map (color × size) with per-variation pricing and stock |
Scrape Products, Categories, and Variations from Wayfair | Apr 6, 2026 |
| Alibaba | 🟢 Working | • Paginated category listings via embedded JSON • Full product details with price tiers, MOQ, and attributes • Paginated reviews via signed MTOP API requests |
How to Scrape Alibaba Product Listings, Prices, and Reviews | Apr 8, 2026 |
| DigiKey | 🟢 Working | • Paginated category listings via LZ-String cursor pagination • Full product details with pricing tiers and dynamic attributes |
How to Scrape DigiKey: Product Data, Pricing, and Specifications | Apr 8, 2026 |