Web Crawler
Crawl websites and extract public data from the most popular websites. With over 1500 crawlers, collecting web data has never been easier.
1595 scrapers
- Retrieve real-time data via API
- Pay only for successfully delivered results
- Bulk request handling, up to 5K URLs
- Retrieve results in multiple formats
Trusted by 50,000+ customers worldwide
Web Crawler API
Web Crawler API Library
Remove the need to develop and maintain the infrastructure. Simply extract high volume web data, and ensure scalability and reliability using our Web Scraper API.
125K+
11.6K+
Web Crawler Pricing
Only pay for what’s successfully delivered. No hidden fees, no charges for failed deliveries.
We accept these payment methods:
Every plan gives you full access - pay less per record as you scale
Data Collection
- Automated proxy management
- Full browser rendering
- CAPTCHA solving
Performance at Scale
- Unlimited concurrency
- Batch & scheduled collection
- Job management APIs
Data Delivery
- Data validation & discovery
- Data parsing (JSON or CSV)
- Webhook or API delivery
CODE EXAMPLES
Dedicated endpoints for 800+ domains.
Input
JSON
curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '[{"url":"https://www.linkedin.com/in/elad-moshe-05a90413/"},{"url":"https://www.linkedin.com/in/jonathan-myrvik-3baa01109"},{"url":"https://www.linkedin.com/in/aviv-tal-75b81/"},{"url":"https://www.linkedin.com/in/bulentakar/"},{"url":"https://www.linkedin.com/in/nnikolaev/"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l1viktl72bvl7bjuj0&format=json&uncompressed_webhook=true"
Output
JSON
[
{
"db_source": "1788642333624",
"timestamp": "2026-09-05",
"id": "cel***lac***a-2*********",
"name": "Celia L*****a",
"city": "Canada",
"country_code": "CA",
"position": "--",
"about": null
},
{
"db_source": "1788642333624",
"timestamp": "2026-09-05",
"id": "ama***-er***dso*********57",
"name": "Amanda E*******n",
"city": "Surrey, British Columbia, Canada",
"country_code": "CA",
"position": "Sales Specialist at COBS Bread",
"about": null
},
{
"db_source": "1788642333624",
"timestamp": "2026-09-05",
"id": "ron***for***l-0*********",
"name": "Ronja F*****l",
"city": "Greater Malmö Metropolitan Area",
"country_code": "SE",
"position": "Procurement Specialist på Akind",
"about": "Thr***ng ***env********* th*********************ear***************************************"
},
{
"db_source": "1788642333624",
"timestamp": "2026-09-05",
"id": "mar***-li***n-1*********",
"name": "Margot L****n",
"city": "Canada",
"country_code": "CA",
"position": "--",
"about": null
},
{
"db_source": "1788642333624",
"timestamp": "2026-09-05",
"id": "che***bda***-ta*********7a1***",
"name": "Chef A*****a T****i",
"city": "Pesaro, Marche, Italy",
"country_code": "IT",
"position": "Head chef",
"about": "******"
}
]
Input
JSON
curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '[{"url":"https://www.amazon.com/Quencher-FlowState-Stainless-Insulated-Smoothie/dp/B0CRMZHDG8","asin":"B0CRMZHDG8","origin_url":"https://www.amazon.com/Quencher-FlowState-Stainless-Insulated-Smoothie/dp/B0CRMZHDG8","zipcode":"94107","language":""},{"url":"https://www.amazon.com/KitchenAid-Protective-Dishwasher-Stainless-8-72-Inch/dp/B07PZF3QS3","asin":"B07PZF3QS3","zipcode":"","language":""},{"url":"https://www.amazon.com/TruSkin-Naturals-Vitamin-Topical-Hyaluronic/dp/B01M4MCUAF","asin":"","origin_url":"https://www.amazon.com/TruSkin-Naturals-Vitamin-Topical-Hyaluronic/dp/B01M4MCUAF","zipcode":"94124","language":""}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&format=json&uncompressed_webhook=true"
Output
JSON
[
{
"db_source": "1788360619459",
"timestamp": "2026-09-02",
"title": "43065504 – 18\/2 Solid Unshielded Fire Alarm Cable, Riser Rated (FPLR), 500\u0027 Pull Box, Red",
"seller_name": "NATO S******y",
"brand": "Generic",
"description": "The 43065504 is a high-quality 18 AWG 2-conductor solid unshielded fire alarm cable, designed for reliable signal transm...",
"initial_price": 198,
"currency": "USD"
},
{
"db_source": "1788360619459",
"timestamp": "2026-09-02",
"title": "18 Pcs Simmer Pot Potpourri Mix Blends with Natural Dried Botanical",
"seller_name": "Swa***t",
"brand": "Naucoos",
"description": "Features: Multiple quantities available: 6, 12, and 18 pieces per pack. Different quantities meet your different needs. ...",
"initial_price": 55.99,
"currency": "USD"
},
{
"db_source": "1788360619459",
"timestamp": "2026-09-02",
"title": "Vintage Tel Aviv-yafo City Map 8 x 12 Inch Aluminum Wall Decor",
"seller_name": "vwr***ebt***yrn***",
"brand": "RUPNXO",
"description": "This 8 x 12 inch aluminum metal sign centers the Vintage Tel Aviv-yafo City Map artwork or wording in an easy-to-place f...",
"initial_price": 9.98,
"currency": "USD"
},
{
"db_source": "1788360619459",
"timestamp": "2026-09-02",
"title": "TV Revue 5\/1954-Martha Raye-Perry Como-Marie Wilson-Lucy-Red Skelton-Bishop Shee",
"seller_name": "DTA***lle***ble***",
"brand": null,
"description": "-Martha Raye-Perry Como-Marie Wilson cover -Lucy-Red Skelton-Bishop Sheen-\u0026 more -Star photos-program info-fashions-vint...",
"initial_price": 98,
"currency": "USD"
},
{
"db_source": "1788360619459",
"timestamp": "2026-09-02",
"title": "6 Pack Microfiber Mop Heads Replacement, 14oz Commercial Loop End Mop Head",
"seller_name": "Millsten O******l S***e",
"brand": "Millsten",
"description": "About this item High-Quality Microfiber Mop Heads: Say goodbye to inefficient cotton mops and experience faster, more ef...",
"initial_price": 53.99,
"currency": "USD"
}
]
Input
JSON
curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '[{"url":"https://www.zillow.com/homedetails/2506-Gordon-Cir-South-Bend-IN-46635/77050198_zpid/?t=for_sale"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_lfqkr8wm13ixtbd8f5&format=json&uncompressed_webhook=true"
Output
JSON
[
{
"db_source": "1788622898694",
"timestamp": "2026-09-05",
"zpid": 79690090,
"city": "Eagle",
"state": "ID",
"homeStatus": "OTHER",
"address:city": "Eagle",
"address:streetAddress": "1546 W Stafford Dr"
},
{
"db_source": "1788622898694",
"timestamp": "2026-09-05",
"zpid": 28627012,
"city": "Alvarado",
"state": "TX",
"homeStatus": "OTHER",
"address:city": "Alvarado",
"address:streetAddress": "805 N Spears St"
},
{
"db_source": "1788622898694",
"timestamp": "2026-09-05",
"zpid": 247131272,
"city": "Orlando",
"state": "FL",
"homeStatus": "OTHER",
"address:city": "Orlando",
"address:streetAddress": "17687 Sailfin Dr"
},
{
"db_source": "1788622898694",
"timestamp": "2026-09-05",
"zpid": 47989020,
"city": "Deltona",
"state": "FL",
"homeStatus": "OTHER",
"address:city": "Deltona",
"address:streetAddress": "2006 Jefferson Ave"
},
{
"db_source": "1788622898694",
"timestamp": "2026-09-05",
"zpid": 85184528,
"city": "Butler",
"state": "IN",
"homeStatus": "RECENTLY_SOLD",
"address:city": "Butler",
"address:streetAddress": "4188 County Road 71"
}
]
Input
JSON
curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '[{"url":"https://www.instagram.com/p/Cuf4s0MNqNr","country":""},{"url":"https://www.instagram.com/p/DP861NijuwE","country":""}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_lk5ns7kz21pck8jpis&format=json&uncompressed_webhook=true"
Output
JSON
[
{
"db_source": "1788606841266",
"timestamp": "2026-09-05",
"url": "https:\/\/www.instagram.com\/p\/DbGJpXgzRWC",
"user_posted": "samsungmobile",
"description": "Meet the #GalaxyZFold8Ultra and the all-new #GalaxyZFold8—the ultimate powerhouse and the world\u0027s lightest fold. Two dis...",
"hashtags": [
"#GalaxyZFold8Ultra",
"#GalaxyZFold8—the",
"#GalaxyAI"
],
"num_comments": 437,
"date_posted": "2026-07-22T13:02:07.000Z"
},
{
"db_source": "1788606841266",
"timestamp": "2026-09-05",
"url": "https:\/\/www.instagram.com\/p\/DcyuG05ES5A\/",
"user_posted": "socalthrills",
"description": "New Zealand’s North Island isn’t just one kind of experience. 🇳🇿\n\nIt’s Māori culture, geothermal landscapes, incredibl...",
"hashtags": [
"#NZMustDo",
"#NewZealand",
"#NorthIslandNZ",
"#TravelAdvisor"
],
"num_comments": 0,
"date_posted": "2026-09-02T16:58:41.000Z"
},
{
"db_source": "1788606841266",
"timestamp": "2026-09-05",
"url": "https:\/\/www.instagram.com\/p\/DbToLqrs0zd\/",
"user_posted": "wari__art",
"description": "„Energie“ ist bei @artundweise_veganicecream eingezogen - Ausstellung ist noch bis Ende August in der Monheimer Altstadt...",
"hashtags": [
"#kunst",
"#abstrakt",
"#artundweise",
"#ausstellung",
"#acryl"
],
"num_comments": 0,
"date_posted": "2026-07-27T18:45:13.000Z"
},
{
"db_source": "1788606841266",
"timestamp": "2026-09-05",
"url": "https:\/\/www.instagram.com\/p\/Dc06usDEyLI\/",
"user_posted": "drjockers",
"description": "👇 Comment with \u0022Guide\u0022 for My 30 Cancer Fighting Foods Guide!\n\nCancer risk isn’t influenced by just one factor. Our eve...",
"hashtags": [
"#Cancer",
"#WomensHealth",
"#PreventiveHealth",
"#HolisticHealth",
"#HealthyLiving",
"#Wellness",
"#Health",
"#Nutrition"
],
"num_comments": 40,
"date_posted": "2026-09-03T13:27:26.000Z"
},
{
"db_source": "1788606841266",
"timestamp": "2026-09-05",
"url": "https:\/\/www.instagram.com\/p\/DRtrQjyk3Gn\/",
"user_posted": "matisseella",
"description": "hello summer in @sundaysthelabel",
"hashtags": null,
"num_comments": 41,
"date_posted": "2025-12-01T08:12:13.000Z"
}
]
BEST-IN-CLASS DX
Easy to start. Easier to scale.
Unmatched Stability
Ensure consistent performance and minimize failures by relying on the world’s leading proxy infrastructure.
Simplified Web Scraping
Put your scraping on auto-pilot using production-ready APIs, saving resources and reducing maintenance.
Unlimited Scalability
Effortlessly scale your scraping projects to meet data demands, maintaining optimal performance.
UNDER THE HOOD
Never worry about proxies and CAPTCHAs again
- Automatic IP Rotation
- CAPTCHA Solver
- User Agent Rotation
- Custom Headers
- JavaScript Rendering
- Residential Proxies
API for Seamless Web Crawler Data Access
Comprehensive, Scalable, and Compliant Web Data Extraction
API for Seamless Web Crawler Data Access
Comprehensive, Scalable, and Compliant Web Data Extraction
Tailored to your workflow
Get structured data in JSON, NDJSON, or CSV files through Webhook or API delivery.
Built-in infrastructure and unblocking
Get maximum control and flexibility without maintaining proxy and unblocking infrastructure. Easily scrape data from any geo-location while avoiding CAPTCHAs and blocks.
Battle-proven infrastructure
Bright Data’s platform powers 50,000+ companies worldwide, offering peace of mind with 99.99% uptime, access to 400M+ real user IPs covering 195 countries.
Industry leading compliance
Our privacy practices comply with data protection laws, including the EU data protection regulatory framework, GDPR, and CCPA.
Bright Data's products are used by the world’s top brands
We help businesses grow with secure, scalable, flexible data management.
I recommend using Bright Data’s products for any company, especially in the finance industry. Bright Data is trustworthy and compliant, the service is great, the products are flawless and their network is fast and stable.
Xiaolong ShiCrawler Engineer at Bitget- Bright Data has their own proxy infrastructure which helps keep your web data flowing plus, their web unlocker helps beat any pesky CAPTCHAs that might be holding you back.Watch now
Nicholas RenotteData Science Specialist 
From my experience, Bright Data’s service has been invaluable. Bright Data helped us collect enough public web data to meet our needs, and with its support and development staff, we optimized many of our processes.
Charmagne CruzHead of Reporting & Analytics, Business Technologies and Pricing at Shopee Philippines Inc.- Having the best quality and quantity of data is the most important thing, and that’s where the combination of Bright Data and tgndata works.Watch now
George KoutsoudopoulosCEO at tgndata - We are really impressed with the reliability, and very happy with Bright Data overall. We have a regular communication channel with our account manager, who is very helpful.
Yorgos PanzarisCTO at Convert Group - We are very pleased with the partnership with Bright Data. Everything’s been good, the network has been very stable, we’re happy with the customer service and the support staff is bar none in our book.
Cheddi RaiCEO at AdRetreaver
