Crawl API
Automate content extraction from any domain. Simply define the root URL and retrieve the full website content as Markdown, Text, HTML, or JSON files.
No credit card required

- Supports dynamic data collection
- Retrieve data as Markdown, Text, HTML, or JSON files
- Customize crawling with advanced parameters
- Also available in no-code mode
TRUSTED BY 20,000+ CUSTOMERS WORLDWIDE
const options = {
method: 'POST',
headers: {Authorization: 'Bearer ', 'Content-Type': 'application/json'},
body: '[{"url":"https://il.linkedin.com/company/bright-data"}]'
};
fetch('https://api.brightdata.com/datasets/v3/trigger', options)
.then(response => response.json())
.then(response => console.log(response))
.catch(err => console.error(err));
import requests
url = "https://api.brightdata.com/datasets/v3/trigger"
payload = [{"url": "https://il.linkedin.com/company/bright-data"}]
headers = {
"Authorization": "Bearer ",
"Content-Type": "application/json"
}
response = requests.request("POST", url, json=payload, headers=headers)
print(response.text)
Easy to start, easier to scale
-
Choose target domainDefine target URL and connect to the API with a single line of code
-
Send requestEdit crawl parameters and insert your custom logic using Python or JavaScript
-
Get your dataRetrieve website data as Markdown, Text, HTML, or JSON files
Developer-first experience
Quick Start
Connect to the Crawl API with a single line of code, or use the control panel to get results directly through the Control Panel.
Custom Collection
Use request parameters to customize collection and delivery, including pagination, scheduling and log collection.
Data Parsing
Efficiently converts raw HTML into structured data files, delivered as Markdown, Text, HTML, or JSON, directly to your database.
Crawl API pricing
Leading the way in ethical web data collection
Bright Data sets the gold standard in compliance, effectively self-regulating the industry. With transparent operations validated by top security firms, clear peer consent, and pioneering compliance units, we ensure legitimate and safe data collection. Upholding international privacy laws and utilizing tools like BrightBot, we minimize your legal exposure, making partnership with us a strategic move to curtail legal risks and associated costs.