Real-Time Data APIsfor Every Business Need
Access structured, filtered, and real-time data from 120+ domains via powerful APIs. Filter datasets with AI, automate data delivery, and integrate seamlessly into your workflows.
- AI-powered filtering with natural language
- Real-time data delivery via API
- Pay only for the data you need
- Ready-to-use code examples in 7+ languages
Trusted by 20,000+ customers worldwide
Filter massive datasets with a single prompt
Describe exactly what you need in plain English, and let AI apply the perfect filters automatically. Cut costs by retrieving only relevant data.
- Natural language filtering - no coding required
- AI applies accurate filters automatically
- Narrow billions of records to what matters
- Filter by any field: location, industry, job title, date
- Export filtered data in JSON, CSV, or Parquet
Real-Time Data APIs for Every Industry
Access structured, validated data from 120+ domains via powerful filter APIs. Get only the data you need, delivered in real-time to your application.
- 120+ domains
- 190+ dataset APIs
- Billions of records
121.3K+
11.3K+
Data Feeds Pricing
- Clean and validated
- Refreshed monthly
- JSON/CSV/Parquet
Flexible delivery to fit your workflow
Receive filtered data via API response, Webhook, S3, Snowflake, Azure, or direct download
curl --request POST
--url https://api.brightdata.com/datasets/filter
--header 'Authorization: Bearer '
--header 'Content-Type: application/json'
--data '{
"dataset_id": "gd_l1viktl72bvl7bjuj0",
"records_limit": 100,
"filter": {
"name": "job_title",
"operator": "=",
"value": "Software Engineer"
}
}'
import requests
url = "https://api.brightdata.com/datasets/filter"
headers = {"Authorization": "Bearer "}
payload = {
"dataset_id": "gd_l1viktl72bvl7bjuj0",
"records_limit": 100,
"filter": {
"name": "job_title",
"operator": "=",
"value": "Software Engineer"
}
}
response = requests.post(url, json=payload, headers=headers)
print(response.json())
const url = 'https://api.brightdata.com/datasets/filter';
const options = {
method: 'POST',
headers: {
'Authorization': 'Bearer ',
'Content-Type': 'application/json'
},
body: JSON.stringify({
dataset_id: 'gd_l1viktl72bvl7bjuj0',
records_limit: 100,
filter: {
name: 'job_title',
operator: '=',
value: 'Software Engineer'
}
})
};
try {
const response = await fetch(url, options);
const data = await response.json();
console.log(data);
} catch (error) {
console.error(error);
}
HttpResponse response = Unirest.post("https://api.brightdata.com/datasets/filter")
.header("Authorization", "Bearer ")
.header("Content-Type", "application/json")
.body("{"dataset_id":"gd_l1viktl72bvl7bjuj0","records_limit":100,"filter":{"name":"job_title","operator":"=","value":"Software Engineer"}}")
.asString();
Filter and retrieve data with simple API calls
Why Data Feed APIs?
AI-Powered Filtering
Build precise filters with simple JSON operators. No complex scraping logic required.
Real-Time API Access
Retrieve filtered data directly to your application via RESTful API with instant response times.
Structured & Clean Data
Pre-validated, deduplicated data with consistent schemas, ready for immediate use in your applications.
Flexible Delivery Options
Download via API, Webhook, S3, Snowflake, or direct download in JSON, CSV, or Parquet formats.
Developer-Friendly
Complete API documentation with code examples in Python, Node.js, cURL, PHP, Go, Java, and Ruby.
Pay Only for What You Use
Filter billions of records down to exactly what you need. No paying for irrelevant data.
Maximize value with strategic cost savings
Pay Per Record
Only pay for the exact records you retrieve. Filter billions down to thousands—pay only for what you use.
Smart Data Updates
Subscribe to "New Records" or "Updated Records" only—avoid re-downloading unchanged data.
Volume Discounts
Scale your data access with significant savings on large volume requests and subscriptions.
API Efficiency
Retrieve pre-filtered data directly—no need to download and process massive files locally.
Discover how Data APIs can work for you
Power your sales pipeline with targeted data
Filter LinkedIn profiles by job title, company, location, and skills. Build precise prospect lists and automate lead scoring.
Stay ahead with real-time competitive insights
Monitor competitor pricing, track product launches, analyze reviews and sentiment across e-commerce and social platforms.
Feed your models with high-quality, structured data
Access millions of text, image, and structured data records to train AI models, power LLMs, and build intelligent applications.
Make data-driven investment decisions
Track company growth metrics, funding rounds, leadership changes, and market signals to identify investment opportunities.
Discover property opportunities with market data
Monitor listings, pricing trends, rental rates, and property features across multiple markets to identify opportunities.
Ready to power your business with data APIs?
Access 190+ real-time data APIs with AI-powered filtering and flexible delivery
Data Feed APIs FAQs
What are Data Feed APIs?
Data Feed APIs are RESTful APIs that allow you to filter datasets and create snapshots with only the data you need. The Filter Dataset API endpoint starts an asynchronous job that filters your selected dataset and creates a snapshot based on your criteria, which you can then download or access programmatically.
How does the filtering process work?
When you call the Filter Dataset API, it creates an asynchronous job that applies your filters and generates a snapshot. The process has a maximum timeout of 5 minutes. Once complete, you receive a snapshot_id that you can use to access your filtered data. You're only charged for the snapshot size based on the record price.
What filter operators can I use?
The API supports a wide range of operators: = (equal), != (not equal), < and > (less/greater than), <= and >= (less/greater or equal), 'in' and 'not_in' (value matching), 'includes' and 'not_includes' (substring/array contains), 'array_includes' and 'not_array_includes' (exact array matching), and 'is_null' / 'is_not_null' for checking empty values. Different operators apply to different field types (text, number, date, array).
Can I combine multiple filters?
Yes. You can combine multiple filters using 'and' or 'or' logical operators in filter groups. The API supports a maximum nesting depth of 3 levels. For arrays of objects, you can use the 'combine_nested_fields' parameter to ensure all filters match within a single object rather than across the entire array.
How do I filter with large lists of values?
The API supports two modes: JSON Mode for standard filtering, and Multipart/Form-Data Mode for uploading CSV or JSON files containing filter values. In Multipart mode, send dataset_id and records_limit as query parameters, then include your filter definition and files in the form-data body. This is ideal for filtering by hundreds or thousands of values.
What credentials do I need to use the API?
You only need two things: your dataset ID (found on the dataset's detail page) and your Bright Data API key. Include your API key as a Bearer token in the Authorization header. No proxy zones, passwords, or additional credentials are required for dataset filtering.
What data formats are supported?
Filtered snapshots are available in JSON, NDJSON, CSV, and Parquet formats. You can specify your preferred format when downloading the snapshot or configuring your delivery method.
How do I retrieve my filtered data?
After the API returns a snapshot_id, you can retrieve your filtered data via direct download, API response, Webhook callbacks, Amazon S3, Google Cloud Storage, Snowflake, Azure, SFTP, or email. Choose the delivery method that best fits your workflow.
What happens if my filter doesn't match any records?
The API will return a 422 error with the message 'Provided filter did not match any records.' Review your filter syntax and values to ensure they match data in the dataset. You can preview dataset fields and sample data in the Control Panel before filtering.
Are there limits on how many filter jobs I can run?
Yes. Each dataset has a maximum limit of 100 parallel jobs. If you exceed this limit, you'll receive a 429 error. Wait for existing jobs to complete before starting new ones, or contact sales for high-volume use cases requiring increased limits.
What if I don't have enough funds?
If your account balance is insufficient to process the filtered snapshot, you'll receive a 402 error indicating the amount needed. Add funds to your account or adjust your filter to reduce the number of records before trying again.
Where can I find code examples?
Complete code examples are available in the API documentation for cURL, Python, JavaScript, and Java. Each example shows how to authenticate, structure your filter request, and handle the response. You can also generate filter syntax directly from the Control Panel by applying filters in the UI and clicking 'Filter API.'
Start accessing real-time data today
Filter, retrieve, and integrate web data via powerful APIs