Real-Time Data APIs
for Every Business Need

Data Feed APIs are RESTful APIs that allow you to filter datasets and create snapshots with only the data you need. The Filter Dataset API endpoint starts an asynchronous job that filters your selected dataset and creates a snapshot based on your criteria, which you can then download or access programmatically.

When you call the Filter Dataset API, it creates an asynchronous job that applies your filters and generates a snapshot. The process has a maximum timeout of 5 minutes. Once complete, you receive a snapshot_id that you can use to access your filtered data. You're only charged for the snapshot size based on the record price.

The API supports a wide range of operators: = (equal), != (not equal), < and > (less/greater than), <= and >= (less/greater or equal), 'in' and 'not_in' (value matching), 'includes' and 'not_includes' (substring/array contains), 'array_includes' and 'not_array_includes' (exact array matching), and 'is_null' / 'is_not_null' for checking empty values. Different operators apply to different field types (text, number, date, array).

Yes. You can combine multiple filters using 'and' or 'or' logical operators in filter groups. The API supports a maximum nesting depth of 3 levels. For arrays of objects, you can use the 'combine_nested_fields' parameter to ensure all filters match within a single object rather than across the entire array.

The API supports two modes: JSON Mode for standard filtering, and Multipart/Form-Data Mode for uploading CSV or JSON files containing filter values. In Multipart mode, send dataset_id and records_limit as query parameters, then include your filter definition and files in the form-data body. This is ideal for filtering by hundreds or thousands of values.

You only need two things: your dataset ID (found on the dataset's detail page) and your Bright Data API key. Include your API key as a Bearer token in the Authorization header. No proxy zones, passwords, or additional credentials are required for dataset filtering.

Filtered snapshots are available in JSON, NDJSON, CSV, and Parquet formats. You can specify your preferred format when downloading the snapshot or configuring your delivery method.

After the API returns a snapshot_id, you can retrieve your filtered data via direct download, API response, Webhook callbacks, Amazon S3, Google Cloud Storage, Snowflake, Azure, SFTP, or email. Choose the delivery method that best fits your workflow.

The API will return a 422 error with the message 'Provided filter did not match any records.' Review your filter syntax and values to ensure they match data in the dataset. You can preview dataset fields and sample data in the Control Panel before filtering.

Yes. Each dataset has a maximum limit of 100 parallel jobs. If you exceed this limit, you'll receive a 429 error. Wait for existing jobs to complete before starting new ones, or contact sales for high-volume use cases requiring increased limits.

If your account balance is insufficient to process the filtered snapshot, you'll receive a 402 error indicating the amount needed. Add funds to your account or adjust your filter to reduce the number of records before trying again.

Complete code examples are available in the API documentation for cURL, Python, JavaScript, and Java. Each example shows how to authenticate, structure your filter request, and handle the response. You can also generate filter syntax directly from the Control Panel by applying filters in the UI and clicking 'Filter API.'