--- title: "Skyvern Automation with Bright Data API Integration" slug: skyvern-with-bright-data-apis date: 2025-09-28T12:17:19+00:00 modified: 2026-07-23T08:18:27+00:00 permalink: https://brightdata.com/blog/ai/skyvern-with-bright-data-apis type: blog --- [ Blog ](https://brightdata.com/blog "Blog") / [AI](https://brightdata.com/blog/ai) [AI](https://brightdata.com/blog/ai) # Skyvern Automation with Bright Data API Integration Learn how to integrate Bright Data’s Scraper API with Skyvern for seamless browser automation, eCommerce assistance, and real-time web data extraction. 10 min read [ ](https://brightdata.com/blog/authors/arindam-majumder) [Arindam Majumder Technical Writer ](https://brightdata.com/blog/authors/arindam-majumder) ![Skyvern Automation with Bright Data API Integration blog image](https://media.brightdata.com/2025/09/Skyvern-Automation-with-Bright-Data-API-Integration-blog-image.png) By the end of this article, you’ll understand how to: - Use Bright Data Google AI Mode Scraper API service - Leverage Skyvern for task automation - Use Bright Data API service with Skyvern to automate web tasks. - Combine automation and data feeds to build an eCommerce assistant. - Retrieve cart product details automatically Let’s dive in! ## Leveraging Bright Data’s API Service The foundation of browser automation is the ability to bypass challenges such as CAPTCHA, IP bans, and dynamic web loading. This is where Bright Data becomes essential. With [Bright Data’s Web Scraper](/products/web-scraper), which supports more than 120 web domains, browser automation is more efficient and reliable. It manages [common scraping challenges](/blog/web-data/web-scraping-challenges) such as IP bans, CAPTCHA, cookies, and other forms of bot detection. To get started, [sign up for free](/) and obtain your `API key` and `dataset_id` for the domain you want to scrape. Once you have these, you are ready to begin. Below are the steps to retrieve fresh data from any domain, like BBC News: 1. Create a [Bright Data account](/) if you have not already done so. A free tier is available. 2. Go to the Web Scrapers page. Under Web Scrapers Library, explore the available scraper templates. 3. Search for your target domain, such as BBC News, and select it. 4. From the list of [BBC News scrapers](/products/web-scraper/bbc), select BBC News — collect by URL. This scraper allows you to retrieve data without logging in to the domain. 5. Choose the Scraper API option. The No-Code Scraper helps to retrieve datasets without code. 6. Click API Request Builder, then copy your `API key`, `BBC Dataset URL`, and `dataset_id`. 7. The `API-key` and `dataset_id` are required to enable automation capabilities in your workflow. They allow you to access Bright Data capabilities directly while programming. ## What is Skyvern [Skyvern](https://www.skyvern.com/) is an AI [browser automation tool](/blog/web-data/best-browser-automation-tools) that uses artificial intelligence to automate tasks within web browsers. It combines machine learning, natural language processing, and computer vision to handle complex browser actions. Skyvern differs from traditional automation tools such as Selenium and Playwright in the following ways: - Adaptability to UI changes: Self-healing abilities allow Skyvern to dynamically adapt to UI changes without breaking scripts. - Workflow complexity: Capable of handling multi-step workflows with AI reasoning through a single prompt. - Visual recognition: Uses computer vision to understand and interact with UI elements visually. With these capabilities, you can use Skyvern to log into booking sites, complete forms, or add items to shopping carts. When integrated with Bright Data’s web scraping capabilities, Skyvern can provide a powerful framework for addressing diverse web automation needs. ## Automation Workflow For example, if you want to purchase a vehicle part from an online store, you may want to compare available options and automatically add one to your cart. The workflow would look like this: - Bright Data AI Mode [Scraper API](https://docs.brightdata.com/api-reference/web-scraper-ide-api/Getting_started_wtih_the_API) fetches the description and product details, such as the part number, from your specified manufacturer. - You review the output and make your selection. Bright Data provides fast and reliable web data retrieval. - Skyvern uses the retrieved details from Bright Data to access finditparts.com. It then navigates the site, adds the selected product(s) to the cart, and outputs the cart details and cart URL. - Proceed directly to checkout and payment. ## Prerequisites - Basic knowledge of Python programming. Download Python [here](https://www.python.org/downloads/) - An active Bright Data account. Sign up [here](/) and retrieve your API key from the welcome email - Basic knowledge of JSON and REST APIs ## Setting Up the Project ### Step 1: Set Up Bright Data Retrieve your Bright Data `API key`, `dataset ID`, and Google AI Mode URL by following the same steps described in Leveraging Bright Data’s Robust API Service for your use case. ### Step 2: Sign Up on Skyvern Cloud 1. Go to https://app.skyvern.com/ and sign up to receive 5 USD free credits. 2. Ask the Skyvern agent to run a task to see it in action. For example: Navigate to the Hacker News homepage and retrieve the top three posts. 3. Check the history to track the progress of the task. A status of **Completed** indicates the task finished successfully. 4. Once completed, click the task under history to view the output, parameters, and additional details about the task. Now that Skyvern is set up, you can begin writing your code script. ### Step 3: Install Skyvern on Your Machine **3.1 Create a Virtual Environment** In your desired project folder, create a virtual environment with Python: `python -m venv .venv` Activate the environment. `.venv\Scripts\activate` **3.2 Install Skyvern on any device with** `pip install skyvern` If you encounter installation issues, you can use the Ubuntu terminal on Windows. Check this [post](https://www.reddit.com/r/ClaudeAI/comments/1lantwy/complete_wsl_claude_code_cursor_setup_guide_for/) to learn how to set up the Ubuntu terminal. After the terminal is running, navigate to your desired directory and run: ```none pip install uv ``` Create a virtual environment with: ```none uv venv venv ``` Then install Skyvern with: ```none uv pip install skyvern ``` **3.3 Quickstart Skyvern** When the installation is complete, run: ```none skyvern quickstart ``` - When prompted with ‘Would you like to run Skyvern locally or in the cloud?, ‘type – ‘cloud’. - When prompted with ‘Enter Skyvern baseURL’, press Enter. - Type ‘n’ for every installation prompt except the MCP prompt, where you should type ‘y’. After setup, run: ```none skyvern init ``` Create a Python script called app.py. ### Step 4: Retrieve Product Details with Bright Data 4.1 Retrieve the part number with Bright Data using this code in app.py: ```none import asyncio import requests import time import json def trigger_scraping_job(api_key, data): """ Trigger a Bright Data dataset job with a list of dicts containing url, prompt, country. Returns the snapshot_id if successful. """ endpoint = "https://api.brightdata.com/datasets/v3/trigger" params = { "dataset_id": "gd_mcswdt6z2elth3zqr2", # Your dataset ID "include_errors": "true", } headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json", } response = requests.post(endpoint, headers=headers, params=params, json=data) if response.status_code == 200: snapshot_id = response.json().get("snapshot_id") print(f"Request successful! Snapshot ID: {snapshot_id}") return snapshot_id else: print(f"Request failed! Status: {response.status_code}") print(response.text) return None def poll_and_retrieve_snapshot(api_key, snapshot_id, output_file, polling_timeout=20): """ Poll the Bright Data snapshot endpoint until data is ready. Save the JSON response to an output file. """ snapshot_url = f"https://api.brightdata.com/datasets/v3/snapshot/{snapshot_id}?format=json" headers = { "Authorization": f"Bearer {api_key}" } print(f"Polling snapshot for ID: {snapshot_id}...") while True: response = requests.get(snapshot_url, headers=headers) if response.status_code == 200: print("Snapshot is ready. Downloading...") snapshot_data = response.json() with open(output_file, "w", encoding="utf-8") as file: json.dump(snapshot_data, file, indent=4) print(f"Snapshot saved to {output_file}") return elif response.status_code == 202: print(f"Snapshot is not ready yet. Retrying in {polling_timeout} seconds...") time.sleep(polling_timeout) else: print(f"Request failed! Status: {response.status_code}") print(response.text) break if __name__ == "__main__": BRIGHT_DATA_API_KEY = "YOUR_BRIGHT_DATA_API_KEY" # Your API key # Match the curl JSON data structure exactly data = [ { "url": "https://google.com/aimode", "prompt": "find the part number of a wheel seal from finditparts.com whose manufacturer is SKF", "country": "" } ] snapshot_id = trigger_scraping_job(BRIGHT_DATA_API_KEY, data) if snapshot_id: poll_and_retrieve_snapshot(BRIGHT_DATA_API_KEY, snapshot_id, "product.json") ``` The prompt is: ‘find the part number of a wheel seal from finditparts.com whose manufacturer is SKF.’ This will create a `product.json` file containing product descriptions and part numbers from the SKF manufacturer. ```none { "url": "https://www.finditparts.com/products/16775486/skf-45093xt?srcid=CHL01SCL010-Npla-Dmdt-Gusa-Svbr-Mmuu-K16775486-L22", "title": "www.finditparts.com", "description": "SKF 45093XT Wheel Seal | FinditParts", "icon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.finditparts.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", "domain": "https://www.finditparts.com", "cited": true }, { "url": "https://www.finditparts.com/products/193780/cr-slash-skf-14115?srcid=CHL01SCL010-Npla-Dmdt-Gusa-Svbr-Mmuu-K193780-L1464", "title": "www.finditparts.com", "description": "SKF 14115 Wheel Seal | FinditParts", "icon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.finditparts.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", "domain": "https://www.finditparts.com", "cited": true }, { ``` Next, choose your preferred part number (located in the description) and rerun the Bright Data code with this prompt: ‘Find the product ID for the SKF wheel seal with part number 47691’ ```none # Match the curl JSON data structure exactly data = [ { "url": "https://google.com/aimode", "prompt": "Find the product ID for the SKF wheel seal with part number 47691", "country": "" } ] ``` Skyvern requires the product ID to add details to the cart on `finditparts.com` (a vehicle parts e-commerce website). This process will generate a `product.json` file with the desired product ID. ### Step 5: Prompt Skyvern for Its Tasks First, navigate to . This URL is a shortcut for creating tasks on Skyvern. Click **Advanced Settings** under the **Base Content** section, and update the product ID and prompt for your use case. The prompt is: ‘Search for the specified product ID, add it to the cart, and then navigate to the cart page. Your goal is COMPLETE when you’re on the cart page and the specified product is in the cart.’ The **Extraction** section below **Advanced Settings** is also important. Modify the **Data Extraction Goal** to: ‘Extract the cart page URL and all product quantity information from the cart page.’ Click **Copy API Command** at the bottom of the page, paste it into your terminal, and press **Enter**. This will create a `task_id` in your terminal and an instance of the task on your Skyvern Cloud. You can check its status in **History** to see if it is queued, running, or completed. A status of **Completed** means the task has finished. You can now view the cart details and product URL returned by Skyvern. Congratulations. Your workflow is complete. Click the URL to proceed with payment. [Bright Data](/) eliminates the need to manually search through products online by bringing options directly to your machine. This allows you to select the best product and automate the purchasing process with Skyvern. ## Next Steps You can expand the workflow to include adding multiple products to the cart for checkout and generating a Natural Language Processing (NLP) summary of the total products. You can also deploy the workflow to the cloud for continuous monitoring. Finally, you can integrate it with Google Calendar to track discounts. ## Conclusion In this tutorial, you’ve learned how to combine [Bright Data’s Scraper API](/products/web-scraper) with Skyvern to automate the process of finding and purchasing products online. Beyond the Scraper API, Bright Data offers other tools that can power your AI agents, such as [ready-to-use datasets](/products/datasets) tailored for [e-commerce](/products/datasets/ecommerce), [social media](/products/datasets/social-media), and more, as well as the [Bright Data MCP server](/ai/mcp-server) for advanced multi-step automation and access to 40+ specialized tools. Together, these products make it easy to build AI-driven workflows that can gather, analyze, and act on web data efficiently. Start exploring [Bright Data’s full suite](/) to enhance your AI automation projects today. Contact usStart free trial No credit card required [ ](https://www.linkedin.com/in/arindam2004/) Arindam Majumder Technical Writer Arindam Majumder is a developer advocate, YouTuber, and technical writer who simplifies LLMs, agent workflows, and AI content for 5,000+ followers. Expertise RAG AI Agents Python [ View all articles ](https://brightdata.com/blog/authors/arindam-majumder) Table of Contents Data for AI Supercharge your AI with instant and reliable access to web data. No blockers. No hassle. Talk to an expert Bright Data MCP Get started with Bright Data’s Web MCP Server today with 5000 free monthly requests and unlock your AI’s full potential. Start free now [ ](https://news.ycombinator.com/submitlink?t=Skyvern+Automation+with+Bright+Data+API+Integration&u=https://brightdata.com/blog/ai/skyvern-with-bright-data-apis) [ ](https://www.linkedin.com/shareArticle?mini=true&title=Skyvern+Automation+with+Bright+Data+API+Integration&url=https://brightdata.com/blog/ai/skyvern-with-bright-data-apis) [ ](http://www.reddit.com/submit?title=Skyvern+Automation+with+Bright+Data+API+Integration&url=https://brightdata.com/blog/ai/skyvern-with-bright-data-apis) ## You might also be interested in [ ](https://brightdata.com/blog/ai/openhuman-with-bright-data "Production-Ready Web Access in OpenHuman Through the Bright Data CLI") [AI Antonello Zanini Technical Writer ### Production-Ready Web Access in OpenHuman Through the Bright Data CLI Integrate Bright Data CLI with OpenHuman to enable production-ready web access and data collection for AI agents. 09-Sep-2026 12 min read ](https://brightdata.com/blog/ai/openhuman-with-bright-data) [ ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data "Giving self-hosted MiniMax M3 agents live web access with Bright Data") [AI Satyam Tripathi Technical Writer ### Giving self-hosted MiniMax M3 agents live web access with Bright Data Self-hosted MiniMax M3 agents get live web access using Bright Data’s search and scraping tools. Bypass blocks and CAPTCHAs. 09-Sep-2026 54 min read ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data) [ ](https://brightdata.com/blog/web-data/multimodal-web-scraping-with-minimax "Multimodal Web Scraping with MiniMax") [Web Data Antonello Zanini Technical Writer ### Multimodal Web Scraping with MiniMax Pair Bright Data Web Unlocker with MiniMax M3 vision to extract structured data from images and web page screenshots. 09-Sep-2026 4 min read ](https://brightdata.com/blog/web-data/multimodal-web-scraping-with-minimax)