Blog / AI
AI

Integrate Qwen-Agent with MCP to Build Agents with Live Data Access

Connect Qwen-Agent with the Bright Data MCP server to power AI agents with real-time web data retrieval and automation.
16 min read
Qwen-Agent with Bright Data's MCP

In this guide, you will learn:

  • What the Qwen-Agent library is and how it helps you build AI agents.
  • Why using Qwen3 with the Bright Data MCP server is ideal for creating next-gen AI agents.
  • How to integrate Qwen-Agent with Bright Data’s MCP to build a competent agent powered by Qwen3.

Let’s dive in!

What Is Qwen-Agent?

Qwen-Agent is an open-source framework developed by Alibaba Cloud for building advanced LLM-powered applications. It harnesses the capabilities of Qwen models to create AI agents with support for instruction following, tool usage, planning, and memory management.

This library notably serves as the backend for Qwen Chat. It offers core components for building AI agents, with native support for function and tool calling—even via MCP (Model Context Protocol).

Qwen-Agent enables flexible model deployment, whether through Alibaba Cloud’s DashScope service or self-hosted Qwen models. It also includes a Gradio-based web UI for quickly deploying and testing your agents in the browser.

Why Combine Qwen3 with an MCP Server in Qwen-Agent

Qwen3 is the latest model developed by Alibaba Cloud. It delivers strong performance and, being open-source, is available for free through multiple online providers (and you can even self-host it!). That makes it an ideal choice for building cost-effective, yet powerful, AI agents.

Now, AI agents powered by Qwen3 and built with Qwen-Agent are already capable. Still, they inherit the limitations of the underlying LLM. These limitations—such as the lack of access to real-time information—can be overcome by providing the agent with accurate data and enabling it to perform live web exploration.

That is where the Bright Data MCP server comes in. Built on Node.js, the MCP server integrates with Bright Data’s suite of AI data retrieval tools. Through these tools, your agent gains the ability to access real-time web content, query structured datasets, and perform live web scraping.

As of now, the supported MCP tools are:

Tool Description
search_engine Scrape search results from Google, Bing, or Yandex. Returns SERP results in markdown format (URL, title, description).
scrape_as_markdown Scrape a single webpage and return the extracted content in Markdown format. Works even on bot-protected or CAPTCHA-secured pages.
scrape_as_html Same as above, but returns content in raw HTML.
session_stats Provides a summary of tool usage during the current session.
web_data_amazon_product Retrieve structured Amazon product data using a /dp/ URL. More reliable than scraping due to caching.
web_data_amazon_product_reviews Retrieve structured Amazon review data using a /dp/ URL. Cached and reliable.
web_data_linkedin_person_profile Access structured LinkedIn profile data. Cached for consistency and speed.
web_data_linkedin_company_profile Access structured LinkedIn company data. Cached version improves reliability.
web_data_zoominfo_company_profile Retrieve structured ZoomInfo company data. Requires a valid ZoomInfo URL.
web_data_instagram_profiles Structured Instagram profile data. Requires a valid Instagram URL.
web_data_instagram_posts Retrieve structured data for Instagram posts.
web_data_instagram_reels Retrieve structured data for Instagram reels.
web_data_instagram_comments Retrieve Instagram comments as structured data.
web_data_facebook_posts Access structured data for Facebook posts.
web_data_facebook_marketplace_listings Retrieve structured listings from Facebook Marketplace.
web_data_facebook_company_reviews Retrieve Facebook company reviews. Requires a company URL and number of reviews.
web_data_x_posts Retrieve structured data from X (formerly Twitter) posts.
web_data_zillow_properties_listing Access structured Zillow listing data.
web_data_booking_hotel_listings Retrieve structured hotel listings from Booking.com.
web_data_youtube_videos Structured YouTube video data. Requires a valid video URL.
scraping_browser_navigate Navigate the scraping browser to a new URL.
scraping_browser_go_back Navigate back to the previous page.
scraping_browser_go_forward Navigate forward in the browser history.
scraping_browser_click Click a specific element on the page. Requires element selector.
scraping_browser_links Retrieve all links on the current page along with their selectors and text.
scraping_browser_type Simulate typing text into an input field.
scraping_browser_wait_for Wait for a specific element to become visible.
scraping_browser_screenshot Take a screenshot of the current page.
scraping_browser_get_html Retrieve the full HTML of the current page. Use with care if full-page content is not needed.
scraping_browser_get_text Retrieve the visible text content of the current page.

To explore some real-world examples, read our guide on scraping with the MCP server or see how it integrates with tools like the Google SDK.

Note: Bright Data continuously expands the MCP toolset, so expect even more capabilities over time.

Now, take a look at how you can use these MCP tools through Qwen3 with Qwen-Agent!

How to Integrate Qwen-Agent with the Bright MCP Server in Python

In this section, you will learn how to use Qwen-Agent to build a powerful Python AI agent powered by Qwen3. The agent will be equipped with live data scraping, retrieval, and transformation capabilities via the Bright Data MCP server.

As an example, we will show how the AI agent can retrieve live product data from Amazon—which is known for its scraping challenges. Note that this is just one possible use case. The AI agent can leverage any of the 20+ tools available through the MCP server to handle a wide variety of tasks.

Note: By simply giving the AI agent different prompts, you can accomplish any other scenario or use case.

Follow the steps below to build your Bright Data MCP-powered AI agent using Qwen-Agent and Qwen 3!

Prerequisites

To follow this tutorial, you must have:

You will also need:

  • A Bright Data account.
  • An OpenRouter API key.

The steps below will guide you through setting up both the Bright Data and OpenRouter credentials when needed. So, do not worry about them right now.

While not required, the following knowledge will help you get the most out of this tutorial:

  • A general understanding of how MCP works.
  • Basic familiarity with how Qwen Agent functions.
  • Some knowledge of the Bright Data MCP server and its available tools.
  • Some experience with asynchronous programming in Python.

Step #1: Project Setup

Open the terminal and create a new folder for your MCP-powered AI agent:

mkdir qwen-agent-mcp

The qwen-agent-mcp folder will contain all the code for your Python AI agent.

Next, navigate into the project folder and create a virtual environment inside it:

cd qwen-agent-mcp
python -m venv venv

Open the project folder in your preferred Python IDE. We recommend using Visual Studio Code with the Python extension or PyCharm Community Edition.

Create a file named agent.py inside the project folder, which should now look like this:

The file structure of the Qwen-Agent MCP agent project

The agent.py file is currently empty, but it will soon contain the logic for integrating Qwen3 with the Bright Data MCP server.

Activate the virtual environment using the terminal in your IDE. In Linux or macOS, execute this command:

source venv/bin/activate

Equivalently, on Windows, launch:

venv/Scripts/activate

You are all set! You now have a Python environment to build an AI agent using the Qwen-Agent and the Bright Data MCP server.

Step #2: Set Up Environment Variables Reading

Your project will interact with third-party services like OpenRouter and Bright Data. Avoid hardcoding API keys into your Python code. Instead, load them from environment variables for better security and maintainability.

To simplify this process, we will take advantage of the python-dotenv library. With your virtual environment activated, install it by running:

pip install python-dotenv

Next, in your agent.py file, import the library and load the environment variables with load_dotenv():

from dotenv import load_dotenv

load_dotenv()

This enables your script to read environment variables from a local .env file. So go ahead and create a .env file inside your project folder:

The .env file in your nested agent folder

You can now access environment variables in your code like this:

env_value = os.getenv("<ENV_NAME>")

Do not forget to import the os module from the Python standard library:

import os

Great! You are now set up to securely load secrets from the environment variables.

Step #3: Get Started With Google ADK

In your virtual environment activated, install the Qwen-Agent library by running:

pip install "qwen-agent[gui,mcp]"

Note that qwen-agent supports a few optional submodules. For this tutorial, these two are enough:

  • [gui] for a Gradio-based user interface.
  • [mcp] for integrating with any MCP server.

Then, open your agent.py file and include the following imports:

from qwen_agent.agents import Assistant
from qwen_agent.gui import WebUI

These will be used in the next steps to connect Qwen-Agent with the MCP server and implement your AI agent.

By default, Qwen-Agent expects a DashScope API key in your environment to access Qwen models. However, in this tutorial, we will use Qwen3 via the free OpenRouter platform.

⚠️ Warning: You might wonder why we are not using DashScope from Alibaba Cloud to access Qwen3 directly. The reason is that, as of this writing, there are limitations for international users (i.e., users outside of China). These restrictions currently lead to the following (somewhat misleading) error message:

Error code: InvalidApiKey. Error message: Invalid API-key provided.

To avoid this issue, you can rely on OpenRouter or any other platfrom that provides access to Qwen3 without regional restrictions.

Let’s now walk through how to set up your OpenRouter API key for Qwen3 integration!

Step #4: Retrieve Your OpenRouter API Key

If you have not already, sign up for an OpenRouter account. If you already have one, simply log in.

Next, navigate to the “API Keys” page by hovering over your profile image in the top-right corner and selecting the “Keys” option:

The “API Keys” page in the OpenRouter platform

Click the “Create API Key” button and follow the instructions to generate your key. Then, copy the key and add it to your .env file like so:

OPEN_ROUTER_API_KEY="<YOUR_OPEN_ROUTER_API_KEY>"

Replace the <YOUR_OPEN_ROUTER_API_KEY> placeholder with your actual OpenRouter API key.

Next, in your agent.py file, load the key using:

OPEN_ROUTER_API_KEY = os.getenv("OPEN_ROUTER_API_KEY")

Terrific! You are now ready to use Qwen3 with Qwen-Agent via OpenRouter.

Step #5: Set Up the Bright Data MCP Server

Start by [creating a new Bright Data account]() or just log in to your existing account.

Next, follow the official instructions to:

  1. Retrieve your Bright Data API token.
  2. Configure Web Unlocker and Scraping Browser for MCP integration.

Once done, you should have:

  • A Bright Data API token.
  • A Web Unlocker zone (here, we will assume the zone name is the default “mcp_unlocker”).
  • Scraping Browser credentials in the format <BRIGHT_DATA_SB_USERNAME>:<BRIGHT_DATA_SB_PASSWORD>.

Now, install the Bright Data MCP server globally in your Node.js environment with:

npm install -g @brightdata/mcp

Then, make sure to specify two environments in your terminal:

API_TOKEN="<YOUR_BRIGHT_DATA_API_TOKEN>"
BROWSER_AUTH="<BRIGHT_DATA_SB_USERNAME>:<BRIGHT_DATA_SB_PASSWORD>"

Follow the correct procedure for environment variable definition based on the operating system.

You can now start the MCP server via @brightdata/mcp npm package with:

npx -y @brightdata/mcp

This command launches the MCP server locally, reading the required environment variables (API_TOKEN and BROWSER_AUTH). If everything is configured correctly, you should see an output like this:

Checking for required zones...
Required zone "mcp_unlocker" already exists
Starting server...

Nice work! The Bright Data MCP server works like a charm.

Next, add those two environment variables to your .env file:

BRIGHT_DATA_API_TOKEN="<YOUR_BRIGHT_DATA_API_TOKEN>"
BRIGHT_DATA_BROWSER_AUTH="<BRIGHT_DATA_SB_USERNAME>:<BRIGHT_DATA_SB_PASSWORD>"

Replace the placeholders with the actual values.

In your agent.py, load these variables with:

BRIGHT_DATA_API_TOKEN = os.getenv("BRIGHT_DATA_API_TOKEN")
BRIGHT_DATA_BROWSER_AUTH = os.getenv("BRIGHT_DATA_BROWSER_AUTH")

Perfect! You now have everything set up to integrate the Bright Data MCP server with Qwen-Agent.

Step #6: Define the Qwen3 MCP-Powered Agent

Create a function to initialize your AI agent with Qwen3 and MCP support:

def initialize_mcp_agent():
    # To connect to Qwen3 via OpenRouter
    llm_cfg = {
        "model": "qwen/qwen3-32b:free",
        "model_server": "https://openrouter.ai/api/v1",
        "api_key": OPEN_ROUTER_API_KEY,
    }

    # To connect to the Bright Data MCP server
    tools = [{
        "mcpServers": {
            "brightdata": {
                "command": "npx",
                "args": ["-y", "@brightdata/mcp"],
                "env": {
                    "API_TOKEN": BRIGHT_DATA_API_TOKEN,
                    "BROWSER_AUTH": BRIGHT_DATA_BROWSER_AUTH,
                }
            }
        }
    }]


    # Define the Qwen-Agent assistant with MCP integration
    agent = Assistant(
        llm=llm_cfg,
        function_list=tools,
        name="MCP-Powered Assistant with Access to Live Data",
        description="This agent can answer questions by retrieving up-to-date information from the Internet using configured MCP tools"
    )

    return agent

As you can see, in the above code:

  • The llm_cfg dictionary sets up access to the free version of Qwen3 via the OpenRouter API.
  • The tools array defines how to connect to the Bright Data MCP server, enabling the AI agent to call external data retrieval tools.
  • Finally, the Assistant() class provided by Qwen-Agent is used to define the AI agent with both the LLM and tool integrations.

This is it! Thanks to Qwen-Agent, MCP integration into your AI agent takes just a few lines of code.

Step #7: Launch Your Qwen3 MCP Agent

Add the following code to agent.py to run the defined AI agent in a Gradio interface:

if __name__ == "__main__":
    # Initialize the MCP-powered agent
    mcp_agent = initialize_mcp_agent()

    # Launch the Gradio-based web UI to interact with the AI agent in the browser
    WebUI(mcp_agent).run()

Mission complete! It only remains to test the code and see what the AI agent is capable of.

Step #8: Put It All Together

This is the final code in agent.py:

from dotenv import load_dotenv
import os
from qwen_agent.agents import Assistant
from qwen_agent.gui import WebUI

# Load the environment variables from the .env file
load_dotenv()

# Read the API key from OpenRouter to use Qwen3
OPEN_ROUTER_API_KEY = os.getenv("OPEN_ROUTER_API_KEY")

# Read the envs for integration with the Bright Data MCP server
BRIGHT_DATA_API_TOKEN = os.getenv("BRIGHT_DATA_API_TOKEN")
BRIGHT_DATA_BROWSER_AUTH = os.getenv("BRIGHT_DATA_BROWSER_AUTH")

# Define the agent with Qwen 3 and MCP configuration
def initialize_mcp_agent():
    # To connect to Qwen3 via OpenRouter
    llm_cfg = {
        "model": "qwen/qwen3-32b:free",
        "model_server": "https://openrouter.ai/api/v1",
        "api_key": OPEN_ROUTER_API_KEY,
    }

    # To connect to the Bright Data MCP server
    tools = [{
        "mcpServers": {
            "brightdata": {
                "command": "npx",
                "args": ["-y", "@brightdata/mcp"],
                "env": {
                    "API_TOKEN": BRIGHT_DATA_API_TOKEN,
                    "BROWSER_AUTH": BRIGHT_DATA_BROWSER_AUTH,
                }
            }
        }
    }]


    # Define the Qwen-Agent assistant with MCP integration
    agent = Assistant(
        llm=llm_cfg,
        function_list=tools,
        name="MCP-Powered Assistant with Access to Live Data",
        description="This agent can answer questions by retrieving up-to-date information from the Internet using configured MCP tools"
    )

    return agent

if __name__ == "__main__":
    # Initialize the MCP-powered agent
    mcp_agent = initialize_mcp_agent()

    # Launch the Gradio-based web UI to interact with the AI agent in the browser
    WebUI(mcp_agent).run()

Wow! With just around 50 lines of code, you can build a powerful MCP-powered AI agent—thanks to Qwen-Agent and OpenRouter.

Execute the AI agent with:

python agent.py

In the terminal, you should see an output as follows:

2025-05-27 15:40:58,783 - mcp_manager.py - 122 - INFO - Initializing MCP tools from mcp servers: ['brightdata']
2025-05-27 15:40:58,800 - mcp_manager.py - 340 - INFO - Initializing a MCP stdio_client, if this takes forever, please check the config of this mcp server: brightdata
2025-05-27 15:41:01,098 - mcp_manager.py - 350 - INFO - No list resources: Method not found
* Running on local URL:  http://127.0.0.1:7860

This tells you that the MCP server is up and running, and your AI agent is connected and accessible at http://127.0.0.1:7860. Visit that URL in your browser to interact with the agent through the Gradio web UI.

For example, try asking it a prompt like this:

From the Amazon product page "https://www.amazon.com/PlayStation%C2%AE5-console-slim-PlayStation-5/dp/B0CL61F39H/", extract the main info and return it in JSON format

This is a great test to verify whether the AI agent can retrieve (and therefore utilize/learn from) real-time data from the Amazon PS5 product page:

Amazon PS5 page

When you run this prompt in the UI, here is what should happen:

Executing the Amazon data extraction request in Qwen-Agent

First, note that on the right-hand panel, there is a list of all available Bright Data MCP server tools. Also, see how the Qwen-Agent uses the web_data_amazon_product tool from the Bright Data MCP server to fulfill the request.

You can verify that by checking the bullet list in the interface, which shows the result of the tool execution:

See the result of the web_data_amazon_product tool

In the end, the final JSON output should look like this:

{
  "title": "PlayStation®5 console (slim)",
  "url": "https://www.amazon.com/PlayStation%C2%AE5-console-slim-PlayStation-5/dp/B0CL61F39H",
  "brand": "Sony",
  "model_number": "CFI-2015",
  "price": {
    "currency": "USD",
    "final_price": 499
  },
  "availability": "In Stock",
  "rating": 4.7,
  "reviews_count": 6824,
  "description": "The PS5 console unleashes new gaming possibilities that you never anticipated. Experience lightning fast loading with an ultra-high speed SSD, deeper immersion with support for haptic feedback, adaptive triggers, and 3D Audio*, and an all-new generation of incredible PlayStation games...",
  "key_features": [
    "Custom CPU, GPU, and SSD with Integrated I/O",
    "Support for haptic feedback, adaptive triggers, and 3D Audio",
    "Backward compatibility with PS4 games",
    "1TB SSD and 10.6 lb console weight",
    "Includes DualSense Wireless Controller"
  ],
  "delivery_info": "FREE delivery Sunday, June 1. Or Prime members get FREE Same-Day delivery Today 10 AM - 3 PM.",
  "images": [
    "https://m.media-amazon.com/images/I/41ECK5cY-2L.SL1000.jpg",
    "https://m.media-amazon.com/images/I/41srF-iY93L.SL1000.jpg"
    // more image URLs...
  ],
  "product_details": {
    "ASIN": "B0CL61F39H",
    "Release Date": "December 10, 2023",
    "Dimensions": "14 x 17 x 7 inches",
    "Weight": "10.6 pounds",
    "Best Seller Rank": "#18 in Video Games",
    "Manufacturer": "Sony"
  },
  // omitted for brevity...
}

Now, this simple example demonstrates just how powerful your AI agent is. It can extract live, structured data from any site on the fly. This is perfect for live search, RAG (Retrieval-Augmented Generation), and up-to-date decision-making.

Even better, the agent can also use Agent Browser (previously Scraping Browser) to interact with web pages directly, enabling high-level automation and complex workflows.

Et voilà! That is the power of Qwen3 + MCP integration with Qwen-Agent for building next-gen AI agents.

Conclusion

In this blog post, you learned how to build a powerful AI agent in Python using the Qwen-Agent library in combination with Qwen3 and Bright Data MCP.

As shown, integrating a feature-rich MCP server with Qwen-Agent enables your AI agent to retrieve real-time data from the web and perform advanced tasks autonomously. This is just one example of how Bright Data’s tools can power intelligent, automated AI workflows.

Explore the solutions in our AI infrastructure:

  • Autonomous AI agents: Search, access, and interact with any website in real-time using a powerful set of APIs.
  • Vertical AI apps: Build reliable, custom data pipelines to extract web data from industry-specific sources.
  • Foundation models: Access compliant, web-scale datasets to power pre-training, evaluation, and fine-tuning.
  • Multimodal AI: Tap into the world’s largest repository of images, videos, and audio—optimized for AI.
  • Data providers: Connect with trusted providers to source high-quality, AI-ready datasets at scale.
  • Data packages: Get curated, ready-to-use, structured, enriched, and annotated datasets.

For more information, explore our AI hub.

Create a Bright Data account and try all our products and services for AI agent development!

No credit card required