Give AstrBot the Ability to Interact With the Web Using Bright Data (MCP + Skills)

In this tutorial, you will learn:

• What AstrBot is and what it provides as an open-source AI agent platform.
• Why adding live web access makes AstrBot more effective for real-world tasks.
• How Bright Data equips AstrBot with enterprise-grade web search, scraping, discovery, and browser automation tools.
• How to integrate the Bright Data Web MCP with AstrBot.
• How to enhance AstrBot with Bright Data product expertise using the Bright Data skills.
• The results of combining AstrBot with the Web MCP and the Bright Data skills.

Let’s dive in!

What Is AstrBot?

AstrBot is an open-source, cross-platform AI agent framework. It brings LLM-powered assistants directly into popular chat applications such as Telegram, Slack, Discord, WeChat, and more.

It goes beyond traditional chatbots by supporting agent workflows, tool calling, MCP, skills, and knowledge bases. It also comes with plugin extensions for automation, customer support, enterprise knowledge management, and personal AI assistants.

AstrBot has earned 36.5k+ GitHub stars, has 1,000+ community plugins, and 200k+ weekly active users.

Key Features

The main capabilities supported by AstrBot:

• Multi-platform support: Chat to your agent via Telegram, Slack, Discord, QQ, WeChat, Lark, DingTalk, and many other messaging platforms.
• Native MCP support: Integrate external tools and services through the Model Context Protocol.
• Skills integration: Extend agents with reusable skills that provide specialized capabilities and workflows.
• 1,000+ plugin ecosystem: Add new features and integrations through a rich community-maintained plugin marketplace.
• Knowledge base: Build AI assistants that answer questions using PDFs, DOCX files, Markdown, and other documents.
• Multi-provider LLM support: Works with OpenAI, Anthropic, Google Gemini, Ollama, and many OpenAI-compatible providers.
• Agent sandbox: Execute code and shell commands safely in isolated runtime environments.
• Powerful WebUI: Configure models, plugins, MCP servers, logs, and debugging from a centralized interface.
• Flexible deployment: Deploy with uv, Docker, desktop apps, launchers, or cloud environments.
• Workflow automation: Schedule tasks, orchestrate tools, and build autonomous AI agents for real-world workflows.

Find out more in the official docs.

Why AstrBot Needs Access to the Web

There is no doubt that AstrBot is a solid AI framework packed with advanced features. However, like all LLM-powered applications, it inherits these fundamental limitations:

• Outdated knowledge: LLMs are trained on static datasets that stop at a specific point in time.
• Limited access to the web: On their own, they cannot reliably interact with web pages.

AstrBot helps overcome part of this challenge with a built-in web search tool. Still, this tool can struggle at scale or when accessing websites protected by anti-bot technologies such as CAPTCHAs, rate limits, and fingerprinting.

This is where Bright Data comes in. By connecting AstrBot to Bright Data solutions, the agent receives access to enterprise-grade tools for live web search, data extraction, website discovery, browser automation, and dataset retrieval.

What sets Bright Data apart is its global web data infrastructure, powered by more than 400 million residential IPs. This architecture delivers the scalability and reliability required for production workloads, supporting unlimited concurrency, 99.99% SLA-backed uptime, and achieving a 99.95% request success rate.

How Bright Data Supports AstrBot With Web Search, Scraping, Discovery, and Interaction

Bright Data can connect to AstrBot via two integrations:

• Bright Data Web MCP: An MCP server that exposes 70+ tools for web search, scraping, structured data extraction, and browser automation.
• Bright Data skills: A collection of Agent Skills that provide AstrBot with specialized knowledge about Bright Data products, recommended workflows, and best practices. Most of these skills rely on the Bright Data CLI.

Note: The two approaches are complementary. For example, a specific Bright Data skill helps AstrBot understand when and how to use the Web MCP tools effectively.

Together, the Web MCP and the Bright Data skills transform AstrBot into a more capable AI agent. Thanks to them, it can perform real-time web research, browser automation, and structured data extraction.

Bright Data Web MCP

The Bright Data Web MCP equips AstrBot with a large set of tools powered by Bright Data APIs.

In Rapid mode (the free tier with 5,000 free requests per month), AstrBot can utilize essential tools for web research and data extraction. These include:

Tool(s)	Description
search_engine (and search_engine_batch for parallel searching)	Retrieve structured search results from Google, Bing, Yandex, and others.
scrape_as_markdown (and scrape_batch for parallel scraping)	Extract clean Markdown from web pages while automatically handling anti-bot protections.
discover	Perform AI-powered web discovery with relevance ranking and intent-based filtering.

By enabling Pro mode, AstrBot gains access to tools for extracting structured data from LinkedIn, Amazon, Instagram, Walmart, YouTube, Google Maps, Zillow, TikTok, and more than 40 other platforms. It also unlocks multiple browser automation tools backed by Bright Data’s Browser API.

Bright Data Skills

The Bright Data skills extend AstrBot with specialized knowledge and workflows. They help the agent choose the right Bright Data services and follow recommended best practices.

Some of the available skills include:

Skill	Description
agent-onboarding	Guides new agents through Bright Data setup and recommends MCP, CLI, API, and authentication workflows.
search	Performs live Google searches and returns structured search results.
scrape	Explains how to extract clean webpage content with automatic anti-bot handling.
scraper-builder	Helps build production-ready scrapers by analyzing websites and generating extraction logic.
scraper-studio	Creates and runs AI-generated scrapers through the Bright Data CLI.
brightdata-cli	Teaches the agent how to use the Bright Data CLI for scraping, search, datasets, proxies, and account management.
bright-data-mcp	Provides guidance for orchestrating the 70+ Web MCP tools.
brightdata-proxy	Generates proxy integration code for Python, Node.js, Playwright, Puppeteer, Selenium, and other environments.
seo-audit	Performs SEO analysis using live website data and search signals.
competitive-intel	Supports competitor monitoring, pricing analysis, review tracking, and market research.
price-comparison	Compares product prices and availability across major online retailers.
live-research	Builds research reports by combining web discovery, extraction, and AI summarization.
brand-listening	Analyzes brand mentions, sentiment, and online discussions across multiple sources.
rag-pipeline	Helps build retrieval-augmented generation (RAG) pipelines using live web data.
bright-data-best-practices	Recommends best practices for Bright Data APIs and web data workflows.

AstrBot Setup: Step-by-Step Guide

In this section, you will see how to set up AstrBot and connect it to Telegram.

Note: You can easily adapt the instructions below to any other messaging platform supported by the library.

Prerequisites

Make sure you have:

• uv installed on your local machine.
• An API key from one of the LLM providers supported by AstrBot. Here, we will use an OpenAI API key.
• A Telegram account with a bot ready to connect to AstrBot. Follow the official guide to create a Telegram bot and obtain its bot token.

Step #1: Install and Run AstrBot

First, install AstrBot by running the following command:

uv tool install astrbot, python 3.12

Next, navigate to your project folder and initialize the AstrBot setup:

astrbot init

You will receive a prompt asking whether you want to install AstrBot in the current directory. Enter “Y” to continue:

After the setup is complete, start AstrBot with:

astrbot run

You should see output similar to the following:

The AstrBot WebUI will now be available at http://localhost:6185. Open this URL in your browser and log in using the credentials provided in the terminal logs:

After logging in, set a new password to access the dashboard in the AstrBot WebUI. Well done!

Step #2: Add the LLM Integration

On the AstrBot WebUI “Welcome” page, click the “Configure” button in the “Configure AI Model” section to get started:

Here, we will connect AstrBot to OpenAI (but the process is similar for any other supported LLM provider). Select Add and choose the “OpenAI Compatible” option. The “openai” configuration modal will open. Paste your OpenAI API key into the corresponding field:

Click “Save and Fetch Models” to dynamically retrieve the available OpenAI models. Then, select the model you want to use. In this example, we will rely on OpenAI GPT-5.4 Mini:

Finally, press “Save” to apply the configuration and return to the “Welcome” page. Your AstrBot agent is now powered by an OpenAI LLM. Amazing!

Step #3: Configure for Telegram Connection

On the AstrBot WebUI “Welcome” page, click the “Configure” button under “Configure Platform Bot”. In the “Add Platform Adapter” model, select “Telegram” and paste your Telegram bot token into the corresponding field:

Press “Save” and then open your Telegram bot (named “AstrBot Agent” in this example). You should receive a message containing the following text:

Hello, I'm AstrBot!

To verify that the LLM integration is working correctly, send a test message such as:

Hey, how's it going?

You should receive a response like this:

That was generated by GPT-5.4 Mini and sent via AstrBot.

Setup complete! The framework is now fully configured, and you can interact with it through Telegram.

Connect AstrBot to the Bright Data Web MCP

In this chapter, you will learn how to set up the Bright Data Web MCP in AstrBot.

Prerequisites

For a smoother experience following this section, verify that you have:

• A basic understanding of how MCP works.
• Familiarity with the tools provided by the Bright Data Web MCP.

Also, you must have a Bright Data account with an API key configured. Read the official guide to generate your Bright Data API key.

Step #1: Get Familiar with Bright Data Web MCP

Before getting started, keep in mind that AstrBot recommends connecting it to remote MCP servers. For this reason, instead of setting up a local Bright Data Web MCP server with the @brightdata/mcp package, we will integrate the remote Web MCP server.

Note: The remote Bright Data Web MCP is built for enterprise-scale deployments. It supports unlimited scalability and concurrent connections, just like all other Bright Data services.

To begin, familiarize yourself with the standard Web MCP connection URL:

https://mcp.brightdata.com/mcp?token=<YOUR_BRIGHT_DATA_API_KEY>&pro=1

Replace <YOUR_BRIGHT_DATA_API_KEY> with your Bright Data API key. The token parameter authenticates your requests and associates them with your Bright Data account.

The &pro=1 parameter is optional:

• With &pro=1: Access to the complete catalog of 70+ tools and advanced capabilities. Standard Bright Data usage fees apply.
• Without &pro=1: Get only the free tools available in Rapid mode (up to 5,000 requests per month).

Now, suppose finer-grained control over the exposed tools. For instance, you want to configure access to only specific tools or tool groups. In that case, you can generate a custom Web MCP connection URL from the Bright Data dashboard.

Sign in to your Bright Data account and navigate to the “AI Gateways > MCP” page. Follow the setup wizard to configure your MCP server and generate a custom connection URL:

Copy the generated “Streamable HTTP” connection URL, or construct one using the format shown above. You will use this URL in the next step to connect AstrBot to the Bright Data Web MCP. Cool!

Step #2: Add the Bright Data Web MCP Connection in AstrBot

AstrBot allows you to configure MCP connections directly from its WebUI.

To add a new MCP server, open the “Extensions” dropdown on the left and select the “MCP Servers” option. On the “MCP” page, click the “+” button in the bottom-right corner:

Give your MCP server a name (for example, “Bright Data Web MCP”) and paste the following configuration:

{
  "transport": "streamable_http",
  "url": "<YOUR_WEB_MCP_CONNECTION_URL>"
}

Replace <YOUR_WEB_MCP_CONNECTION_URL> with the Bright Data Web MCP connection URL retrieved earlier.

Press “Test Connection” to verify that the MCP server is configured correctly. If the connection is successful, AstrBot will display a confirmation message followed by a list of the available tools:

When using Pro mode (&pro=1), you will see all 70+ Bright Data Web MCP tools.

Complete the integration by clicking “Save”. The ./data/mcp_server.json AstrBot configuration file will include:

{
  "mcpServers": {
    "Bright Data Web MCP": {
      "active": true,
      "transport": "streamable_http",
      "url": "<YOUR_WEB_MCP_CONNECTION_URL>"
    }
  }
}

Alternatively, you can achieve the same result by manually editing this configuration file.

After saving, the “MCP” page will display the new “Bright Data Web MCP” entry, which should be enabled by default:

Terrific! AstrBot is now connected to Bright Data Web MCP and can call its web interaction tools.

Later in this guide, you will see an AstrBot setup with Web MCP together with the Bright Data skills.

Add Knowledge About Bright Data to AstrBot Using the Official Skills

Below, you will be guided through the process of adding the Bright Data skills to AstrBot using Vercel’s skills CLI tool.

Note: If you do not want to use skills, you can download the skills directly from the official repository. Navigate to the skills/ folder, create a ZIP archive with its content, and manually upload it from the “Extensions > Skills” page in AstrBot.

Prerequisites

Before going through this section, check that you have:

• A UNIX-based environment (Linux, macOS, or WSL).
• Node.js 22+ installed locally.
• The Bright Data CLI installed and configured locally. This is the main requirement for using the Bright Data skills. For a quick setup, refer to the “Bright Data CLI: Installation & Setup” guide. Alternatively, consult the dedicated blog post for more details.
• A basic understanding of what the Bright Data skills provide. For more information, read the Bright Data skills article on our blog.

Some familiarity with the Agent Skills standard skills CLI tool is also recommended, but not required.

Step #1: Install the Bright Data Skills

To add the Bright Data skills to AstrBot using the skills CLI tool, run:

npx skills add brightdata/skills -a astrbot

This command installs the skills npm package and starts the setup process. This will:

1. Download the Bright Data skills from the Agent Skills Directory.
2. Configure them for use either globally or within your local project.

You will then be prompted to select which skills to install:

To install all available skills, use the Spacebar to select each one, then press Enter to continue.

Next, choose the installation scope. Since the Web MCP integration was configured locally, you should install the Bright Data skills locally as well. Thus, select the “Project” option and press Enter:

Press Enter again to confirm the installation. Once completed, you will see a confirmation message:

The Bright Data skills are now available in the AstrBot ./data/skills directory. Awesome!

Step #2: Confirm the Skills Are Available

Return to the AstrBot WebUI and open the “Extensions > Skills” page:

You will see a list containing all the Bright Data skills you just installed.

Note: If the skills do not appear, try refreshing the page.

Optionally, if you have not configured the Bright Data CLI yet, run the onboarding skill. In Telegram write:

/agent-onboarding

This loads the dedicated Bright Data onboarding skill, which guides you through the setup process required by the other Bright Data skills. To run it, you must also provide AstrBot with permission to execute shell commands.

Mission complete! In the next chapter, you will learn how to use the Bright Data Web MCP and skills together within AstrBot.

AstrBot + Bright Data Web MCP and Skills: Full Example

Your AstrBot agent now has enterprise-ready tools and knowledge for interacting with the web. This enables it to handle a wide range of real-world use cases that require live web data, browser automation, and structured data extraction.

For example, suppose you want to use AstrBot as a shopping assistant. Modern e-commerce websites are often protected by advanced anti-bot technologies, making it difficult for AI agents to interact with them. For more info, see our guide on e-commerce scraping.

By integrating Bright Data, AstrBot can bypass these limitations while searching the web, interacting with web pages, and extracting the information needed to complete your requests.

Now, imagine you want to use AstrBot through Telegram to help you find a product on the Zara website. See the steps below!

Step #1: Launch the Prompts

Start with a prompt like this:

Search Google for the Zara men's polo shirts sale page. Visit the page in a browser and return a full-page screenshot.

You should receive a response similar to:

Notice how the agent completed the task. Behind the scenes, it:

• Called the search_engine tool (backed by Bright Data’s SERP API) to search Google for the “Zara men’s polo shirts sale page” query.
• Parsed the returned SERP in JSON to identify the correct page.
• Opened the page in a cloud browser instance using the scraping_browser_navigate tool (powered by Bright Data’s Browser API).
• Captured a full-page screenshot with the scraping_browser_screenshot tool.
• Returned both the screenshot and the page URL.

If you open the AstrBot WebUI and navigate to “Platforms > Platform Logs”, you can inspect every Web MCP tool and Bright Data skill the agent used:

Suppose you open the screenshot, like the third product, and want AstrBot to retrieve all the information about it. You can send a prompt such as:

I like the third product, the Zip Polo Sweatshirt. Get the URL of its product page, then scrape it to extract the main product information. Return the results in a structured text message.

The response should look similar to this:

Under the hood, AstrBot calls multiple Bright Data tools to identify the product page and extract its information. Depending on the workflow, it can retrieve the data using scrape_as_markdown or a specialized tool such as web_data_zara_products (which is based on Bright Data’s Zara Web Data API).

Step #2: Analyze the Output

Notice that the returned information from the last message matches what is available on the official Zara product page:

A product page can require several manual interactions to reveal all of its information (e.g., expanding the “PRODUCT MEASUREMENTS” section). Despite that, AstrBot was able to return the full product details thanks to the specialized tools exposed by Bright Data’s Web MCP.

From there, you can continue the conversation naturally. For example, you could ask AstrBot whether the product is worth buying, compare it with similar alternatives, or recommend other products within the same price range.

Et voilà! This simple example highlights how much more capable AstrBot becomes when integrated with Bright Data. Without the Web MCP and the Bright Data skills, it would not be able to reliably complete workflows that depend on live web search, website discovery, data extraction, and automated browser interactions.

Conclusion

In this blog post, you learned what AstrBot is and what it brings to the table. In particular, you saw how to extend its functionality by integrating Bright Data through the Web MCP and the Bright Data skills.

With Bright Data, AstrBot can leverage tools for live web research, scraping, and source discovery. This enables it to generate more grounded responses based on accurate, contextual, verifiable information. It can also use cloud browser interaction tools, allowing you to delegate complex web workflows directly to the agent.

For even more advanced scenarios, explore the full range of AI-ready web data solutions available through the Bright Data ecosystem.

Sign up for Bright Data for free today and start exploring our web data solutions!