AI Agents Using AWS Strands SDK and Bright Data’s Web MCP

AI agents powered by large language models (LLMs) can reason and make decisions, but they’re limited by their training data. To build truly useful agents, you need to connect them to real-time web data. This guide shows you how to combine AWS Strands SDK with Bright Data’s Web MCP server to create autonomous AI agents that can access and analyze live web data.

In this guide, you will learn:

• What AWS Strands SDK is and what makes it unique as a framework for building AI agents
• Why AWS Strands SDK pairs perfectly with Bright Data’s Web MCP server for web-aware agents
• How to integrate AWS Strands with Bright Data’s Web MCP to create an autonomous competitive intelligence agent
• How to build agents that autonomously decide which web scraping tools to use based on their goals

Let’s dive in!

What Is AWS Strands SDK?

AWS Strands SDK is a lightweight, code-first framework developed by AWS for building AI agents with minimal code. It takes a model-driven approach where agent capabilities emerge from model decisions rather than hardcoded logic.

Compared to other AI agent frameworks, AWS Strands SDK emphasizes simplicity, flexibility, and production readiness. In detail, some of its main characteristics are:

• Model Agnostic: Support for multiple LLM providers including AWS Bedrock, OpenAI, Anthropic, and others
• Native MCP Support: Built-in integration with Model Context Protocol for accessing 1000+ pre-built tools
• Minimal Code: Build sophisticated agents with just a few lines of code
• Production Ready: Includes error handling, retries, and observability out of the box
• Agentic Loop: Implements perception-reasoning-action cycles for autonomous decision-making
• Multi-Agent Support: Orchestration primitives for coordinating multiple specialized agents
• State Management: Session and context management across interactions

Understanding AWS Strands SDK

Core Architecture

AWS Strands SDK simplifies agent development with a clean three-component design that doesn’t compromise power.

This approach lets you build smart agents with minimal code that would otherwise require thousands of lines.:

1. Model Component: The brain that works with multiple AI providers
2. Tools Integration: Connects agents to external systems through MCP servers
3. Prompt-Based Tasks: Define agent behavior with natural language instead of code

Agentic Loop Implementation

The agentic loop is what makes Strands agents so smart. It’s like a continuous cycle where the agent perceives what’s happening, thinks about it, and takes action allowing it to handle complex tasks all on its own.

Strands changes the game by letting the AI model decide what to do next. Instead of coding every possible scenario, the model figures things out based on the current situation.

Here’s how it works in practice:

1. Your agent gets a task from a user.
2. The model looks at what’s happening and what tools it has available.
3. It decides whether to use a tool, ask for clarification, or give a final answer.
4. If it uses a tool, Strands runs it and feeds the results back to the model.
5. This cycle continues until the job is done or human help is needed.

Think about building a price monitoring tool, with traditional coding, you’d have to write logic for checking competitor websites, handling different page layouts, managing errors, collecting results, and setting up alert thresholds.

With Strands, you just provide web scraping tools and tell the agent: “Monitor these competitor sites for price changes above 5% and alert me with a summary.” The model figures out which sites to check, how to handle problems, and when to send alerts all on its own.

Why Combine AWS Strands SDK with an MCP Server for Web Data Retrieval

AI agents built with AWS Strands inherit the limitations of their underlying LLMs – particularly the lack of access to real-time information. This can lead to outdated or inaccurate responses when agents need current data like competitor prices, market conditions, or customer sentiment.

This is where Bright Data’s Web MCP server comes in. Built on Node.js, this MCP server integrates with Bright Data’s suite of AI-ready data retrieval tools. These tools empower your agent to:

• Access any website content, even those with anti-bot protection
• Query structured datasets from 120+ popular sites
• Search across multiple search engines simultaneously
• Interact with dynamic web pages in real-time

As of now, the MCP server include 40 specialized tools for collecting structured data from sites like Amazon, LinkedIn, TikTok, and more using Web Scraper APIs.

Now, let’s see how you can use these MCP tools with AWS Strands SDK!

How to Integrate AWS Strands SDK with Bright Data MCP Server in Python

In this section, you’ll learn how to use AWS Strands SDK to build an AI agent equipped with live data scraping and retrieval capabilities from the Web MCP server.

As an example, we’ll build a competitive intelligence agent that can autonomously analyze markets and competitors. The agent will decide which tools to use based on its goals, demonstrating the power of the agentic loop.

Follow this step-by-step guide to build your Claude + Bright Data MCP-powered AI agent using AWS Strands SDK!

Prerequisites

To replicate the code example, make sure you have:

Software Requirements:

• Python 3.10 or higher
• Node.js (latest LTS version recommended)
• A Python IDE (VS Code with Python extension or PyCharm)

Account Requirements:

• A Bright Data account (free tier offers 5,000 monthly requests)
• An Anthropic account with Claude API access and credits

Background Knowledge (helpful but not required):

• Basic understanding of how MCP works
• Familiarity with AI agents and their capabilities
• Basic knowledge of asynchronous programming in Python

Step #1: Create Your Python Project

Open your terminal and create a new folder for your project:

mkdir strands-mcp-agent
cd strands-mcp-agent

Set up a Python virtual environment:

python -m venv venv

Activate the virtual environment:

# On Linux/macOS:
source venv/bin/activate

# On Windows:
venv\Scripts\activate

Create the main Python file:

touch agent.py

Your folder structure should look like this:

strands-mcp-agent/
├── venv/
└── agent.py

You’re all set! You now have a Python environment ready to build an AI agent with web data access.

Step #2: Install AWS Strands SDK

In your activated virtual environment, install the required packages:

pip install strands-agents python-dotenv

This installs:

• strands-agents: The AWS Strands SDK for building AI agents
• python-dotenv: For secure environment variable management

Next, add these imports to your agent.py file:

from strands import Agent
from strands.models.anthropic import AnthropicModel
from strands.tools.mcp.mcp_client import MCPClient
from mcp.client.stdio import stdio_client, StdioServerParameters

Cool! You can now use AWS Strands SDK for agent building.

Step #3: Set Up Environment Variables

Create a .env file in your project folder for secure API key management:

touch .env

Add your API keys to the .env file:

# Anthropic API for Claude models
ANTHROPIC_API_KEY=your_anthropic_key_here

# Bright Data credentials for web scraping
BRIGHT_DATA_API_KEY=your_bright_data_token_here

In your agent.py, set up environment variable loading:

import os
from dotenv import load_dotenv

load_dotenv()

# Read API keys
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY")
BRIGHT_DATA_API_KEY = os.getenv("BRIGHT_DATA_API_KEY")

Here we go! You’re now set up to securely load API keys from the .env file.

Step #4: Install and Test the Bright Data MCP Server

Install the Bright Data Web MCP globally via npm:

npm install -g @brightdata/mcp

Test that it works with your API key:

# On Linux/macOS:
API_TOKEN="<YOUR_BRIGHT_DATA_API>" npx -y @brightdata/mcp

# On Windows PowerShell:
$env:API_TOKEN="<YOUR_BRIGHT_DATA_API>"; npx -y @brightdata/mcp

If successful, you’ll see logs showing the MCP server starting up. The first run will automatically create two zones in your Bright Data account:

• mcp_unlocker: For Web Unlocker
• mcp_browser: For Browser API

You can verify these in your Bright Data dashboard under “Proxies & Scraping Infrastructure“.

Terrific! The Web MCP server works like a charm.

Step #5: Initialize the Strands Model

Configure the Anthropic Claude model in your agent.py:

# Initialize Anthropic model
model = AnthropicModel(
    model_id="claude-3-opus-20240229",  # You can also use claude-3-sonnet for lower cost
    max_tokens=4096,
    params={"temperature": 0.3}
)

# Set the API key
os.environ["ANTHROPIC_API_KEY"] = ANTHROPIC_API_KEY

This configures Claude as your agent’s LLM with appropriate parameters for consistent, focused responses.

Step #6: Connect to the Web MCP Server

Create the MCP client configuration to connect to Bright Data’s tools:

import asyncio
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

async def connect_mcp_tools():
    """Connect to Bright Data MCP server and discover tools"""

    logger.info("Connecting to Bright Data MCP...")

    # Configure connection to Bright Data hosted MCP
    server_params = StdioServerParameters(
        command="npx",
        args=["-y", "@brightdata/mcp"],
        env={"API_TOKEN": BRIGHT_DATA_API_KEY, "PRO_MODE": "true"}
    )

    # Create MCP client
    mcp_client = MCPClient(lambda: stdio_client(server_params))

    # Discover available tools
    with mcp_client:
        tools = mcp_client.list_tools_sync()
        logger.info(f"📦 Discovered {len(tools)} MCP tools")

        for tool in tools:
            logger.info(f"  - {tool.tool_name}")

    return mcp_client, tools

This establishes the connection to Bright Data’s MCP server and discovers all available web scraping tools.

Step #7: Define the Competitive Intelligence Agent

Create an agent with a specialized prompt for competitive intelligence:

def create_agent(model, tools):
    """Create a competitive intelligence agent with web data access"""

    system_prompt = """You are an expert competitive intelligence analyst with access to powerful web data tools through MCP.

## Your Mission
Conduct comprehensive market and competitive analysis using real-time web data.

## Available MCP Tools
You have access to these Bright Data MCP tools:
- search_engine: Scrape search results from Google, Bing or Yandex
- scrape_as_markdown: Extract content from any webpage with CAPTCHA bypass
- search_engine_batch: Run multiple searches simultaneously
- scrape_batch: Scrape multiple webpages in parallel

## Autonomous Analysis Workflow
When given an analysis task, autonomously:
1. Decide which tools to use based on the goal
2. Gather comprehensive data from multiple sources
3. Synthesize findings into actionable insights
4. Provide specific strategic recommendations

Be proactive in tool selection - you have full autonomy to use any combination of tools."""

    return Agent(
        model=model,
        tools=tools,
        system_prompt=system_prompt
    )

This creates an agent specialized in competitive intelligence with autonomous decision-making capabilities.

Step #8: Launch Your Agent

Create the main execution function to run your agent:

async def main():
    """Run the competitive intelligence agent"""

    print("🚀 AWS Strands + Bright Data MCP Competitive Intelligence Agent")
    print("=" * 70)

    try:
        # Connect to MCP tools
        mcp_client, tools = await connect_mcp_tools()

        # Create the agent
        agent = create_agent(model, tools)

        print("\n✅ Agent ready with web data access!")
        print("\n📊 Starting Analysis...")
        print("-" * 40)

        # Example: Analyze Tesla's competitive position
        prompt = """
        Analyze Tesla's competitive position in the electric vehicle market.

        Research:
        - Current product lineup and pricing strategy
        - Main competitors and their offerings
        - Recent strategic announcements
        - Market share and positioning

        Use web scraping tools to gather real-time data from tesla.com and search results.
        """

        # Run analysis with MCP context
        with mcp_client:
            result = await agent.invoke_async(prompt)

        print("\n📈 Analysis Results:")
        print("=" * 50)
        print(result.content)

        print("\n✅ Analysis complete!")

    except Exception as e:
        logger.error(f"Error: {e}")
        print(f"\n❌ Error: {e}")

if __name__ == "__main__":
    asyncio.run(main())

Mission accomplished! Your agent is ready to perform autonomous competitive analysis.

Step #9: Put It All Together

Here’s the complete code in agent.py:

import asyncio
import os
import logging
from dotenv import load_dotenv

from strands import Agent
from strands.models.anthropic import AnthropicModel
from strands.tools.mcp.mcp_client import MCPClient
from mcp.client.stdio import stdio_client, StdioServerParameters

# Load environment variables
load_dotenv()
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Read API keys
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY")
BRIGHT_DATA_API_KEY = os.getenv("BRIGHT_DATA_API_KEY")

# Initialize Anthropic model
model = AnthropicModel(
    model_id="claude-3-opus-20240229",
    max_tokens=4096,
    params={"temperature": 0.3}
)

# Set API key
os.environ["ANTHROPIC_API_KEY"] = ANTHROPIC_API_KEY

async def connect_mcp_tools():
    """Connect to Bright Data MCP server and discover tools"""

    logger.info("Connecting to Bright Data MCP...")

    # Configure connection to Bright Data hosted MCP
    server_params = StdioServerParameters(
        command="npx",
        args=["-y", "@brightdata/mcp"],
        env={"API_TOKEN": BRIGHT_DATA_API_KEY, "PRO_MODE": "true"}
    )

    # Create MCP client
    mcp_client = MCPClient(lambda: stdio_client(server_params))

    # Discover available tools
    with mcp_client:
        tools = mcp_client.list_tools_sync()
        logger.info(f"📦 Discovered {len(tools)} MCP tools")

        for tool in tools:
            logger.info(f"  - {tool.tool_name}")

    return mcp_client, tools

def create_agent(model, tools):
    """Create a competitive intelligence agent with web data access"""

    system_prompt = """You are an expert competitive intelligence analyst with access to powerful web data tools through MCP.

## Your Mission
Conduct comprehensive market and competitive analysis using real-time web data.

## Available MCP Tools
You have access to these Bright Data MCP tools:
- search_engine: Scrape search results from Google, Bing or Yandex
- scrape_as_markdown: Extract content from any webpage with CAPTCHA bypass
- search_engine_batch: Run multiple searches simultaneously
- scrape_batch: Scrape multiple webpages in parallel

## Autonomous Analysis Workflow
When given an analysis task, autonomously:
1. Decide which tools to use based on the goal
2. Gather comprehensive data from multiple sources
3. Synthesize findings into actionable insights
4. Provide specific strategic recommendations

Be proactive in tool selection - you have full autonomy to use any combination of tools."""

    return Agent(
        model=model,
        tools=tools,
        system_prompt=system_prompt
    )

async def main():
    """Run the competitive intelligence agent"""

    print("🚀 AWS Strands + Bright Data MCP Competitive Intelligence Agent")
    print("=" * 70)

    try:
        # Connect to MCP tools
        mcp_client, tools = await connect_mcp_tools()

        # Create the agent
        agent = create_agent(model, tools)

        print("\n✅ Agent ready with web data access!")
        print("\n📊 Starting Analysis...")
        print("-" * 40)

        # Example: Analyze Tesla's competitive position
        prompt = """
        Analyze Tesla's competitive position in the electric vehicle market.

        Research:
        - Current product lineup and pricing strategy
        - Main competitors and their offerings
        - Recent strategic announcements
        - Market share and positioning

        Use web scraping tools to gather real-time data from tesla.com and search results.
        """

        # Run analysis with MCP context
        with mcp_client:
            result = await agent.invoke_async(prompt)

        print("\n📈 Analysis Results:")
        print("=" * 50)
        print(result)

        print("\n✅ Analysis complete!")

    except Exception as e:
        logger.error(f"Error: {e}")
        print(f"\n❌ Error: {e}")

if __name__ == "__main__":
    asyncio.run(main())

Execute the AI agent with:

python agent.py

In the terminal, you should see output showing:

1. The MCP connection establishing
2. Discovery of available Bright Data tools
3. The agent autonomously selecting tools to use
4. Real-time data being gathered from Tesla.com and search results
5. A comprehensive competitive analysis with current data

The agent autonomously decides to:

• Use search_engine to find information about Tesla and competitors
• Use scrape_as_markdown to extract data from tesla.com
• Combine multiple data sources for comprehensive analysis

Et voilà! You’ve successfully built an autonomous competitive intelligence agent that can access and analyze real-time web data.

Next Steps

The AI agent built here is functional, but it serves only as a starting point. Consider taking it to the next level by:

• Building a conversation loop: Add a REPL interface to chat with the agent interactively
• Creating specialized agents: Build agents for price monitoring, market research, or lead generation
• Implementing multi-agent workflows: Coordinate multiple specialized agents for complex tasks
• Adding memory and state: Use AWS Strands’ state management for context-aware conversations
• Deploying to production: Leverage AWS infrastructure for scalable agent deployment
• Extending with custom tools: Create your own MCP tools for specialized data sources
• Adding observability: Implement logging and monitoring for production deployments

Real-World Use Cases

The combination of AWS Strands and Bright Data enables more advanced AI agents across diverse business applications:

• Competitive Intelligence Agent: Monitor competitor pricing, product features, and marketing campaigns in real-time
• Market Research Agent: Analyze industry trends, consumer sentiment, and emerging opportunities
• E-Commerce Optimization Agent: Track competitor catalogs and pricing for dynamic pricing strategies
• Lead Generation Agent: Identify and qualify potential customers from web sources
• Brand Monitoring Agent: Track brand mentions, reviews, and reputation across the web
• Investment Research Agent: Gather financial data, news, and market signals for investment decisions

Conclusion

In this article, you learned how to integrate AWS Strands SDK with Bright Data’s Web MCP server to build autonomous AI agents capable of accessing and analyzing live web data. This powerful combination enables you to create agents that can think strategically while staying informed with real-time information.

The key advantages of this approach include:

• Minimal code: Build sophisticated agents with just ~100 lines of Python
• Autonomous decision-making: Agents decide which tools to use based on their goals
• Production-ready: Built-in error handling and scalability from both platforms
• Real-time data access: Overcome LLM limitations with live web data

To build more sophisticated agents, explore the full range of services available in the Bright Data AI infrastructure. These solutions can power a wide variety of agentic scenarios.

Create a Bright Data account for free and start experimenting with AI-powered web data tools today!