Beautifulsoup proxy integration

Beautifulsoup Proxy Integration

Beautifulsoup proxy integration with Bright Data

  • Begin by going to your Bright Data Dashboard and clicking ‘create a Zone’.
  • Choose ‘Network type’ and click save.
  • Within Beautifulsoup fill in the ‘Proxy IP:Port’ in the ‘proxy-server’ value, for example brd.superproxy.io:22225.
  • Under ‘page.authenticate’ input your Bright Data account ID and proxy Zone name in the ‘username’ value, for example: brd-customer-CUSTOMER-zone-YOURZONE and your Zone password found in the Zone settings.
  • For example:
      from bs4 import BeautifulSoup
import requests
import json

proxies = {
   'http': 'http://brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD]@brd.superproxy.io:22225',
   'https': 'http://brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD]@brd.superproxy.io:22225',
}

url = "http://lumtest.com/myip.json"

res = requests.get(url, proxies=proxies, verify=False)
res.raise_for_status()

res_body = res.text

soup = BeautifulSoup(res_body, "html.parser")

html_content = str(soup)

data = {
   "IP": html_content,
  
}

print(json.dumps(data, indent=4))
    

Proxy Manager and Beautifulsop integration

  • Begin by going to your Bright Data Dashboard and clicking ‘create a Zone’.
  • Choose ‘Network type’ and click save.
  • Within Beautifulsoup fill in the ‘Proxy IP:Port’ in the ‘proxy-server’ value, for example brd.superproxy.io:22225.
  • Under ‘page.authenticate’ input your Bright Data account ID and proxy Zone name in the ‘username’ value, for example: brd-customer-CUSTOMER-zone-YOURZONE and your Zone password found in the Zone settings.
  • For example:
      from bs4 import BeautifulSoup
import requests
import json

proxies = {
   'http': '127.0.0.1:24000',
   'https': '127.0.0.1:24000',
}

url = "http://lumtest.com/myip.json"

res = requests.get(url, proxies=proxies, verify=False)
res.raise_for_status()

res_body = res.text

soup = BeautifulSoup(res_body, "html.parser")

html_content = str(soup)

data = {
   "IP": html_content,
  
}

print(json.dumps(data, indent=4))
    

Get proxies for Beautifulsoup

Proxy badges

Powered by an award-winning proxy infrastructure

Over 72 million residential IPs, best-in-class technology and the ability to target any country, city, ZIP Code, carrier, & ASN make our premium proxy services a top choice for developers.

About Bright Data proxies

Residential Proxies

  • 72,000,000+ IPs
  • Available in 195 countries
  • The largest rotating real-peer IP network
  • Access & crawl all sophisticated websites

Datacenter Proxies

  • 770,000+ IPs
  • Available in 98 countries
  • Shared and dedicated IP pools available
  • Access & crawl all sophisticated websites

ISP Proxies

  • 700,000+ IPs
  • Available in 35 countries
  • Real static residential IPs without IP rotation
  • Best for logging into multiple accounts

Mobile Proxies

  • 7,000,000+ IPs
  • Available in 195 countries
  • Largest real-peer 3G/4G IP network in the world
  • Verify mobile ads & crawl mobile sites

The best customer experience in the industry

You ask, we develop

New feature releases every day

24/7 global support

To answer any questions right when you need it

Full transparency

Real-time network performance dashboard

Dedicated Account Managers

To optimize your performance

Tailored solutions

To meet your data collection goals

The category leader in proxies and data collection

Category leader in proxies

650TB of public data collected every day

Number of ISP proxies

New feature releases every day

Serving 7/10 universities

Serving 7/10 of the world’s leading universities

Trust pilot ratings
4.5/5 Trustpilot rating

Ready to get Beautifulsoup proxies?