Beautifulsoup Proxy Integration

Beautifulsoup proxy integration with Bright Data

Begin by going to your Bright Data Dashboard and clicking ‘create a Zone’.
Choose ‘Network type’ and click save.
Within Beautifulsoup fill in the ‘Proxy IP:Port’ in the ‘proxy-server’ value, for example brd.superproxy.io:22225.
Under ‘page.authenticate’ input your Bright Data account ID and proxy Zone name in the ‘username’ value, for example: brd-customer-CUSTOMER-zone-YOURZONE and your Zone password found in the Zone settings.
For example:

      from bs4 import BeautifulSoup
import requests
import json

proxies = {
   'http': 'http://brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD]@brd.superproxy.io:22225',
   'https': 'http://brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD]@brd.superproxy.io:22225',
}

url = "http://lumtest.com/myip.json"

res = requests.get(url, proxies=proxies, verify=False)
res.raise_for_status()

res_body = res.text

soup = BeautifulSoup(res_body, "html.parser")

html_content = str(soup)

data = {
   "IP": html_content,
  
}

print(json.dumps(data, indent=4))

Proxy Manager and Beautifulsop integration

Begin by going to your Bright Data Dashboard and clicking ‘create a Zone’.
Choose ‘Network type’ and click save.
Within Beautifulsoup fill in the ‘Proxy IP:Port’ in the ‘proxy-server’ value, for example brd.superproxy.io:22225.
Under ‘page.authenticate’ input your Bright Data account ID and proxy Zone name in the ‘username’ value, for example: brd-customer-CUSTOMER-zone-YOURZONE and your Zone password found in the Zone settings.
For example:

      from bs4 import BeautifulSoup
import requests
import json

proxies = {
   'http': '127.0.0.1:24000',
   'https': '127.0.0.1:24000',
}

url = "http://lumtest.com/myip.json"

res = requests.get(url, proxies=proxies, verify=False)
res.raise_for_status()

res_body = res.text

soup = BeautifulSoup(res_body, "html.parser")

html_content = str(soup)

data = {
   "IP": html_content,
  
}

print(json.dumps(data, indent=4))

Get proxies for Beautifulsoup

Start free trial

Powered by an award-winning proxy infrastructure

Over 72 million residential IPs, best-in-class technology and the ability to target any country, city, ZIP Code, carrier, & ASN make our premium proxy services a top choice for developers.