In this tutorial, you will see:
- The definition of a Google Maps scraper
- What data you can extract with it
- How to build a Google Maps scraping script with Python
Let’s dive in!
What Is a Google Maps Scraper?
A Google Maps scraper is a specialized tool to extract data from Google Maps. It automates the process of gathering Maps data, for example through a Python scraping script. The data retrieved with such a scraper is commonly used for market research, competitor analysis, and more.
What Data You Can Retrieve From Google Maps
Information you can scrape from Google Maps includes:
- Business name: The name of the business or location listed on Google Maps.
- Address: Physical street address of the business or location.
- Phone number: Contact phone number for the business.
- Website: URL to the business’s website.
- Business hours: Opening and closing times for the business.
- Reviews: Customer reviews, including ratings and detailed feedback.
- Ratings: Average star rating based on user feedback.
- Photos: Images uploaded by the business or customers.
Step-By-Step Instructions to Scrape Google Maps With Python
In this guided section, you will learn how to build a Python script to scrape Google Maps.
The end goal is to retrieve the data contained in Google Maps items on the “Italian restaurants” page:
Follow the steps below!
Step #1: Project Setup
Before getting started, you need to verify that Python 3 is installed on your machine. If it is not, download it, install it, and follow the installation wizard.
Then, use the following commands to create a folder for the project, enter it, and create a virtual environment within it:
mkdir google-maps-scraper
cd google-maps-scraper
python -m venv env
The google-maps-scraper
directory represents the project folder of the Python Google Maps scraper.
Load the project folder in your favorite Python IDE. PyCharm Community Edition or Visual Studio Code with the Python extension will do.
Inside the project’s folder, create a scraper.py
file. This is the file structure your project should have right now:scraper.py
is now a blank Python script, but it will soon contain the scraping logic.
In the IDE’s terminal, activate the virtual environment. To so do, in Linux or macOS, fire this command:
./env/bin/activate
Alternatively, on Windows, execute:
env/Scripts/activate
Wonderful, you now have a Python environment for your scraper!
Step #2: Choose the Scraping Library
Google Maps is a highly interactive platform, and there is no point in spending time determining whether it is a static or dynamic site. In cases like this, the best approach for scraping is to use a browser automation tool.
If you are not familiar with that technology, browser automation tools let you render and interact with web pages in a controllable browser environment. In addition, creating a valid Google Maps search URL is not easy. The simplest way to handle this is by performing the search directly in a browser.
One of the best browser automation tools for Python is Selenium, making it an ideal choice for scraping Google Maps. Prepare to install it, as it will be the primary library used for this task!
Step #3: Install and Configure the Scraping Library
Install Selenium through the selenium
pip package with this command in an activated Python virtual environment:
pip install selenium
For more details on how to use this tool, follow our tutorial on web scraping with Selenium.
Import Selenium in scraper.py
and create a WebDriver
object to control a Chrome instance in headless mode:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
# to launch Chrome in headless mode
options = Options()
options.add_argument("--headless") # comment it while developing
# create a Chrome web driver instance with the
# specified options
driver = webdriver.Chrome(
service=Service(),
options=options
)
The snippet above initializes a Chrome WebDriver
instance to control a Chrome browser window programmatically. The --headless
flag is for starting Chrome in headless mode, which launches it in the background—without loading its window. For debugging, you can comment out this line to watch the script’s actions in real time.
As the last line of your Google Maps scraping script, do not forget to close the web driver:
driver.quit()
Amazing! You are now fully configured to start scraping Google Maps pages.
Step #4: Connect to the Target Page
Use the get()
method to connect to the Google Maps home page:
driver.get("https://www.google.com/maps")
Right now, scraper.py
will contain these lines:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
# to launch Chrome in headless mode
options = Options()
options.add_argument("--headless") # comment it while developing
# create a Chrome web driver instance with the
# specified options
driver = webdriver.Chrome(
service=Service(),
options=options
)
# connect to the Google Maps home page
driver.get("https://www.google.com/maps")
# scraping logic...
# close the web browser
driver.quit()
Awesome, it is time to start scraping a dynamic website like Maps!
Step #5: Deal With the GDPR Cookie Dialog
Note: If you are not located in the EU (European Union), you can skip this step.
Run the scraper.py
script in headed mode, adding a breakpoint before the last line, if possible. This will keep the browser window open, enabling you to observe it without it closing immediately. If you are in the EU, you should see something like this:
Note: The “Chrome is being controlled by automated test software.” message confirms that Selenium is successfully controlling Chrome.
Google has to show some cookie policy options to EU users due to GDPR requirements. If this applies to you, you will need to handle this choice if you want to interact with the page. If that is not the case, you can skip to Step 6.
Take a look at the URL in the browser’s address bar, and you will notice it does not match the page specified in get()
. The reason is that Google redirected you. After clicking the “Accept All” button, you will be taken back to the target page—which is the Google Maps home page.
To handle the GDPR options, open the Google Maps home page in incognito mode on your browser and wait for the redirect. Right-click on the “Accept All” button and select the “Inspect“ option:
As you may have noticed, the CSS classes of the HTML elements on the page appear to be randomly generated. This means they are unreliable for web scraping, as they are likely updated with each deployment. So, you need to focus on targeting more stable attributes, like aria-label
:
accept_button = driver.find_element(By.CSS_SELECTOR, "[aria-label=\"Accept all\"]")
find_element()
is a method in Selenium used to locate HTML elements on a page using various strategies. In this case, we used a CSS selector. If you would like to learn more about different selector types, read our XPath vs CSS selector.
Do not forget to import By
by adding this import to scraper.py
:
from selenium.webdriver.common.by import By
The next instruction is to click the button:
accept_button.click()
Here is how everything fits together to handle the optional Google cookie page:
try:
# select the "Accept all" button from the GDPR cookie option page
accept_button = driver.find_element(By.CSS_SELECTOR, "[aria-label=\"Accept all\"]")
# click it
accept_button.click()
except NoSuchElementException:
print("No GDPR requirenments")
The click()
command presses the “Accept all” button, letting Google redirect you to the Maps home page. If you are not located in the EU, this button will not be on the page—which will cause a NoSuchElementException
. The script will catch the exception and proceed, as it is not a critical error, just a potential scenario.
Make sure to import NoSuchElementException
:
from selenium.common import NoSuchElementException
Well done! You’re now ready to focus on scraping Google Maps.
Step #6: Submit the Search Form
Right now, your Google Maps scraper should reach a page like the one below:
Note that the location on the maps depends on the location of your IP. In this example, we are located in New York.
Next, you want to fill in the “Search Google Maps” field and submit the search form. To locate this element, open the Google Maps homepage in incognito mode on your browser. Right-click on the search input field and choose “Inspect” option:
search_input = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CSS_SELECTOR, "#searchboxinput"))
)
WebDriverWait
is a specialized Selenium class that pauses the script until a specified condition on the page is met. In the example above, it waits up to 5 seconds for the input HTML element to appear. This wait ensures that the page has fully loaded, which is required if you followed Step 5 (because of the redirect).
To make the above lines work, add these imports to scraper.py
:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Next, fill out the input with the `[send_keys()](https://www.selenium.dev/documentation/webdriver/actions_api/keyboard/#send-keys)` method:
search_query = "italian restaurants"
search_input.send_keys(search_query)
In this case, “italian restaurants” is the search query, but you can search for any other term.
It only remains to submit the form. Inspect the submit button, which is a magnifying glass:
Select it by targeting its aria-label
attribute and click it:
search_button = driver.find_element(By.CSS_SELECTOR, "button[aria-label=\"Search\"]")
search_button.click()
Incredible! The controlled browser will now load the data to scrape.
Step #7: Select the Google Maps Items
This is where your script should currently be:
The data to scrape is contained in the Google Maps items on the left. Since that is a list, the best data structure to contain the scraped data is an array. Initialize one:
items = []
The goal is now to select the Google Maps item on the left. Inspect one of them:
Once again, the CSS classes appear to be randomly generated, so they are not reliable for scraping. Instead, you can target the jsaction
attribute. Since parts of this attribute’s content also appear randomly generated, focus on the consistent string within it—specifically, "mouseover:pane"
.
The XPath selector below will help you select all <div>
elements within the parent <div>
where role="feed"
, whose jsaction
attribute contains the "mouseover:pane"
string:
maps_items = WebDriverWait(driver, 10).until(
EC.presence_of_all_elements_located((By.XPATH, '//div[@role="feed"]//div[contains(@jsaction, "mouseover:pane")]'))
)
Again, WebDriverWait
is required as the content on the left is loaded dynamically into the page.
Iterate over each element and prepare your Google Maps scraper to extract some data:
for maps_item in maps_items:
# scraping logic...
Terrific! The next step is to extract data from those elements.
Step #8: Scrape the Google Maps Items
Inspect a single Google Maps item and focus on the elements inside it:
Here, you can see that you can scrape:
- The link of the Maps item from the
a[jsaction][jslog]
element - The title from the
div.fontHeadlineSmall
element - The stars and number of reviews from
span[role="img"]
You can achieve that with the logic below:
link_element = maps_item.find_element(By.CSS_SELECTOR, "a[jsaction][jslog]")
url = link_element.get_attribute("href")
title_element = maps_item.find_element(By.CSS_SELECTOR, "div.fontHeadlineSmall")
title = title_element.text
reviews_element = maps_item.find_element(By.CSS_SELECTOR, "span[role=\"img\"]")
reviews_string = reviews_element.get_attribute("aria-label")
# define a regular expression pattern to extract the stars and reviews count
reviews_string_pattern = r"(\d+\.\d+) stars (\d+[,]*\d+) Reviews"
# use re.match to find the matching groups
reviews_string_match = re.match(reviews_string_pattern, reviews_string)
reviews_stars = None
reviews_count = None
# if a match is found, extract the data
if reviews_string_match:
# convert stars to float
reviews_stars = float(reviews_string_match.group(1))
# convert reviews count to integer
reviews_count = int(reviews_string_match.group(2).replace(",", ""))
The get_attribute()
function returns the content inside the specified HTML attribute, while .text
returns the string content inside the node.
Note the use of a regular expression to extract the specific data fields from the “X.Y stars in Z reviews” string. Find out more in our article on using regex for web scraping.
Do not forget to import the re
package from the Python standard library:
import re
Continue to inspect the Google Maps item:
Inside the <div>
with the fondBodyMedium
class, you can get most of the information from <span>
nodes with no attributes or only the style
attribute. As for the optional pricing element, you can select it by targeting the node that contains “Price” in the aria-label
attribute:
info_div = maps_item.find_element(By.CSS_SELECTOR, ".fontBodyMedium")
# scrape the price, if present
try:
price_element = info_div.find_element(By.XPATH, ".//*[@aria-label[contains(., 'Price')]]")
price = price_element.text
except NoSuchElementException:
price = None
info = []
# select all <span> elements with no attributes or the @style attribute
# and descendant of a <span>
span_elements = info_div.find_elements(By.XPATH, ".//span[not(@*) or @style][not(descendant::span)]")
for span_element in span_elements:
info.append(span_element.text.replace("⋅", "").strip())
# to remove any duplicate info and empty strings
info = list(filter(None, list(set(info))))
Since the price element is optional, you must wrap that logic with a try ... except
block. This way, if the price node is not on the page, the script will continue withouth failing. If you skip Step 5, add the import for NoSuchElementException
:
from selenium.common import NoSuchElementException
To avoid empty strings and duplicate info elements, note the use of filter()
and set()
.
Now, focus on the image:
You can scrape it with:
img_element = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CSS_SELECTOR, "img[decoding=\"async\"][aria-hidden=\"true\"]"))
)
image = img_element.get_attribute("src")
Keep in mind that WebDriverWait
is required, as images are loaded asynchronously and might take a while to appear.
The last step is to scrape the tags in the element on the bottom:
You can retrieve them all from the <span>
nodes with the style
attribute in the last .fontBodyMedium
element:
tags_div = maps_item.find_elements(By.CSS_SELECTOR, ".fontBodyMedium")[-1]
tags = []
tag_elements = tags_div.find_elements(By.CSS_SELECTOR, "span[style]")
for tag_element in tag_elements:
tags.append(tag_element.text)
Tremendous! The Python Google Maps scraping logic is complete.
Step #9: Collect the Scraped Data
You now have the scraped data in several variables. Create a new item
object and populate it with that data:
item = {
"url": url,
"image": image,
"title": title,
"reviews": {
"stars": reviews_stars,
"count": reviews_count
},
"price": price,
"info": info,
"tags": tags
}
Then, append it to the items
array:
items.append(item)
At the end of the for
loop on Google Maps item nodes, items
will contain all your scraping data. You only have to export that information to a human-readable file like CSV.
Step #10: Export to CSV
Import the csv
package from the Python standard library:
import csv
Next, use it to populate a flat CSV file with your Google Maps data:
# output CSV file path
output_file = "items.csv"
# flatten and export to CSV
with open(output_file, mode="w", newline="", encoding="utf-8") as csv_file:
# define the CSV field names
fieldnames = ["url", "image", "title", "reviews_stars", "reviews_count", "price", "info", "tags"]
writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
# write the header
writer.writeheader()
# write each item, flattening info and tags
for item in items:
writer.writerow({
"url": item["url"],
"image": item["image"],
"title": item["title"],
"reviews_stars": item["reviews"]["stars"],
"reviews_count": item["reviews"]["count"],
"price": item["price"],
"info": "; ".join(item["info"]),
"tags": "; ".join(item["tags"])
})
The above snippet exports items
to a CSV file named items.csv
. Key functions used are:
open()
: Open the specified file in write mode with UTF-8 encoding to handle text output.csv.DictWriter()
: Create a CSV writer object using specified field names, allowing rows to be written as dictionaries.writeheader()
: Write the header row to the CSV file based on the field names.writer.writerow()
: Write each item as a row in the CSV.
Note the use of the join()
string function to transform the arrays to flat strings. This ensures that the output CSV will be a clean, single-level file.
Step #11: Put It All Together
Here is the final code of the Python Google Maps scraper:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.common import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import re
import csv
# to launch Chrome in headless mode
options = Options()
options.add_argument("--headless") # comment it while developing
# create a Chrome web driver instance with the
# specified options
driver = webdriver.Chrome(
service=Service(),
options=options
)
# connect to the Google Maps home page
driver.get("https://www.google.com/maps")
# to deal with the option GDPR options
try:
# select the "Accept all" button from the GDPR cookie option page
accept_button = driver.find_element(By.CSS_SELECTOR, "[aria-label=\"Accept all\"]")
# click it
accept_button.click()
except NoSuchElementException:
print("No GDPR requirenments")
# select the search input and fill it in
search_input = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CSS_SELECTOR, "#searchboxinput"))
)
search_query = "italian restaurants"
search_input.send_keys(search_query)
# submit the search form
search_button = driver.find_element(By.CSS_SELECTOR, "button[aria-label=\"Search\"]")
search_button.click()
# where to store the scraped data
items = []
# select the Google Maps items
maps_items = WebDriverWait(driver, 10).until(
EC.presence_of_all_elements_located((By.XPATH, '//div[@role="feed"]//div[contains(@jsaction, "mouseover:pane")]'))
)
# iterate over the Google Maps items and
# perform the scraping logic
for maps_item in maps_items:
link_element = maps_item.find_element(By.CSS_SELECTOR, "a[jsaction][jslog]")
url = link_element.get_attribute("href")
title_element = maps_item.find_element(By.CSS_SELECTOR, "div.fontHeadlineSmall")
title = title_element.text
reviews_element = maps_item.find_element(By.CSS_SELECTOR, "span[role=\"img\"]")
reviews_string = reviews_element.get_attribute("aria-label")
# define a regular expression pattern to extract the stars and reviews count
reviews_string_pattern = r"(\d+\.\d+) stars (\d+[,]*\d+) Reviews"
# use re.match to find the matching groups
reviews_string_match = re.match(reviews_string_pattern, reviews_string)
reviews_stars = None
reviews_count = None
# if a match is found, extract the data
if reviews_string_match:
# convert stars to float
reviews_stars = float(reviews_string_match.group(1))
# convert reviews count to integer
reviews_count = int(reviews_string_match.group(2).replace(",", ""))
# select the Google Maps item <div> with most info
# and extract data from it
info_div = maps_item.find_element(By.CSS_SELECTOR, ".fontBodyMedium")
# scrape the price, if present
try:
price_element = info_div.find_element(By.XPATH, ".//*[@aria-label[contains(., 'Price')]]")
price = price_element.text
except NoSuchElementException:
price = None
info = []
# select all <span> elements with no attributes or the @style attribute
# and descendant of a <span>
span_elements = info_div.find_elements(By.XPATH, ".//span[not(@*) or @style][not(descendant::span)]")
for span_element in span_elements:
info.append(span_element.text.replace("⋅", "").strip())
# to remove any duplicate info and empty strings
info = list(filter(None, list(set(info))))
img_element = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CSS_SELECTOR, "img[decoding=\"async\"][aria-hidden=\"true\"]"))
)
image = img_element.get_attribute("src")
# select the tag <div> element and extract data from it
tags_div = maps_item.find_elements(By.CSS_SELECTOR, ".fontBodyMedium")[-1]
tags = []
tag_elements = tags_div.find_elements(By.CSS_SELECTOR, "span[style]")
for tag_element in tag_elements:
tags.append(tag_element.text)
# populate a new item with the scraped data
item = {
"url": url,
"image": image,
"title": title,
"reviews": {
"stars": reviews_stars,
"count": reviews_count
},
"price": price,
"info": info,
"tags": tags
}
# add it to the list of scraped data
items.append(item)
# output CSV file path
output_file = "items.csv"
# flatten and export to CSV
with open(output_file, mode="w", newline="", encoding="utf-8") as csv_file:
# define the CSV field names
fieldnames = ["url", "image", "title", "reviews_stars", "reviews_count", "price", "info", "tags"]
writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
# write the header
writer.writeheader()
# write each item, flattening info and tags
for item in items:
writer.writerow({
"url": item["url"],
"image": item["image"],
"title": item["title"],
"reviews_stars": item["reviews"]["stars"],
"reviews_count": item["reviews"]["count"],
"price": item["price"],
"info": "; ".join(item["info"]),
"tags": "; ".join(item["tags"])
})
# close the web browser
driver.quit()
In around 150 lines of code, you just built a Google Maps scraping script!
Verify that it works by launching the scraper.py
file. On Windows, run the scraper with:
python scraper.py
Equivalently, on Linux or macOS, run:
python3 scraper.py
Wait for the scraper to finish running, and an items.csv
file will appear in your project’s root directory. Open the file to view the extracted data, which should contain data like this:
Congrats, mission complete!
Conclusion
In this tutorial, you learned what a Google Maps scraper is and how to build one in Python. As you saw, building a simple script to automatically retrieve data from Google Maps takes just a few lines of Python code.
While the solution works for small projects, it’s not practical for large-scale scraping. Google has advanced anti-bot measures, like CAPTCHAs and IP bans, that can block you. Scaling the process across multiple pages would also increase infrastructure costs. Moreover, this simple example does not account for all the complex interactions needed on Google Maps pages.
Does this mean that scraping Google Maps efficiently and reliably is impossible? Not at all! You just need an advanced solution like Bright Data’s Goolge Maps Scraper API.
Google Maps Scraper API offers endpoints to retrieve data from Google Maps, forgetting about all the main challenges. With simple API calls, you can get the data you need in JSON or HTML format. If API calls are nott your thing, you can also explore our ready-to-use Google Maps datasets.
Create a free Bright Data account today to try our scraper APIs or explore our datasets!
No credit card required