As one of Google’s most popular services, Google Maps offers immense opportunities for businesses to connect with customers and expand their research. With an extensive database of over 200 million places and businesses, Google Maps provides a rich source of data that can be leveraged for various purposes.
To access the data available on the Google Maps Platform, Google provides the Google Maps API. However, this API can be restrictive and pricey. In this article, you’ll learn how to scrape Google Maps without relying on the Google Maps API, using the Bright Data SERP API.
How to Use the Bright Data SERP API for Scraping Google Maps Data
Getting started with the Bright Data SERP API is simple. In this section, you’ll learn how to set up a new Bright Data account, sign up for the SERP API, and integrate it into your Node.js application.
Signing Up for the SERP API
To get started, head over to the SERP API home page and click on the Start free trial button:
Next, sign up for a free Bright Data account with your details (or using Google) on the dialog that opens up:
Once you’re done with the registration, you are redirected to your Bright Data dashboard. Click on the View proxy products button under the Proxies & Scraping Infrastructure section:
On the Proxies & Scraping Infrastructure page, click on the Get Started button on the SERP API card:
On the next page, you are shown the details of the SERP API, with an estimated monthly cost that you’ll be charged. Your trial account should have enough credits to cover your initial requests.
On this page, choose a name for your API instance (something like “maps” works fine). For the Ads mode question, leave it set to Off. The Ads mode option comes in handy when you want to include or exclude ads from the Google Search result pages.
Additionally, leave Asynchronous request off for the sake of this tutorial. When implementing the API in your app, you could consider turning it on if your system supports async processing. It can help improve your system’s efficiency, especially when working with large data objects.
Once you’re done, click on the Add button:
At this point, you’ve created a new instance of the SERP API. Now, you can see the credentials that you can use to access the API. The portal also provides you with a sample curl request that you can copy and use in your terminal to test the new set of credentials right away:
This completes the setup on the Bright Data platform. Make sure to note down the proxy host, username, and password provided on this screen, as you need them to access the SERP API from your Node.js/Python apps.
Using the SERP API in Node.js
To use the SERP API in Node.js, start by creating a new directory and initializing a new Node.js project in it:
mkdir serp-node
cd serp-node
npm init
Choose the default values for creating the project. Once the project is created, create a new file, index.js, to store your Node.js script in:
touch index.js
Before saving the script, install the request-promise module to help make HTTP requests easily by running the following command:
npm install request-promise
Store the following code in it (make sure to update the proxy details with your SERP API instance values):
const request = require('request-promise');
// Define parameters to pass into the API request
const keyword = process.argv[2]
const username = 'xxxx'; // your proxy username
const password = 'xxxx'; // your proxy password
const port = 0000; // your proxy port
const host = 'xxxx' // your proxy host
const session_id = (1000000 * Math.random())|0;
const super_proxy = 'http://'+username+'-session-'+session_id+':'+password+'@'+ host + ':'+port;
const options = {
url: 'https://www.google.com/maps/search/' + keyword.replace(' ', '+') + '/?gl=us&lum_json=1',
proxy: super_proxy,
rejectUnauthorized: false
};
// Send request to the Maps search API
request(options)
.then(function(data){
// Parse the response as JSON
const result = JSON.parse(data)
// Print the response JSON
console.log("Printing Maps search results...")
console.log(result)
// Extract the f_id of the first search result
const f_id = result.organic[0].fid
// Define API request parameters for retrieving reviews for the search result
var options = {
url: 'https://www.google.com/reviews?fid=' + f_id + '&lum_json=1',
proxy: super_proxy,
rejectUnauthorized: false
};
// Send request to the Reviews API
request(options)
.then(function(data){
console.log("Printing Google Reviews results...")
// Parse and print the response
console.log(JSON.stringify(JSON.parse(data), null, 2))
},
function(err){ console.error(err); })
},
function(err){ console.error(err); });
The script takes in the search keywords as command line arguments. After saving the file, run it using the following command:
node index.js "hotels california"
Here’s what the output looks like:
Printing Maps search results...
{
general: {
query: 'hotels california',
language: 'en',
country: 'United States',
country_code: 'US',
gl: 'US'
},
organic: [
{
"title": "Freehand Los Angeles",
"display_link": "freehandhotels.com",
"link": "https://freehandhotels.com/los-angeles/?utm_source=google-my-business&utm_medium=organic&utm_campaign=hotel-los-angeles",
"address": "416 W 8th St, Los Angeles, CA 90014",
"phone": "+12136120021",
"category": [
{
"title": "4-star hotel"
},
{
"id": "hotel",
"title": "Hotel"
}
],
"tags": [
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 21,
"value_title": "Pool"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 22,
"value_title": "Free Wi-Fi"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 2,
"value_title": "Air-conditioned"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 10,
"value_title": "Breakfast"
}
],
"summary": "Hip hotel with a rooftop pool & bar",
"description": "Trendy dorms & rooms in a funky lodging with dining & breakfast, plus a rooftop pool & bar.",
"rating": 4.1,
"reviews_cnt": 2263,
"latitude": 34.044755099999996,
"longitude": -118.2564556,
"claimed": true,
"fid": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"map_id_encoded": "ChIJE80IjrXHwoARZo6wiQusTrc",
"map_id": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"map_link": "https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"original_image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w408-h272-k-no",
"image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
"thumbnail": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w128-h86-k-no",
"icon": "https://lh5.googleusercontent.com/-1FySCxGYcRE/AAAAAAAAAAI/AAAAAAAAAAA/cD1i5xM2CPs/s44-p-k-no-ns-nd/photo.jpg",
"image_url": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
"rank": 1
}
// 19 records omitted
],
ads: [
{
title: 'Hampton Inn & Suites Rohnert Park - Sonoma County',
address: '6248 Redwood Dr, Rohnert Park, CA 94928',
category: [Array],
tags: [Array],
summary: 'Low-key hotel with a pool & a gym',
description: 'Unpretentious hotel offering an outdoor pool & a fitness room, plus parking & breakfast.',
rating: 4.4,
reviews_cnt: 215,
latitude: 38.350170999999996,
longitude: -122.71469599999999,
claimed: true,
fid: '0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
map_id_encoded: 'ChIJ7WG1cglKhIARzOfAOsHEFR0',
map_id: '0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
map_link: 'https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
original_image: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w408-h272-k-no',
image: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w138-h92-k-no',
thumbnail: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w129-h86-k-no',
image_url: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w138-h92-k-no',
rank: 1
},
// 1 record omitted
]
}
Printing Google Reviews results...
{
"reviews": [
{
"reviewer": {
"profile_photo_url": "https://lh3.googleusercontent.com/a-/AD_cMMTCnapnhVKwImX7-W1lGGDw8-jYDbjZ7bi7EQcRjQ=s40-c-c0x00000000-cc-rp-mo-ba3-br100",
"display_name": "Leroy T"
},
"rating": "4/5",
"review_reply": "Thank you for your review, Leroy! It is great to see that you enjoyed your time with us. …",
"comment": "Overall it was a great stay, i had the deluxe king corner one. Which I suggest go to their website to book where you can select a corner unit.I rated 4 out of 5 because i feel that they can improve by having breakfast included, kettle/coffee maker, complimentary water, toothbrush toothpaste but I guess since I'm writing this review at least future clients will be aware.As far as safety, the elevator and 24/7 door by the elevator is card key access. So felt pretty safe.Costumer service was great smooth checkin and out even the housekeeper was nice.Facilities are nice the pool hours are in the pictures didn't use the gym so I can't speak about that.",
"photos": [
"https://lh5.googleusercontent.com/p/AF1QipPfJceU9nK8p0FhUDl0mIObGUC1AoEF2bZMDDs6=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipPCpi4ED4kk7FFvDxdh9HrsB60Pr01sUkL3FP_o=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipMjNO_OAG8QA74JszAAE2K6gicxbGEMAQbRXtwt=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipMjDjq7wP8iUbUiFeBNFHjzOFBr6aVUPDcfDSHe=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNcccwmp8K24QID0jU9PcyWq2QswyyK6oc6gf1d=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNCnCfWeAdQ7JUbZHzFL_myn_S9Uv6rIiq7e6bV=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNtFZMp7PZNr7e9OabzzVeGWzadGsgW7DUY2pcC=w100-h100-p-n-k-no"
],
"created": "5 months ago on Google",
"star_rating": "4/5"
},
// More records omitted
]
Understanding the Information You Retrieved
The script used in this example follows this structure:
- Retrieve the Google Maps search results for the given keywords
- Access and retrieve the reviews for the first search result for the given keyword
Here’s what a search result from the SERP API looks like:
{
"title": "Freehand Los Angeles",
"display_link": "freehandhotels.com",
"link": "https://freehandhotels.com/los-angeles/?utm_source=google-my-business&utm_medium=organic&utm_campaign=hotel-los-angeles",
"address": "416 W 8th St, Los Angeles, CA 90014",
"phone": "+12136120021",
"category": [
{
"title": "4-star hotel"
},
{
"id": "hotel",
"title": "Hotel"
}
],
"tags": [
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 21,
"value_title": "Pool"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 22,
"value_title": "Free Wi-Fi"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 2,
"value_title": "Air-conditioned"
},
{
"group_id": "hotel_amenities",
"group_title": "Hotel amenities",
"key_id": 10,
"value_title": "Breakfast"
}
],
"summary": "Hip hotel with a rooftop pool & bar",
"description": "Trendy dorms & rooms in a funky lodging with dining & breakfast, plus a rooftop pool & bar.",
"rating": 4.1,
"reviews_cnt": 2263,
"latitude": 34.044755099999996,
"longitude": -118.2564556,
"claimed": true,
"fid": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"map_id_encoded": "ChIJE80IjrXHwoARZo6wiQusTrc",
"map_id": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"map_link": "https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
"original_image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w408-h272-k-no",
"image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
"thumbnail": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w128-h86-k-no",
"icon": "https://lh5.googleusercontent.com/-1FySCxGYcRE/AAAAAAAAAAI/AAAAAAAAAAA/cD1i5xM2CPs/s44-p-k-no-ns-nd/photo.jpg",
"image_url": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
"rank": 1
}
As you can see, the API returns all basic information about a search result (ie a place), which includes its title, link, address, and contact details. Additionally, the API also returns other details, such as amenities, a review summary, rating, coordinates, images, and category details.
Here’s what a review retrieved using the SERP looked like:
{
"reviewer": {
"profile_photo_url": "https://lh3.googleusercontent.com/a-/AD_cMMTCnapnhVKwImX7-W1lGGDw8-jYDbjZ7bi7EQcRjQ=s40-c-c0x00000000-cc-rp-mo-ba3-br100",
"display_name": "Leroy T"
},
"rating": "4/5",
"review_reply": "Thank you for your review, Leroy! It is great to see that you enjoyed your time with us. …",
"comment": "Overall it was a great stay, i had the deluxe king corner one. Which I suggest go to their website to book where you can select a corner unit.I rated 4 out of 5 because i feel that they can improve by having breakfast included, kettle/coffee maker, complimentary water, toothbrush toothpaste but I guess since I'm writing this review at least future clients will be aware.As far as safety, the elevator and 24/7 door by the elevator is card key access. So felt pretty safe.Costumer service was great smooth checkin and out even the housekeeper was nice.Facilities are nice the pool hours are in the pictures didn't use the gym so I can't speak about that.",
"photos": [
"https://lh5.googleusercontent.com/p/AF1QipPfJceU9nK8p0FhUDl0mIObGUC1AoEF2bZMDDs6=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipPCpi4ED4kk7FFvDxdh9HrsB60Pr01sUkL3FP_o=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipMjNO_OAG8QA74JszAAE2K6gicxbGEMAQbRXtwt=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipMjDjq7wP8iUbUiFeBNFHjzOFBr6aVUPDcfDSHe=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNcccwmp8K24QID0jU9PcyWq2QswyyK6oc6gf1d=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNCnCfWeAdQ7JUbZHzFL_myn_S9Uv6rIiq7e6bV=w100-h100-p-n-k-no",
"https://lh5.googleusercontent.com/p/AF1QipNtFZMp7PZNr7e9OabzzVeGWzadGsgW7DUY2pcC=w100-h100-p-n-k-no"
],
"created": "5 months ago on Google",
"star_rating": "4/5"
},
Each review retrieved by the SERP API carries complete details of the review, including the review message, replies to the reviews, images (if any), rating, and basic reviewer details, such as name and photo. This data object is sufficient to help you understand how a business is doing based on the reviews that users leave for it on Google Maps.
Conclusion
While the Google Maps API is a great tool for embedding Google Maps in your app, it’s not the best option for accessing Google Maps data. With issues like rate limiting, high pricing, and limited data points, it makes sense to go for a dedicated API that can provide you with a better experience at a lower cost.
In this article, you saw how the Bright Data SERP API is better than the Places API offered as a part of the Google Maps API for retrieving Google Maps data. You also saw how to set it up in your own project.
This article should help you easily and efficiently extract the information you need from Google Maps. Make sure to sign up for your free trial of the SERP API today.
