---
title: "A Guide to Puppeteer Web Scraping in 2026"
slug: web-scraping-puppeteer
date: 2023-05-25T10:29:37+00:00
modified: 2026-08-24T13:40:36+00:00
permalink: https://brightdata.com/blog/how-tos/web-scraping-puppeteer
type: blog
---
[ Blog ](https://brightdata.com/blog "Blog") / [How Tos](https://brightdata.com/blog/how-tos)
[How Tos](https://brightdata.com/blog/how-tos)
# A Guide to Puppeteer Web Scraping in 2026
Learn how to use Puppeteer for web scraping with a step-by-step tutorial, advanced techniques, and a comparison with Playwright and Selenium.
15 min read
[ ](https://brightdata.com/blog/authors/antonello-zanini)
[Antonello Zanini
Technical Writer
](https://brightdata.com/blog/authors/antonello-zanini)
In this guide, you will see:
- What Puppeteer is and what it offers for web scraping
- How to use it with a step-by-step tutorial to extract data from a dynamic site
- Advanced Puppeteer web scraping scenarios
- A comparison between Puppeteer, Playwright, and Selenium
- Its limitations and how to overcome them
Let’s dive in!
## What Is Puppeteer?
[Puppeteer](https://pptr.dev/) is a Node.js library that provides a high-level API for automating Chromium-based browsers—and recently, also Firefox. It is widely used for web scraping as a [browser automation tool to control headless browsers](/blog/web-data/best-headless-browsers).
The library allows you to appear like a regular user by simulating user behavior in a controlled browser. Puppeteer also offers a rich API for selecting HTML elements and extracting data from them.
Key features of Puppeteer for web scraping include:
- JavaScript execution for handling dynamic content
- API for simulating user interactions, such as navigating pages, clicking elements, and more
- API for data extraction
- Ability to take screenshots of HTML elements and pages for visual scraping (e.g., competitor sites)
- Headless browser support for [faster scraping](/blog/web-data/speed-up-web-scraping)
- Network request interception
- Extensive customization options like user-agent, proxy settings, and more
In short, Puppeteer lets you interact with pages like a real user, making it perfect for scraping modern, JavaScript-heavy pages.
## Web Scraping With Puppeteer: Step-By-Step Example
In this guided section, you will learn how to build a scraping script using Puppeteer. The target site will be a modified version of [Quotes to Scrape](https://quotes.toscrape.com/js-delayed/?delay=500), which loads quotes dynamically via JavaScript rendering:
This page uses JavaScript to render quotes after a specified delay, controlled by the `delay` query parameter. In this case, the delay is set to 500ms (`?delay=500`).
The script will use browser automation to:
1. Connect to the target page
2. Wait for the quote elements to render
3. Scrape data from them
4. Export the scraped data to CSV
Time to build your Puppeteer web scraping script!
### Step #1: Project Setup
First, make sure you have the latest LTS version of Node.js installed on your machine. If not, [download it from the official site](https://nodejs.org/en/download) and follow the installation steps.
Create a new folder for your JavaScript project and navigate to it in the terminal:
```none
mkdir puppeteer-web-scraper
cd puppeteer-web-scraper
```
Run the command below to initialize a Node.js project inside the folder:
```none
npm init -y
```
Next, open your Puppeteer web scraping project in your preferred JavaScript IDE, and add `index.js` to it:
Amazing! You are ready to start working with Puppeteer.
### Step #2: Puppeteer Installation and Setup
Install Puppeteer by adding the [`puppeteer`](https://www.npmjs.com/package/puppeteer) npm package to your project’s dependencies:
```none
npm install puppeteer
```
During installation, Puppeteer automatically downloads the appropriate version of Chrome.
To get started, import `puppeteer` in `index.js`:
```none
const puppeteer = require("puppeteer");
```
Next, create an `async` function where to initialize a browser instance:
```none
const puppeteer = require("puppeteer");
(async () => {
// launch the browser to control and open a new page
const browser = await puppeteer.launch();
const page = await browser.newPage();
// browser automation for web scraping...
// close the browser and release resources
await browser.close();
})();
```
The `launch()` method starts a new browser instance [in headless mode](/blog/proxy-101/what-is-a-headless-browser) by default. Then, `newPage()` opens a new tab within that instance. Finally, `close()` properly shuts down the browser.
To run the browser in headed mode—useful for debugging and visually tracking the script’s execution—set the `headless` option to `false`:
```none
const browser = await puppeteer.launch({
headless: false,
});
```
Wonderful! Your Puppeteer scraping setup is complete.
### Step #3: Visit the Target Page
Use the `goto()` method to navigate the browser to the target page:
```none
await page.goto("https://quotes.toscrape.com/js-delayed/?delay=1000");
```
The `delay=1000` query parameter instructs the page to load the quotes asynchronously after 1 second.
Enable headed mode in your script, then set a breakpoint before the `close()` method, and run the script in the debugger. You should see something like this:
Perfect! Your Puppeteer web scraping navigation is working perfectly.
### Step #4: Inspect the Target Page
Before implementing the scraping logic, it is important to study the site’s structure and the DOM of the target page. To do that, open the website in incognito mode in your browser.
Since you need to extract data from each quote on the page, inspect a quote’s HTML element. Right-click on the first quote and select “Inspect” to open the DevTools panel:
See that each quote is contained within a `.quote` HTML element. Each quote includes:
- The quote text inside a `.text` node.
- The author’s name inside a `.author` element.
- Multiple `.tag` elements inside a `.tags` node.
Good! You now have all the necessary information to do web scraping with Puppeteer.
### Step #5: Implement the Scraping Logic
Since you want to scrape multiple quotes, start by initializing an array where to store the extracted data:
```none
quotes = []
```
Remember that quotes are loaded dynamically via JavaScript. Initially, the page appears empty, and the content is populated later:
To verify that all quotes are loaded before scraping, wait for the `.quote` elements to appear:
```none
await page.waitForSelector(".quote");
```
The `waitForSelector()` method waits for the specified selector to be on the DOM of the page.
Now that all quotes are on the page, use `$$()` to select all elements matching the CSS selector `.quote`:
```none
const quoteElements = await page.$$(".quote");
```
Note that `$$()` supports both CSS and XPath selectors. Discover more in our guide on [XPath vs CSS Selector](/blog/web-data/xpath-vs-css-selectors).
Next, iterate through each quote element:
```none
for (const quoteElement of quoteElements) {
// scraping logic...
}
```
Inside the loop, extract the quote text, author, and tags:
```none
const textElement = await quoteElement.$(".text");
const text = await textElement.evaluate((el) =>
el.textContent.replace("“", "").replace("”", "")
);
const authorElement = await quoteElement.$(".author");
const author = await authorElement.evaluate((el) => el.textContent);
const tags = await quoteElement.$$eval(".tags .tag", (tagElements) =>
tagElements.map((el) => el.textContent)
);
```
`$()` works just like `$$()` but returning the first element matching the given selector. The snippet above uses the `evaluate()` method to execute a function within the context of an HTML element. Specifically, it extracts the text content from the selected element. The `$$eval()` method combines `$$()` with `evaluate()` to collect all tag texts into an array.
After extracting the data, store it in an object and add it to the `quotes` array:
```none
const quote = {
"text": text,
"author": author,
"tags": tags,
};
quotes.push(quote);
```
Terrific! The data extraction logic is finalized.
### Step #6: Export Data to CSV
While Node.js has built-in tools for populating CSV files, using a dedicated library like [`csv-writer`](https://www.npmjs.com/package/csv-writer) simplifies the process. Add it to your project’s dependencies with:
```none
npm install csv-writer
```
Then, import it into your `index.js` file:
```none
const createCsvWriter = require("csv-writer").createObjectCsvWriter;
```
Before exporting the data to CSV, make sure to flatten it into an object with all string values. Currently, the `tags` field is an array, so use `join()` to convert it into a string:
```none
const flattenQuotes = quotes.map((quote) => {
return {
...quote,
tags: quote.tags.join(";"),
};
});
```
Now, use the flattened data to write the scraped quotes into a `quotes.csv` file:
```none
const csvWriter = createCsvWriter({
path: "quotes.csv",
header: [
{ id: "text", title: "Text" },
{ id: "author", title: "author" },
{ id: "tags", title: "Tags" },
],
});
// write the scraped quotes in a flattened format to the CSV
await csvWriter.writeRecords(flattenQuotes);
```
Fantastic! You started with quotes from a webpage and now have them neatly organized in a CSV file.
### Step #7: Put It All Together
Your final Puppeteer web scraping script should look like this:
```none
const puppeteer = require("puppeteer");
const createCsvWriter = require("csv-writer").createObjectCsvWriter;
(async () => {
// launch the browser to control and open a new page
const browser = await puppeteer.launch({
headless: false, // comment out in production
});
const page = await browser.newPage();
// navigate to the target page
await page.goto("https://quotes.toscrape.com/js-delayed/?delay=1000");
// where to store the scraped data
const quotes = [];
// wait for the quote elements to load
await page.waitForSelector(".quote");
// select all quote containers
const quoteElements = await page.$$(".quote");
// extract data from each quote
for (const quoteElement of quoteElements) {
// scraping logic
const textElement = await quoteElement.$(".text");
const text = await textElement.evaluate((el) =>
el.textContent.replace("“", "").replace("”", "")
);
const authorElement = await quoteElement.$(".author");
const author = await authorElement.evaluate((el) => el.textContent);
const tags = await quoteElement.$$eval(".tags .tag", (tagElements) =>
tagElements.map((el) => el.textContent)
);
// populate a new quote with the scraped data
// and add it to the list
const quote = {
text: text,
author: author,
tags: tags,
};
quotes.push(quote);
}
// prepare the scraped data to be exported to CSV
const flattenQuotes = quotes.map((quote) => {
return {
...quote,
tags: quote.tags.join(";"),
};
});
// configurations for the CSV output file
const csvWriter = createCsvWriter({
path: "quotes.csv",
header: [
{ id: "text", title: "Text" },
{ id: "author", title: "author" },
{ id: "tags", title: "Tags" },
],
});
// write the scraped quotes in a flattened format to the CSV
await csvWriter.writeRecords(flattenQuotes);
// close the browser
await browser.close();
})();
```
Great job! In less than 100 lines of code, you built a script to scrape quotes using Puppeteer.
To run the script, execute this command in your terminal:
```none
node index.js
```
The script will take some time to run. After a while, a `quotes.csv` file will appear in your project’s directory. Open it, and you will see the scraped data:
Et voilà! The output CSV file now contains the same data from the target page, neatly organized in a structured format.
## Advanced Web Scraping Techniques Using Puppeteer
Now that you know the basics of web scraping with Puppeteer, you are ready to explore more advanced scenarios!
### User Interaction
Over time, web pages have become more complex than ever. They now include advanced navigation and interaction patterns like “Load More” buttons, forms, and dynamic dropdowns.
Learn more in our guide on [scraping sites with complex navigation](/blog/web-data/scraping-complex-navigation).
Puppeteer is built to handle many of these scenarios with dedicated methods. For instance, you can locate and click a “Load More” button with:
```none
await page.click("#load-more");
```
The `click()` function simulates a real user interaction on the button identified by the `#load-more` selector.
Keep in mind that the recommended way to interact with an element is by using a [`Locator`](https://pptr.dev/api/puppeteer.locator). This defines a strategy for finding elements and performing actions on them. If you already have a locator for the button, simply call `click()` on it:
```none
const button = await page.locator("#load-more")
await button.click();
```
**Note**: When using locators, Puppeteer automatically waits for the element to appear in the DOM and reach the right state before performing an action. For example, before clicking, Puppeteer:
- Ensures the element is within the viewport.
- Waits for the element to become visible.
- Waits for the element to be enabled.
- Confirms the element’s bounding box remains stable across two consecutive animation frames.
To fill out and submit a form, you can use:
```none
await page.type('input[name="email"]', "[email protected]");
await page.type('input[name="password"]', "mysecurepassword");
await page.click('button[type="submit"]');
```
The `type()` method inputs text into form fields, while `click()` submits the form by clicking the submit button. Again, you can call those methods also on locators.
Finally, you can select an option from a dropdown with:
```none
await page.select("#country", "USA");
```
The `select()` function picks a dropdown option based on its value. The above snippet will select the “USA” item in the `