The Fetch API is a modern JavaScript interface that simplifies HTTP requests with its promise-based syntax that makes your code cleaner and easier to use. With the Fetch API, you can request data from the server without having to reload the entire page, making your application faster and more interactive. You can easily send GET, POST, PUT, or DELETE requests to interact with various APIs. The Fetch API makes handling different types of data and HTTP methods straightforward as well. Customizing request headers and managing different content types becomes simple, allowing you to work seamlessly with diverse services.
In this article, you’ll learn how to use the Fetch API, its benefits for web applications, and the use of proxies to ensure secure and efficient data handling. Go over this list of the best proxy providers to see why Bright Data is the top choice.
Understanding the Fetch API
The Fetch API is used for making asynchronous HTTP requests by sending a request to the server and returning a promise that resolves with the response data once available. In JavaScript development, using the Fetch API enables dynamic interaction with servers through HTTP requests (eg GET, POST, PUT) by allowing applications to communicate with the server without requiring a page reload. The following are some of the benefits of the Fetch API:
- Simplified syntax with promises: The Fetch API eliminates the complexity of XMLHttpRequest using promises, allowing you to write cleaner and more readable code.
- Various data formats are supported: You can use JSON, text, and blob formats, which makes it easy to parse and work with different response types.
- Stream and response objects are provided: You can examine and alter responses with readable streams and adjust your data retrieval approach to suit your application.
- Better error handling: Fetch doesn’t throw an error for HTTP error status codes by default, but it does allow you to do more explicit response checks.
- Less complicated callbacks: You don’t have to write many callbacks; instead, you can use
.then()
,.catch()
, orasync/await
to know when requests are completed.
Now that you have explored the benefits of using the Fetch API, let’s look into its key features and capabilities.
Asynchronous Operations
The Fetch API is asynchronous, and the promise is resolved once the server response is available. That way, you don’t block the UI for your users, and you give them smooth user experiences. Let’s explore some examples.
GET Request
Here’s an example of how to use the Fetch API to send a GET request:
fetch('https://jsonplaceholder.typicode.com/posts')
.then((response) => {
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status}`);
}
return response.json();
})
.then((data) => {
console.log('Data fetched:', data);
})
.catch((error) => {
console.error('Fetch error:', error);
});
In this snippet, the Fetch API gets posts from jsonplaceholder.typicode.com
. It checks if the response is okay using response.ok
and throws an error if not. Then, it parses the response to JSON with response.json()
. All errors, whether from network issues, badly formed responses, or parsing failures, are handled in the .catch()
block.
POST Request
Here’s how to use the Fetch API to send a POST request:
fetch('https://jsonplaceholder.typicode.com/posts', {
method: 'POST',
headers: {
'Content-type': 'application/json; charset=UTF-8',
},
body: JSON.stringify({
title: 'New Post',
body: 'Hello World! This is a test.',
userId: 1,
}),
})
.then((response) => {
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status}`);
}
return response.json();
})
.then((data) => {
console.log('New Post created:', data);
})
.catch((error) => {
console.error('Fetch error:', error);
});
Here, the Fetch API sends a request to create a new resource with the POST method. The POST request is similar to the GET request, but the request includes headers to inform the server the API is sending JSON data. The actual JSON data goes in the body key.
Integration with APIs
You can use the Fetch API to make requests to public or private APIs, such as retrieving data for charts, dashboards, or other data-driven features. It allows web applications to communicate with external services using its promise-based flow for handling requests and responses.
Here is a basic example of how you could fetch weather data from the OpenWeatherMap API:
const city = "London,uk";
const apiKey = "YOUR_API_KEY";
const url = `https://api.openweathermap.org/data/2.5/weather?q=${city}&APPID=${apiKey}`;
fetch(url)
.then((response) => {
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status}`);
}
return response.json();
})
.then((weatherData) => {
console.log("Weather data for London:", weatherData);
})
.catch((error) => {
console.error("Error fetching weather data:", error);
});
Replace
YOUR_API_KEY
with your actual API key.
This code retrieves weather data for London using OpenWeather’s API. The weatherData
object includes details such as temperature, humidity, and conditions, which you can integrate into your UI.
Once you run this code from your terminal or shell (using the command node filename.js
) you should see an output like the following:
Weather data for London: {
coord: { lon: -0.1257, lat: 51.5085 },
weather: [
{
id: 804,
main: 'Clouds',
description: 'overcast clouds',
icon: '04d'
}
],
base: 'stations',
main: {
temp: 273.42,
feels_like: 273.42,
temp_min: 272.59,
temp_max: 274.87,
pressure: 1021,
humidity: 87,
sea_level: 1021,
grnd_level: 1016
},
visibility: 10000,
wind: { speed: 0.51, deg: 0 },
clouds: { all: 99 },
dt: 1736525048,
sys: {
type: 2,
id: 268730,
country: 'GB',
sunrise: 1736496173,
sunset: 1736525567
},
timezone: 0,
id: 2643743,
name: 'London',
cod: 200
}
Apps often need to interact with external APIs to pull in external data. Therefore, using a clean, consistent approach for integrating with any external service is a key part of building a scalable web application.
Error Handling
Compared to older AJAX methods like XMLHttpRequest, which required setting up multiple event handlers to manage network errors, Fetch simplifies error handling using promises and a single .catch()
block. Fetch rejects a promise only if a network error occurs (eg no internet connection). It doesn’t throw an error for HTTP error codes like 404
or 500
. To see if one of these errors occurred, you have to manually check response.ok
. If response.ok
is false
, then it means the server responded with a status that is not in the 200–299 range. The following example shows how Fetch handles errors when a request fails due to an invalid endpoint:
fetch('https://jsonplaceholder.typicode.com/invalid-endpoint')
.then((response) => {
if (!response.ok) {
throw new Error(`Failed to fetch. Status: ${response.status}`);
}
return response.json();
})
.then((data) => {
console.log('This will not run if response is not ok');
})
.catch((error) => {
console.error('Network or response error:', error.message);
});
When you run this code, you get an output message of Network or response error: Failed to fetch. Status: 404
. The code checks !response.ok
and throws an error because the endpoint doesn’t exist.
Use of Proxies with Fetch API
Proxies can enhance the Fetch API capabilities by providing additional security, scalability, and geolocation capabilities. Proxies function as intermediaries between your client and the internet, providing unique IP addresses, additional layers of security, and a more flexible approach to resource fetching. Using proxies with Fetch can help prevent your IP address from being blocked by providing anonymity during data extraction or web scraping activities. Some other benefits of using Proxies include the following:
- Enhanced security: Proxies mask your IP address, which reduces the risk of your IP being vulnerable to malicious attacks.
- Scalability: Proxies can distribute requests across multiple IPs, preventing rate limits or blocks.
- Geolocation flexibility: Proxies are used to send HTTP requests from different locations, which is useful for accessing content that is only available in specific regions.
Types of Proxies
There are different types of proxies, each serves a specific use case. These different proxies are as follows:
- Residential proxies: These proxies use IPs from real residential locations and are highly trusted by websites, making them more expensive but also more effective in case the targeted websites block traffic from datacenter IPs.
- Rotating proxies: These proxies automatically rotate IPs with each request or session. They are ideal for large-scale data extraction when you want to reduce the risk of hitting server limits or blacklists.
- Datacenter proxies: These proxies come from datacenters; they are cost-effective and offer high speed, but you get blocked if you use them frequently.
Bright Data Proxies with Fetch API
Bright Data provides advanced proxy services that integrate with Fetch-based applications. These proxies allow you to choose between residential or datacenter options, scale your data retrieval capabilities for enterprise-level projects, and benefit from automatic IP rotation for efficient and reliable data collection.
Here’s how to integrate Bright Data proxies with Fetch API:
import fetch from "node-fetch";
import { HttpsProxyAgent } from "https-proxy-agent";
const proxyHost = "BRIGHT_DATA_PROXY_HOST:PORT";
const proxyUsername = "BRIGHT_DATA_USERNAME";
const proxyPassword = "BRIGHT_DATA_PASSWORD";
const proxyUrl = `http://${proxyUsername}:${proxyPassword}@${proxyHost}`;
const targetUrl = "https://jsonplaceholder.typicode.com/posts";
// Create a proxy agent
const agent = new HttpsProxyAgent(proxyUrl);
fetch(targetUrl, { agent })
.then((response) => {
if (!response.ok) {
throw new Error(`Error fetching data. Status: ${response.status}`);
}
return response.json();
})
.then((data) => {
console.log("Data fetched via proxy:", data);
})
.catch((error) => {
console.error("Proxy fetch error:", error);
});
This code sends a request through a specific proxy using fetch
with an HttpsProxyAgent
. It then defines the proxy details and creates a proxy agent. fetch
also gets the data from the target URL using the proxy.
To configure a proxy, you must use the
Fetch API
fromnode-fetch
. For more information, visit the node-fetch documentation. This code may be different based on the chosen proxy. Consult the Bright Data official documentation for the most recent information.
Fetch API Best Practices
When making Fetch API requests, follow these best practices to improve efficiency and performance:
Caching Responses
To reduce the load on servers and enhance the overall user experience, you can use caching mechanisms that store frequently requested data, thus avoiding redundant server calls. Here’s an example that checks whether your data is already cached before making a network request:
const cache = new Map();
async function fetchWithCache(url) {
if (cache.has(url)) {
console.log("Fetching from cache:", url);
return cache.get(url);
}
try {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
cache.set(url, data);
console.log("Fetched and cached:", url);
return data;
} catch (error) {
console.error("Fetch error:", error);
throw error;
}
}
fetchWithCache("https://jsonplaceholder.typicode.com/posts")
.then((data) => console.log(data))
.catch((error) => console.error("Error:", error));
Preventing unnecessary requests for already cached data can improve performance and reduce load times.
Timeout Handling
Setting some timeouts can keep your Fetch requests from just hanging around indefinitely. You can cancel requests that exceed a specified duration using the AbortController
. In the following example, the timeout is set to three seconds, while the default timeout value is five seconds:
async function fetchWithTimeout(url, timeout = 5000) {
const controller = new AbortController();
const timeoutId = setTimeout(() => {
console.warn("Fetch request timed out:", url);
controller.abort();
}, timeout);
try {
const response = await fetch(url, { signal: controller.signal });
clearTimeout(timeoutId);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
return await response.json();
} catch (error) {
console.error("Fetch timeout or error:", error);
throw error;
}
}
fetchWithTimeout("https://jsonplaceholder.typicode.com/posts", 3000)
.then((data) => console.log(data))
.catch((error) => console.error("Error:", error));
This ensures requests exceeding the specified timeout are aborted while handling other errors appropriately.
Request Throttling
To prevent your APIs from being overloaded with requests, you should implement throttling to control the frequency of requests. Thus, you should limit the number of requests that can be made in a certain time frame. The following example creates a throttle
function that ensures a function is executed at most once every specified interval (limit
):
function throttle(func, limit) {
let lastFunc;
let lastRan;
return function (...args) {
const context = this;
if (!lastRan) {
func.apply(context, args);
lastRan = Date.now();
} else {
clearTimeout(lastFunc);
lastFunc = setTimeout(function () {
if (Date.now() - lastRan >= limit) {
func.apply(context, args);
lastRan = Date.now();
}
}, limit - (Date.now() - lastRan));
}
};
}
const throttledFetch = throttle(async (url) => {
try {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
console.log("Throttled fetch data:", data);
} catch (error) {
console.error("Throttled fetch error:", error);
}
}, 2000);
throttledFetch("https://jsonplaceholder.typicode.com/posts");
console.log("Fetching again in 2 seconds...");
throttledFetch("https://jsonplaceholder.typicode.com/posts");
The throttledFetch
function uses the throttle
utility to control API requests. It ensures that repeated calls to fetch data from a URL are spaced by at least two seconds. This prevents redundant or excessive API calls, handles errors, and logs the fetched data.
Conclusion
The Fetch API simplifies asynchronous operations and data handling in modern web applications. However, challenges like IP blocks, rate limits, and CAPTCHAs can arise when dealing with large-scale or geotargeted requests.
Bright Data’s Scraper APIs and Scraping Browser provide powerful solutions to these challenges. The Scraper APIs enable seamless data extraction by bypassing anti-scraping measures, while the Scraping Browser handles JavaScript-heavy and dynamic content efficiently. These tools ensure secure, scalable, and reliable data collection for even the most complex websites.
No credit card required