How To Make REST API Calls With cURL

cURL (Client URL) has been the go-to command line HTTP client for decades. It’s widely used by both developers and administrators in all facets of web development such as testing APIs, fetching web pages, and even file transfer. In web scraping, we primarily use cURL for testing HTTP requests.

REST (Representational State Transfer) is a standard used for creating an API (Application Programming Interface). With a REST API, a server is configured to handle four basic request types: GET, POST, PUT and DELETE. Each of these methods is used for interacting with a web server. We’ll go into more detail throughout this article.

Getting Started

Installation

cURL comes preinstalled on most major operating systems today. It’s been a staple in Linux for many years, but macOS and Windows now also include it by default. You can check your install of cURL with the following command.

curl --version

If cURL is properly installed, you should see a response similar to this. Most importantly, the --version command should output a version number. Below, our version number is curl 7.81.0. The rest of the output is release and OS specific.

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.18
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

If for some reason, you don’t have cURL installed, you can find a version matching your OS on their downloads page here.

<!–

• how to install curl: be brief, link to os specific instructions
  –>

How To Use cURL

Making a GET Request

The most common HTTP request is the GET request. We use GET to GET information. When you’re fetching a web page, your browser performs a GET under the hood. With cURL, we use the GET flag to perform one. The example below sends a GET to https://jsonplaceholder.typicode.com/posts.

curl -X GET https://jsonplaceholder.typicode.com/posts

When we perform this request, we GET the posts back as a response. Below, we have just a snippet of the response.

{
    "userId": 1,
    "id": 1,
    "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
  },
  {
    "userId": 1,
    "id": 2,
    "title": "qui est esse",
    "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"
  },
  {
    "userId": 1,
    "id": 3,
    "title": "ea molestias quasi exercitationem repellat qui ipsa sit aut",
    "body": "et iusto sed quo iure\nvoluptatem occaecati omnis eligendi aut ad\nvoluptatem doloribus vel accusantium quis pariatur\nmolestiae porro eius odio et labore et velit aut"
  },
  {
    "userId": 1,
    "id": 4,
    "title": "eum et est occaecati",
    "body": "ullam et saepe reiciendis voluptatem adipisci\nsit amet autem assumenda provident rerum culpa\nquis hic commodi nesciunt rem tenetur doloremque ipsam iure\nquis sunt voluptatem rerum illo velit"
  },
  {
    "userId": 1,
    "id": 5,
    "title": "nesciunt quas odio",
    "body": "repudiandae veniam quaerat sunt sed\nalias aut fugiat sit autem sed est\nvoluptatem omnis possimus esse voluptatibus quis\nest aut tenetur dolor neque"
  },

<!–

• what is a get request?
  –>

Making a POST Request

We use a POST request when we want to send information to a server to be saved in a database. Most common examples of this are social media posts. The example below sends a POST to create a new post using the API.

curl -X POST https://jsonplaceholder.typicode.com/posts \
     -H "Content-Type: application/json" \
     -d '{
           "title": "foo",
           "body": "bar",
           "userId": 1
         }'

After our request, we receive the actual post object back as the response.

{
  "title": "foo",
  "body": "bar",
  "userId": 1,
  "id": 101
}

If you’d like to learn more about sending in-depth POST requests, take a look at this guide.

Making a PUT Request

As you just learned, POST requests are used to create an object in a database. Sometimes, we wish to edit an object that already exists in a database. To alter an existing object, we use a PUT request. The example below updates our previous post body from bar to updated bar.

curl -X PUT https://jsonplaceholder.typicode.com/posts/1 \
     -H "Content-Type: application/json" \
     -d '{
           "id": 1,
           "title": "foo",
           "body": "updated bar",
           "userId": 1
         }'

When you execute it, you receive the following response. As you can see, the body of our post has been updated.

{
  "id": 1,
  "title": "foo",
  "body": "updated bar",
  "userId": 1
}

Making a DELETE Request

DELETE requests should be pretty self explanatory by this point. The DELETE request is used to remove an existing object from a database using a unique identifier. In this case, our identifier (id), is 1, or post number one. The code below removes the post we created earlier.

curl -X DELETE https://jsonplaceholder.typicode.com/posts/1

As you can see, our response is just an empty JSON object. The post has been deleted.

{}

cURL With Web Unlocker

When you use cURL with Web Unlocker, you can make HTTP requests just like you would using cURL by itself. However, Web Unlocker gives you the power of proxy.

With Web Unlocker, you get geotargeting, and CAPTCHA solving sourced from some of the best proxies in the world. Best of all, these proxies are managed for you. Simply hook up the proxy and continue on with your day as normal.

Once you’ve got Web Unlocker setup, make sure to save your username, zone name, and password. You’ll need these to authenticate your connection. In the example below, we set our connection to use a US based proxy.

curl -i --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<YOUR_USERNAME>-zone-<YOUR_ZONE_NAME>-country-us:<YOUR_PASSWORD> -k "https://geo.brdtest.com/mygeo.json"

• -i: Tells cURL that we want the headers included in our response. This is useful for debugging purposes.
• --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<YOUR_USERNAME>-zone-<YOUR_ZONE_NAME>-country-us:<YOUR_PASSWORD>
  • --proxy brd.superproxy.io:33335: This tells cURL that we want to use a proxy located at brd.superproxy.io:33335.
  • --proxy-user brd-customer-<YOUR_USERNAME>-zone-<YOUR_ZONE_NAME>-country-us:<YOUR_PASSWORD>: This represents our authentication string. The auth string is laid out like this: <username>:<password>. With Web Unlocker, your full username includes all of the following: brd-customer-<YOUR_USERNAME>-zone-<YOUR_ZONE_NAME>-country-us.
• k is used to tell cURL that we want to bypass SSL certification. If you’re interested in using Web Unlocker long-term, you can download and install our SSL certificate here. This will allow your machine to self-certify the connection.

You can view our example response below. As you can see, our location is showing up in New Jersey.

{"country":"US","asn":{"asnum":20473,"org_name":"AS-VULTR"},"geo":{"city":"Piscataway","region":"NJ","region_name":"New Jersey","postal_code":"08854","latitude":40.5511,"longitude":-74.4606,"tz":"America/New_York","lum_city":"piscataway","lum_region":"nj"}}

If you’re interested in learning more about Web Unlocker, take a look at our API Playground. It’s a great place to experiment and learn more about making API requests using many different languages and HTTP libraries. You can learn more about using cURL with proxies here.

Where Can You Go From cURL?

Once you’ve got a decent understanding of cURL and HTTP, you can go on to use HTTP pretty much anywhere. You can use GUI tools like Postman and Insomnia for general API testing.

With Python, you can use Requests or you can even go on to use cURL inside Python. With JavaScript, you can use Node-Fetch or Axios to automate your HTTP requests.

If you’re looking to stick with command line tools, there are a bunch for that as well. Take a look at HTTPie, and wget. Both of these tools are great command line utilities and they speak HTTP quite well.

Conclusion

When it comes to HTTP clients, cURL has been the command line standard for decades and it’s not going to change any time soon. Whether you’re a developer or a system administrator, you’ve now got the required skillset for basic cURL usage and a decent understanding of how HTTP works. You can even download files and set custom headers using cURL. Now that you know how to use Web Unlocker with it, you can be truly effective with HTTP no matter what you’re working on. If you’re an aspiring developer, go and learn how to use HTTP in your language of choice.