Python Requests: A Guide

When you’re building an application in Python, there may come a time when you want to connect to a third-party service. For instance, if you’re building a fitness tool you may want to connect to the Fitbit API so you can see your exercise data; an app that can send text messages may connect to the Twilio API.

In Python, the requests library allows you to make requests so you can connect third-party web services to your applications. In this guide, we’re going to learn about the Python requests library and how you can use it to send HTTP requests.

Web Requests: A Refresher

Requests are at the center of the Internet. When you clicked on this article, a HTTP request was sent to the Career Karma server. That request told our servers what web page you wanted to see so our server could find the page. Once the web page was found, our web server returned it to your browser. This process allows you to see this tutorial.

Web requests come in many different forms. The type of request you made to view this web page was called a GET request. This type of request allows you to retrieve data. There are other types of requests such as POST and PUT, which are used to modify the resources on a server. For the purposes of this tutorial, we’re going to focus on GET and POST requests.

How to Install Requests

Requests is the HTTP library for Python that allows you to make web requests.

Before we can make web requests using Python, we have to install the Python requests code library. We can do so using the following command:

pip install requests

Now that we’ve downloaded the Python package, we’re ready to make a web request.

How to Make a GET Request

The most common type of web request you’ll make is a GET request. This type of request allows you to retrieve data from a server.

Let’s say we are making an app that allows you to retrieve a list of cat facts. To do so, we’re going to use an API called cat-facts. This API is open to the public, so we don’t need any credentials to sign in. We could use this code to make a request to the API:

import requests

res = requests.get('https://cat-fact.herokuapp.com/facts')

print(res.status_code)

This code returned: 200. Let’s break down our code.

On the first line, we imported the requests library. This allowed us to access the requests library that we will use in our tutorial. Then we used the requests.get() method to make a web request. We could have used methods like .put() or .post() to make other types of requests, but in this case we only needed to retrieve data from the cat-facts API.

Next, we printed out the status code from our web request. The number 200 was returned, which told us our web request was made successfully. We’ll discuss what these numbers mean later in the tutorial.

Viewing a Request

In our example above, we have printed out the status code of our request. But how can we see the data we have retrieved? Where are the cat facts? That’s where the .text and .json() methods come in.

View Text Response

We can use the .text method to see the result of our web request:

import requests

res = requests.get('https://cat-fact.herokuapp.com/facts')

print(res.text)

This code returned a list of cat facts. But, this isn’t very practical for us because the data we are querying is an API. While we now have a Python list of cat facts, it is difficult for us to read them because they are in plain text.

View JSON Response

A more useful format in which our cat facts could be stored is JSON. We could retrieve the JSON data from our request using this code:

print(res.json())

Our code returns a long list of cat facts. Here’s the first record in the list:

{'_id': '58e009550aac31001185ed12', 'text': 'The oldest cat video on YouTube dates back to 1894.', 'type': 'cat', 'user': {'_id': '58e007480aac31001185ecef', 'name': {'first': 'Kasimir', 'last': 'Schulz'}}, 'upvotes': 6, 'userUpvoted': None}

Success! We’ve managed to retrieve our list of cat facts.

How to Make a POST Request

You can also use the requests library to make a POST request. This will allow you to modify the data stored on a web server.

For this example, we’re going to use the Airtable API. This is because we need to use an API that supports POST requests and the cat-facts API is read only.

Suppose we have a database where we record all the cups of tea we drink. We’ve just drank another cup of tea and we want to add it to the database. We could do so using this code:

import requests

headers = {
'Authorization': 'Bearer API_KEY',
'Content-Type': 'application/json',
}

data = '{"records": [{"fields": {"Drink": "Black Decaf Tea"}}]}'

res = requests.post('https://cat-fact.herokuapp.com/facts', headers=headers, data=data)

print(res.json())

Our code returned:

{'records': [{'id': 'recqUEPuXEAXaNl1L', 'fields': {'Drink': 'Black Decaf Tea', 'Date': '2020-06-16T08:53:02.000Z'}, 'createdTime': '2020-06-16T08:53:02.000Z'}]}

This tells us our HTTP request was successful. We’ve just added a record to our Airtable database using their API. Let’s discuss these lines of code further.

First, we imported the requests library so we could make HTTP requests in our code.

We then defined a dictionary which contained two keys and values: one which stored our authentication key and another which specified the type of content we were sending to the web server. These are the headers that we needed to send to make a request to the Airtable API.

Next, we declared a variable called data which stored the data we wanted to send alongside our POST request. In this case, we were adding a drink called “Black Decaf Tea” to our database. We then used the requests.post() method and specified our headers and data as parameters, which allowed us to make a POST request to the Airtable API.

Finally, we printed out the result of our request in the JSON format using res.json().

"Career Karma entered my life when I needed it most and quickly helped me match with a bootcamp. Two months after graduating, I found my dream job that aligned with my values and goals in life!"

Venus, Software Engineer at Rockbot

Find Your Bootcamp Match

HTTP Status Codes

The HTTP protocol returns a unique status code when you make a web request. This allows you to see whether your web request has been successful or whether any errors have occurred. You’ve probably seen a few of these already, such as 404.

Here is a reference list of status codes you may encounter when making a request with Python requests:

• 1XX: Information
• 2XX: Request was made successfully
• 3XX: Request was redirected
• 4XX: Client-side error
• 5XX: Server-side error

Knowing these status codes allows us to add debugging into our program. Let’s take the GET request from the cat-facts API example from earlier. Suppose we want to print out a message if the resource we are accessing cannot be found. We could do so using this code:

import requests

res = requests.get('https://cat-fact.herokuapp.com/facts')

if res.status_code == 200:
	print("Success")
else :
	print("Error")

Our code returned: Success. Because the status code of our request is 200, the message “Success” is printed to the console. However, if our request had failed – for instance, we may have specified an invalid URL – then the message “Error” would have been printed to the console.

Conclusion

The Python requests library allows you to make HTTP requests in Python. In this guide, we discussed how the Python requests library is used to make GET and POST requests, but you can also make PUT and DELETE requests using the library.

When you use a method like requests.get() or requests.post(), Python will make a request to the web resource which you want to access. You can attach headers and data with your request, which allows you to send information if you want to make changes to a web resource you own.

Now you’re ready to start using the Python requests library like an expert!