Python Requests: Guide to HTTP Requests and Response Handling

Using Requests in Python: A Quickstart Guide

Table of Content

1. Introduction
2. Installation and Setup
3. Making a Request
4. Custom Headers
5. Error Handling
6. Session Objects
7. File Uploads

Introduction

Welcome to the Quickstart guide for using the Requests library in Python! This guide will help you get started with making HTTP requests and handling responses effectively using Requests.

Installation and Setup

Before you begin, ensure that Requests is installed and up-to-date. You can install it using pip:

pip install requests

Making a Request

To start using Requests, import the module

import requests

Let's begin by making a GET request to fetch a webpage, for example, GitHub's public timeline

r = requests.get('https://api.github.com/events')

Now, r is a Response object that contains all the information from the server.

Different HTTP Methods Requests supports various HTTP methods like POST, PUT, DELETE, HEAD, and OPTIONS, which can be used similarly to GET

requests.post('https://httpbin.org/post', data={'key': 'value'})
requests.put('https://httpbin.org/put', data={'key': 'value'})
requests.delete('https://httpbin.org/delete')
requests.head('https://httpbin.org/get')
requests.options('https://httpbin.org/get')

Passing Parameters in URLs You can send data in the URL's query string using the params parameter

payload = {'key1': 'value1', 'key2': ['value2', 'value3']}
r = requests.get('https://httpbin.org/get', params=payload)
print(r.url)

Response Handling Text Content You can access the content of the server's response as text

r.text

JSON Content Requests provides a built-in JSON decoder for JSON data

r.json()

Binary Content For non-text responses like images, access the content as bytes

r.content

Custom Headers

Add custom HTTP headers using the headers parameter

headers = {'user-agent': 'my-app/0.0.1'}
r = requests.get(url, headers=headers)

More Advanced Features Handling Cookies

cookies = {'cookie_name': 'cookie_value'}
r = requests.get(url, cookies=cookies)

More Advanced Features Handling Cookies

cookies = {'cookie_name': 'cookie_value'}
r = requests.get(url, cookies=cookies)

Redirection and History You can track redirection and history using the history property

r.history

Timeouts Set a timeout for the request to avoid hanging indefinitely

requests.get(url, timeout=0.001)

Error Handling

Requests handles various exceptions like ConnectionError, HTTPError, Timeout, and TooManyRedirects.

import requests
from requests.exceptions import ConnectionError, HTTPError, Timeout, TooManyRedirects

url = 'https://example.com'  # Replace 'example.com' with the actual URL

try:
    response = requests.get(url)
    response.raise_for_status()  # Raise HTTPError for bad responses (4xx and 5xx status codes)
except ConnectionError:
    print("Connection Error: Check your internet connection or the website's URL.")
except HTTPError as http_err:
    print(f"HTTP error occurred: {http_err}")
except Timeout:
    print("Timeout error: The request took too long to complete.")
except TooManyRedirects:
    print("Too many redirects: The server redirected the request too many times.")
except Exception as err:
    print(f"An error occurred: {err}")
else:
    print("Request successful!")
    # Process the response here

• We import specific exceptions related to Requests from requests.exceptions.
• We define a URL to make a GET request to.
• Inside the try block, we send a GET request using requests.get(url).
• We use response.raise_for_status() to raise an HTTPError for bad responses (status codes 4xx and 5xx).
• We handle specific exceptions like ConnectionError, HTTPError, Timeout, and TooManyRedirects individually.
• If no exception occurs, we process the response in the else block.

Customizing Request Headers You can customize request headers using the headers parameter. This allows you to send additional information along with your request, such as authentication tokens or content types

import requests

url = 'https://httpbin.org/headers'
headers = {'Authorization': 'Bearer my_token', 'Content-Type': 'application/json'}

r = requests.get(url, headers=headers)
print(r.json())

This example sends a GET request to 'https://httpbin.org/headers' with custom headers for authorization and content type. Handling Authentication Requests supports various authentication methods like Basic Authentication, OAuth, and more. Here's an example of using Basic Authentication

import requests

url = 'https://api.example.com/login'
auth = ('username', 'password')

r = requests.get(url, auth=auth)
print(r.text)

This code sends a GET request with Basic Authentication credentials (username and password) to 'https://api.example.com/login'.

Session Objects

You can create session objects in Requests to persist certain parameters across multiple requests, such as cookies or headers. This is useful for maintaining a session context

import requests

url = 'https://httpbin.org/cookies/set'
cookies = {'session_id': '123456789'}

session = requests.Session()
session.cookies.update(cookies)

r = session.get(url)
print(r.text)

In this example, a session object is created to handle cookies, and then a GET request is made to 'https://httpbin.org/cookies/set' with the session's cookies.

File Uploads

Requests supports file uploads with the files parameter. You can upload files as part of a POST request

import requests

url = 'https://httpbin.org/post'
files = {'file': open('example.txt', 'rb')}

r = requests.post(url, files=files)
print(r.text)

This code uploads the 'example.txt' file to 'https://httpbin.org/post' using a POST request with the files parameter.

Community and Collaboration

The Requests library is maintained by a vibrant community of developers who actively contribute to its development and improvement. You can join the community by contributing code, reporting issues, and participating in discussions on the Requests GitHub repository and other developer forums. Collaborating with other developers and sharing your experiences can enhance the functionality and usability of Requests for everyone.

Feel free to add more content relevant to your project or interests, such as specific use cases, best practices, or additional advanced features of Requests that you find valuable.