Python Requests: Working with Prepared Requests and SSL Certificate Verification

Prepared Requests in Python's requests Library

• When using the requests library in Python, the PreparedRequest object can be particularly useful for making modifications to requests before sending them. This guide will walk you through the basics of using PreparedRequest and provide detailed explanations with code examples to help you understand its functionality.

Table of content

1. Preserving Session State
2. Merging Environment Settings
3. SSL Certificate Verification

from requests import Request, Session

# Create a session
s = Session()

# Create a request
req = Request('GET', 'https://example.com/api', data={'key': 'value'}, headers={'X-Test': 'true'})

# Prepare the request using the session
prepped = s.prepare_request(req)

# Modify the prepared request
prepped.body = 'Seriously, send exactly these bytes.'
prepped.headers['Keep-Dead'] = 'parrot'

# Send the request
resp = s.send(prepped, stream=False, verify=True, proxies=None, cert=None, timeout=30)

print(resp.status_code)

Explanation

• Create a session: We start by creating a Session object which allows us to persist settings across multiple requests (such as cookies).
• Create a request: We then create a Request object with the desired HTTP method (POST), URL, data, and headers.
• Prepare the request: The prepare() method is called on the Request object to create a PreparedRequest object.
• Modify the prepared request: We can directly modify the attributes of the PreparedRequest object. In this example, we change the body and remove the Content-Type header.
• Send the request: The modified PreparedRequest is sent using the Session.send() method, which allows us to specify additional parameters like stream, verify, proxies, cert, and timeout.
• Print response status code: Finally, we print the status code of the response to verify that the request was successfully processed.

Preserving Session State

When using the requests.Session object, it is important to preserve session state (like cookies). To ensure this, use Session.prepare_request() instead of Request.prepare():

from requests import Request, Session

# Create a session
s = Session()

# Create a request
req = Request('GET', 'https://example.com/api', data={'key': 'value'}, headers={'X-Test': 'true'})

# Prepare the request using the session
prepped = s.prepare_request(req)

# Modify the prepared request
prepped.body = 'Seriously, send exactly these bytes.'
prepped.headers['Keep-Dead'] = 'parrot'

# Send the request
resp = s.send(prepped, stream=False, verify=True, proxies=None, cert=None, timeout=30)

print(resp.status_code)

Explanation:

• Create a session: We create a Session object.
• Create a request: We create a Request object similar to the previous example.
• Prepare the request using the session: Instead of calling prepare(), we call prepare_request() on the session object. This ensures that session-level state (such as cookies) is applied to the PreparedRequest.
• Modify the prepared request: As before, we modify the body and headers of the PreparedRequest.
• Send the request: The modified PreparedRequest is sent using the Session.send() method.
• Print response status code: We print the status code of the response.

Merging Environment Settings

• When using environment variables to configure requests, you might encounter issues with the PreparedRequest flow. Here's how to merge environment settings into your session to avoid such issues:

from requests import Request, Session

# Create a session
s = Session()

# Create a request
req = Request('GET', 'https://example.com/api')

# Prepare the request using the session
prepped = s.prepare_request(req)

# Merge environment settings into the session
settings = s.merge_environment_settings(prepped.url, {}, None, None, None)
resp = s.send(prepped, **settings)

print(resp.status_code)

Explanation

• Create a session: We create a Session object.
• Create a request: We create a Request object.
• Prepare the request using the session: We prepare the request using prepare_request() on the session object.
• Merge environment settings: We call merge_environment_settings() on the session object to merge any environment settings (like proxy configuration or * * SSL certificates) into the session. This helps to avoid issues that might arise if the environment settings are not considered.
• Send the request: The modified PreparedRequest is sent using the Session.send() method with the merged settings.
• Print response status code: We print the status code of the response.

SSL Certificate Verification

• requests verifies SSL certificates for HTTPS requests by default. Here are some ways to handle SSL certificate verification:

Default Verification

import requests

resp = requests.get('https://github.com')
print(resp.status_code)

Explanation

• Default SSL Verification: By default, requests will verify the SSL certificate of the server. If the server's certificate is valid and trusted, the request will succeed.

Custom CA Bundle

import requests

resp = requests.get('https://github.com', verify='/path/to/certfile')
print(resp.status_code)

Explanation

• Custom CA Bundle: You can specify a custom path to a CA bundle file using the verify parameter. This allows requests to use a specific set of trusted certificates.

Disable SSL Verification

import requests

resp = requests.get('https://example.com', verify=False)
print(resp.status_code)

Explanation

• Disable SSL Verification: By setting verify=False, you can disable SSL certificate verification. This can be useful for testing or local development, but it poses a security risk in production environments as it makes the application vulnerable to man-in-the-middle (MitM) attacks.

Persistent SSL Verification in Sessions

import requests

s = requests.Session()
s.verify = '/path/to/certfile'

resp = s.get('https://github.com')
print(resp.status_code)

Explanation

• Persistent SSL Verification: When using a Session object, you can set the verify attribute to a custom CA bundle path. This ensures that all requests made through the session use the specified CA bundle for SSL verification.

Conclusion

• Using PreparedRequest in the requests library allows you to make fine-grained modifications to your HTTP requests before sending them. This can be useful for custom headers, body content, or handling session state. Remember to handle SSL verification appropriately to ensure the security of your applications.

References

Prepared requests

SSL Cert Verification

Stack Overflow: How to modify a prepared request in Python's requests library

Python Requests: SSL Certificate Verification