Guide to Python Requests Library with 10 Examples

Working with the Python Requests Library

The Python requests library is a widely used Python library for making HTTP requests. It allows you to send HTTP requests to web servers and receive responses, which can include data or web pages. If you are interested in implementing Python proxies with the requests library, here are some key examples:

1. Making a GET request and retrieving the HTML content of a page

To retrieve the HTML content of a specific page by sending a GET request to a website's server, you can use the requests library in Python:

import requests

response = requests.get("https://www.example.com")
html_content = response.text

In this code, the `requests.get()` method sends a GET request to the specified URL, in this case, "https://www.example.com". The server's response is stored in the `response` object, and you can access the HTML content using the `.text` attribute.

2. Sending a POST request to a server with form data

To send a POST request to a server with form data, typically used for logging in to a website, you can use the requests library:

import requests

data = {"username": "myusername", "password": "mypassword"}
response = requests.post("https://www.example.com/login", data=data)

In this code, the `requests.post()` method sends a POST request to the specified URL, "https://www.example.com/login". The `data` argument contains a dictionary with the form data. You can modify the dictionary based on the required fields for the login form.

3. Sending a file to a server using the requests.post() method

To upload a file from your local computer to a server using the requests library, you can follow this example:

import requests

file = open("myfile.txt", "rb")
response = requests.post("https://www.example.com/upload", files={"file": file})
file.close()

In this code, you open the file you want to upload using the `open()` function, specifying the file name and the mode "rb" (binary read mode). The `requests.post()` method sends a POST request to the specified URL, "https://www.example.com/upload". The `files` argument contains a dictionary with the file you want to upload, where the key is the name of the file field on the server. Finally, you close the file using `file.close()`.

4. Sending a PUT request to a server to update a resource

To update an existing resource on a server by sending a PUT request, you can use the requests library:

import requests

data = {"title": "My Updated Title", "body": "My updated body text"}
response = requests.put("https://www.example.com/posts/1", data=data)

In this code, the `requests.put()` method sends a PUT request to the specified URL, "https://www.example.com/posts/1". The `data` argument contains a dictionary with the updated data for the resource you want to update, such as the title and body of a post. Adjust the dictionary according to the fields you want to update.

5. Sending a DELETE request to a server to delete a resource

To delete an existing resource on a server by sending a DELETE request, you can use the requests library:

import requests

response = requests.delete("https://www.example.com/posts/1")

In this code, the `requests..delete()` method sends a DELETE request to the specified URL, "https://www.example.com/posts/1". The server will delete the resource identified by the URL.

6. Handling errors and exceptions

When making HTTP requests with the requests library, it's important to handle potential errors and exceptions that may occur. Here's an example of how you can handle errors:

import requests

try:
    response = requests.get("https://www.example.com")
    response.raise_for_status()  # Raises an exception for 4xx or 5xx status codes
    # Process the response
except requests.exceptions.HTTPError as e:
    print(f"HTTP Error: {e}")
except requests.exceptions.ConnectionError as e:
    print(f"Connection Error: {e}")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

In this code, we use a try-except block to catch and handle different types of exceptions that may occur during the request. The raise_for_status() method raises an exception if the response has a 4xx or 5xx status code, indicating an error. You can customize the exception handling based on your specific requirements.

7. Using proxies with requests

If you want to use proxies with the requests library, you can specify the proxy settings using the `proxies` parameter. Here's an example:

import requests

proxies = {
    "http": "http://username:password@proxy_address:proxy_port",
    "https": "http://username:password@proxy_address:proxy_port"
}

response = requests.get("https://www.example.com", proxies=proxies)

In this code, you define a dictionary proxies with the HTTP and HTTPS proxy settings. Replace "username", "password", "proxy_address", and "proxy_port" with the appropriate values for your proxy configuration. Then, you pass the proxies dictionary as the proxies parameter when making the request.

These examples should give you a good starting point for using the requests library in Python. Remember to consult the requests documentation (https://docs.python-requests.org/) for more details and additional features offered by the library.