python logo

python requests library

Python hosting: Host, run, and code Python in the cloud!

If you want to request data from webservers, the traditional way to do that in Python is using the urllib library. While this library is effective, you could easily create more complexity than needed when building something. Is there another way?

Requests is an Apache2 Licensed HTTP library, written in Python. It’s powered by httplib and urllib3, but it does all the hard work for you.

To install type:

git clone
cd requests
sudo python install

The Requests library is now installed. We will list some examples below:

Related course:
If you prefer a course or certification:

Grabbing raw html using HTTP/HTTPS requests
We can now query a website as :

import requests
r = requests.get('')
print r.content

Save it and run with:


It will output the raw HTML code.

Download binary image using Python

from PIL import Image
from StringIO import StringIO
import requests

r = requests.get('$
i =

An image retrieved using python

Website status code (is the website online?)

import requests
r = requests.get('')
print r.status_code

This returns 200 (OK). A list of status codes can be found here:

Retrieve JSON from a webserver
You can easily grab a JSON object from a webserver.

import requests

import requests
r = requests.get('')
print r.json()

HTTP Post requests using Python

from StringIO import StringIO
import requests

payload = {'key1': 'value1', 'key2': 'value2'}
r ="", data=payload)

SSL verification, verify certificates using Python

from StringIO import StringIO
import requests
print requests.get('', verify=True)

Extract data from the HTTP response header
With every request you send to a HTTP server, the server will send you some additional data. You can get extract data from an HTTP response using:

#!/usr/bin/env python
import requests
r = requests.get('')
print r.headers

This will return the data in JSON format. We can parse the data encoded in JSON format to a Python dict.

#!/usr/bin/env python
import requests
import json
r = requests.get('')

jsondata = str(r.headers).replace('\'','"')
headerObj = json.loads(jsondata)
print headerObj['server']
print headerObj['content-length']
print headerObj['content-encoding']
print headerObj['content-type']
print headerObj['date']
print headerObj['x-powered-by']

Extract data from HTML response
Once you get the data from a server, you can parse it using python string functions or use a library. BeautifulSoup is often used. An example code that gets the page title and links:

from bs4 import BeautifulSoup
import requests

# get html data
r = requests.get('')
html_doc = r.content

# create a beautifulsoup object
soup = BeautifulSoup(html_doc)

# get title
print soup.title

# print all links
for link in soup.find_all('a'):

Leave a Reply:

Fabi.Choi Mon, 18 May 2015

We have to get Pillow Library to execute 'Download binary image using Python'

How to :
pip install Pillow

Sid Thu, 18 Jun 2015


Can you extend this section with adding actions(extracting something from response) on response from get request? I want to learn about this.

Frank Fri, 19 Jun 2015

Hi, yes I will extend the section