how to set the logging in DEBUG mode when using the python requests package

By default when using the excellent requests package, there is not much on the screen to figure out was goes wrong with a specific URL requests. In general, you do not need it but from time to time it is useful to switch the logging level to e.g. DEBUG mode.


import requests
r = requests.post(YOUR_URL)

prints nothing on the screen. Now, what about some detais ? Since there is a logging system, let use get it:


requests.logging.getLogger()

This returns nothing special except a root logger. So the logging is actually happening outside of requests.
After some googling, I figured out that this is actcually happenning at a lower level (urllib).


import logging
log = logging.getLogger('urllib3')
log.setLevel(logging.DEBUG)

# logging from urllib3 to console
stream = logging.StreamHandler()
stream.setLevel(logging.DEBUG)
log.addHandler(stream)

This is now better since you have the url printed to the screen but this is not very helpful.
You need to set an extra debug level in the http package itself:

from http.client import HTTPConnection
HTTPConnection.debuglevel = 1
requests.get(YOUR_URL)

Now we get much more valuable information about the header, send and reply contents.

Please follow and like us:
This entry was posted in Computer Science, Linux, Python and tagged . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *