Python requests ignore ssl error

In this article, we will discuss how to disable security certificate checks for requests in Python.

In Python, the requests module is used to send HTTP requests of a particular method to a specified URL. This request returns a response object that includes response data such as encoding, status, content, etc. But whenever we perform operations like get, post, delete, etc. 

We will get SSLCertVerificationError i.e., SSL:Certificate_Verify_Failed self-signed certificate. To get rid of this error there are two ways to disable the security certificate checks. They are

• Passing verify=False to request method.
• Use Session.verify=False

Method 1: Passing verify=False to request method

The requests module has various methods like get, post, delete, request, etc. Each of these methods accepts an URL for which we send an HTTP request. Along with the URL also pass the verify=False parameter to the method in order to disable the security checks.

Output:

<Response [200]>

Explanation: By passing verify=False to the request method we disabled the security certificate check and made the program error-free to execute. But this approach will throw warnings as shown in the output picture. This can be avoided by using urlib3.disable_warnings method.

Handle the abovewarning with requests.packages.urllib3.disable_warnings() method

The above warning that occurred while using verify=False in the request method can be suppressed by using the urllib3.disable_warnings method. 

Output:

<Response [200]>

Method 2: Use Session.verify=False

The alternate way of disabling the security check is using the Session present in requests module. We can declare the Session.verify=False instead of passing verify=True as parameter. Let’s look into the sample code so that one will get the clear picture of using Session.

Output

Ignore SSL Security Certificate Check in Python Requests

Accessing a URL without a secure SSL certificate raises exception warnings when HTTP requests are sent to it. A lot of times, the SSL certificate of these URLs expires, thus creating all sorts of security issues.

If the information is not sensitive, these warnings can be subsided when programs use requests in Python. This article will provide multiple ways to disable security certificate checks using requests .

Understanding the Reason Behind SSL Security Checks and Why It Fails

If a program uses Python requests to get requests from a URL whose SSL certificate is expired, it raises two exceptions. The below program displays what those exceptions are.

Scenario 1

This program uses a URL provided by the SSL community with an expired security certificate for testing purposes. It must be noted that the SSL security exceptions only raises with URLs having expired SSL certificates.

The Python requests do not raise any exception with URLs with a valid SSL certificate or a revoked one. So this article will mainly focus on URLs with expired security certificates.

The example below shows a simple program that imports requests in the first line. The second line of the program sends a post request to the URL to modify occurrences of ‘bar’ as ‘baz’ .

It is done to send a post request to the URL and holds no other significance inside the program.

Executing this Python script throws SSLError exceptions.

This is the first exception to be considered in learning how to disable security certificate checks using requests .

Scenario 2

This program turns off the SSL certificate verification using verify=False to disable the security certificate check using requests .

The requests library is built in a way that it can turn off verification for SSL certificates, but the program throws another exception with links having expired SSL certificates.

These two exceptions are needed to be dealt with to disable security certificate checks using requests .

Ignore SSL Security Check in Python

This section will explain various methods that either disable security certificate check using requests or provides a turnaround with the problem. Every method has its purpose.

Create a Monkey Patch for the requests Library

If a third-party library requires the security checks disabled, the requests library can be monkey patched. A context manager is used for patching to disable security certificate checks using requests .

After the requests is patched, the verify field is given a False value by default, suppressing the warning. Another warning is raised when verify=false is used, as explained in scenario 2 of the previous section.

The patch will then add an exception handling block to disable security certificate check using requests and suppresses the warnings. It will be easier to understand with the following examples.

The program below patches the requests library. Let’s understand what this code does.

Imports:

1. warnings : This library package is a sub-package of the Exceptions library.
2. contextlib : This Python library is used for patching the requests library.
3. requests : The requests library package of Python.
4. urllib3 : It is a module that handles HTTP requests and URLs in Python. This library is used to import a sub-module InsecureRequestWarning , which raises an exception for expired SSL certificates.

At first, the program saves the default environment settings of the requests library in a variable — old_merge_environment_settings . This variable will be used to bring requests back to their default state after the opened adapters are closed.

The method no_ssl_verification is created and decorated with @contextlib.contextmanager . A new variable opened_adapters is created and assigned a set() to it.

A set is a datatype that stores items in an unordered format.

Inside the no_ssl_verification method, another nested method merge_environment_settings is created. This method has six parameters, similar to the merge_environment_settings from the requests module, and it will act as the patch for the original module.

Inside the method, the opened_adapters variable is updated with matching adapter pair from the parameter url . Every connection is made with some matching adapter pair, which gets returned in this step.

Since verification only occurs once per connection, we must shut all opened adapters after we’re done. If not, the effects of verify=False last after this context manager has ended.

It is important to remember the first section of the article to understand the next line of code. When the requests.post function was used on the URL with an expired SSL certificate, it threw two exceptions.

The first exception was caused by verify , which is set with a True value. Though the verify field was switchable, it could be given a False value.

The second exception was a non-mutable entity that stopped the program from making a connection.

The below code modifies the verify field to have a False value by default to solve this problem. A new variable settings is created to execute this step, which gets assigned with data from the variable old_merge_environment_settings .

Once the data is stored inside the variable settings , the verify field is turned to False .

This changes the default value of verify . Lastly, this variable is returned.

The data from the updated merge_environment_settings method is assigned to the function requests.Session.merge_environment_settings . This will assure that the field verify has a False value by default.

We have dealt with the first exception. The second exception, which is not mutable, will be solved using an exception handling block.

Let’s understand what the code does here.

At first, inside the try block, a with block is created that catches all the warnings raised. Inside this with block, warnings.simplefilter function is used to give an ignore value to the urllib3 sub-module InsecureRequestWarning .

As we learned in the imports section, the InsecureRequestWarning sub-module raises an exception for expiring SSL certificates; assigning an ignore value to it will bypass the second exception caused.

Inside finally block, the original settings stored inside old_merge_environment_settings is assigned back to the function requests.Session.merge_environment_settings .

As all the opened adapters need to be closed before the program exits, a for loop is created that iterates for the number of times opened adapters are inside the program.

All the adapters are closed using the function adapter.close() using a try-except block; inside the except block, the iteration is passed.

The complete code is given below.

Use Monkey Patch in Python

The monkey patch can be invoked using the no_ssl_verification() method. Any request sent under this method will follow the directives of the monkey patch and disable security certificate checks using requests .

Let’s look at an example.

The method no_ssl_verification is called under a with block. This will run the method inside the block and then closes itself when the compiler comes out of the block.

Inside the block, a requests.get call is sent to the URL with an expired SSL certificate. With the default settings, this would have caused an exception throw, but this time the requests.get call is sent successfully.

Another request is sent with the requests , where the verify field is set to True . Unlike the default scenario, this time, no error exception is thrown.

It was seen that Verify=False had the directives to reset the patch with default settings. When this field was set inside requests.get() , the monkey patch resets to the default settings, allowing to throw error exceptions.

The requests sub-module session can also be used with the monkey patch.

In a variable named session , the function requests.Session() is loaded. The session.verify value is set to True .

Inside a with block, the session.get() function is used to send the get request to the URL, and the verify field is set to True . A print statement prints a message if the connection is made.

When the context manager exits, this code closes any open adapters that handle a patched request. Since requests maintains a connection pool for each session and certificate validation only occurs once per connection, unexpected events like the following will occur.

Use urllib3 to Disable Warnings in Python

If the program requires disabling security certificate checks without using requests or monkey patching, then using the urllib3 provides a simple solution.

Import requests and urllib3 , and from urllib3 import the submodule InsecureRequestWarning . The warning is disabled using the urllib3.disable_warnings function.

The requests.post() statement is placed under the try block, and the verify field is set to False . This will disable the security check for expired security certificates.

Inside the except block, the SSLError is raised with an error message.

Conclusion

This article explains various methods to disable security certificate checks using requests in Python. The reader, through the article, can disable security checks easily.

Источник

Advanced Usage¶

This document covers some of Requests more advanced features.

Session Objects¶

The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use urllib3 ’s connection pooling. So if you’re making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase (see HTTP persistent connection).

A Session object has all the methods of the main Requests API.

Let’s persist some cookies across requests:

Sessions can also be used to provide default data to the request methods. This is done by providing data to the properties on a Session object:

Any dictionaries that you pass to a request method will be merged with the session-level values that are set. The method-level parameters override session parameters.

Note, however, that method-level parameters will not be persisted across requests, even if using a session. This example will only send the cookies with the first request, but not the second:

If you want to manually add cookies to your session, use the Cookie utility functions to manipulate Session.cookies .

Sessions can also be used as context managers:

This will make sure the session is closed as soon as the with block is exited, even if unhandled exceptions occurred.

Remove a Value From a Dict Parameter

Sometimes you’ll want to omit session-level keys from a dict parameter. To do this, you simply set that key’s value to None in the method-level parameter. It will automatically be omitted.

All values that are contained within a session are directly available to you. See the Session API Docs to learn more.

Request and Response Objects¶

Whenever a call is made to requests.get() and friends, you are doing two major things. First, you are constructing a Request object which will be sent off to a server to request or query some resource. Second, a Response object is generated once Requests gets a response back from the server. The Response object contains all of the information returned by the server and also contains the Request object you created originally. Here is a simple request to get some very important information from Wikipedia’s servers:

If we want to access the headers the server sent back to us, we do this:

However, if we want to get the headers we sent the server, we simply access the request, and then the request’s headers:

Prepared Requests¶

Whenever you receive a Response object from an API call or a Session call, the request attribute is actually the PreparedRequest that was used. In some cases you may wish to do some extra work to the body or headers (or anything else really) before sending a request. The simple recipe for this is the following:

Since you are not doing anything special with the Request object, you prepare it immediately and modify the PreparedRequest object. You then send that with the other parameters you would have sent to requests.* or Session.* .

However, the above code will lose some of the advantages of having a Requests Session object. In particular, Session -level state such as cookies will not get applied to your request. To get a PreparedRequest with that state applied, replace the call to Request.prepare() with a call to Session.prepare_request() , like this:

When you are using the prepared request flow, keep in mind that it does not take into account the environment. This can cause problems if you are using environment variables to change the behaviour of requests. For example: Self-signed SSL certificates specified in REQUESTS_CA_BUNDLE will not be taken into account. As a result an SSL: CERTIFICATE_VERIFY_FAILED is thrown. You can get around this behaviour by explicity merging the environment settings into your session:

SSL Cert Verification¶

Requests verifies SSL certificates for HTTPS requests, just like a web browser. By default, SSL verification is enabled, and Requests will throw a SSLError if it’s unable to verify the certificate:

I don’t have SSL setup on this domain, so it throws an exception. Excellent. GitHub does though:

You can pass verify the path to a CA_BUNDLE file or directory with certificates of trusted CAs:

If verify is set to a path to a directory, the directory must have been processed using the c_rehash utility supplied with OpenSSL.

This list of trusted CAs can also be specified through the REQUESTS_CA_BUNDLE environment variable.

Requests can also ignore verifying the SSL certificate if you set verify to False:

By default, verify is set to True. Option verify only applies to host certs.

Client Side Certificates¶

You can also specify a local cert to use as client side certificate, as a single file (containing the private key and the certificate) or as a tuple of both files’ paths:

If you specify a wrong path or an invalid cert, you’ll get a SSLError:

The private key to your local certificate must be unencrypted. Currently, Requests does not support using encrypted keys.

CA Certificates¶

Requests uses certificates from the package certifi. This allows for users to update their trusted certificates without changing the version of Requests.

Before version 2.16, Requests bundled a set of root CAs that it trusted, sourced from the Mozilla trust store. The certificates were only updated once for each Requests version. When certifi was not installed, this led to extremely out-of-date certificate bundles when using significantly older versions of Requests.

For the sake of security we recommend upgrading certifi frequently!

Body Content Workflow¶

By default, when you make a request, the body of the response is downloaded immediately. You can override this behaviour and defer downloading the response body until you access the Response.content attribute with the stream parameter:

At this point only the response headers have been downloaded and the connection remains open, hence allowing us to make content retrieval conditional:

You can further control the workflow by use of the Response.iter_content() and Response.iter_lines() methods. Alternatively, you can read the undecoded body from the underlying urllib3 urllib3.HTTPResponse at Response.raw .

If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data or call Response.close . This can lead to inefficiency with connections. If you find yourself partially reading request bodies (or not reading them at all) while using stream=True , you should make the request within a with statement to ensure it’s always closed:

Keep-Alive¶

Excellent news — thanks to urllib3, keep-alive is 100% automatic within a session! Any requests that you make within a session will automatically reuse the appropriate connection!

Note that connections are only released back to the pool for reuse once all body data has been read; be sure to either set stream to False or read the content property of the Response object.

Streaming Uploads¶

Requests supports streaming uploads, which allow you to send large streams or files without reading them into memory. To stream and upload, simply provide a file-like object for your body:

It is strongly recommended that you open files in binary mode . This is because Requests may attempt to provide the Content-Length header for you, and if it does this value will be set to the number of bytes in the file. Errors may occur if you open the file in text mode.

Chunk-Encoded Requests¶

Requests also supports Chunked transfer encoding for outgoing and incoming requests. To send a chunk-encoded request, simply provide a generator (or any iterator without a length) for your body:

For chunked encoded responses, it’s best to iterate over the data using Response.iter_content() . In an ideal situation you’ll have set stream=True on the request, in which case you can iterate chunk-by-chunk by calling iter_content with a chunk_size parameter of None . If you want to set a maximum size of the chunk, you can set a chunk_size parameter to any integer.

POST Multiple Multipart-Encoded Files¶

You can send multiple files in one request. For example, suppose you want to upload image files to an HTML form with a multiple file field ‘images’:

To do that, just set files to a list of tuples of (form_field_name, file_info) :

It is strongly recommended that you open files in binary mode . This is because Requests may attempt to provide the Content-Length header for you, and if it does this value will be set to the number of bytes in the file. Errors may occur if you open the file in text mode.

Event Hooks¶

Requests has a hook system that you can use to manipulate portions of the request process, or signal event handling.

response : The response generated from a Request.

You can assign a hook function on a per-request basis by passing a dictionary to the hooks request parameter:

That callback_function will receive a chunk of data as its first argument.

If an error occurs while executing your callback, a warning is given.

If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn’t return anything, nothing else is affected.

Let’s print some request method arguments at runtime:

You can add multiple hooks to a single request. Let’s call two hooks at once:

You can also add hooks to a Session instance. Any hooks you add will then be called on every request made to the session. For example:

A Session can have multiple hooks, which will be called in the order they are added.

Custom Authentication¶

Requests allows you to use specify your own authentication mechanism.

Any callable which is passed as the auth argument to a request method will have the opportunity to modify the request before it is dispatched.

Authentication implementations are subclasses of AuthBase , and are easy to define. Requests provides two common authentication scheme implementations in requests.auth : HTTPBasicAuth and HTTPDigestAuth .

Let’s pretend that we have a web service that will only respond if the X-Pizza header is set to a password value. Unlikely, but just go with it.

Then, we can make a request using our Pizza Auth:

Streaming Requests¶

With Response.iter_lines() you can easily iterate over streaming APIs such as the Twitter Streaming API. Simply set stream to True and iterate over the response with iter_lines :

When using decode_unicode=True with Response.iter_lines() or Response.iter_content() , you’ll want to provide a fallback encoding in the event the server doesn’t provide one:

iter_lines is not reentrant safe. Calling this method multiple times causes some of the received data being lost. In case you need to call it from multiple places, use the resulting iterator object instead:

Proxies¶

If you need to use a proxy, you can configure individual requests with the proxies argument to any request method:

You can also configure proxies by setting the environment variables HTTP_PROXY and HTTPS_PROXY .

To use HTTP Basic Auth with your proxy, use the http://user:password@host/ syntax:

To give a proxy for a specific scheme and host, use the scheme://hostname form for the key. This will match for any request to the given scheme and exact hostname.

Note that proxy URLs must include the scheme.

SOCKS¶

New in version 2.10.0.

In addition to basic HTTP proxies, Requests also supports proxies using the SOCKS protocol. This is an optional feature that requires that additional third-party libraries be installed before use.

You can get the dependencies for this feature from pip :

Once you’ve installed those dependencies, using a SOCKS proxy is just as easy as using a HTTP one:

Using the scheme socks5 causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use socks5h as the scheme.

Compliance¶

Requests is intended to be compliant with all relevant specifications and RFCs where that compliance will not cause difficulties for users. This attention to the specification can lead to some behaviour that may seem unusual to those not familiar with the relevant specification.

Encodings¶

When you receive a response, Requests makes a guess at the encoding to use for decoding the response when you access the Response.text attribute. Requests will first check for an encoding in the HTTP header, and if none is present, will use chardet to attempt to guess the encoding.

The only time Requests will not do this is if no explicit charset is present in the HTTP headers and the Content-Type header contains text . In this situation, RFC 2616 specifies that the default charset must be ISO-8859-1 . Requests follows the specification in this case. If you require a different encoding, you can manually set the Response.encoding property, or use the raw Response.content .

HTTP Verbs¶

Requests provides access to almost the full range of HTTP verbs: GET, OPTIONS, HEAD, POST, PUT, PATCH and DELETE. The following provides detailed examples of using these various verbs in Requests, using the GitHub API.

We will begin with the verb most commonly used: GET. HTTP GET is an idempotent method that returns a resource from a given URL. As a result, it is the verb you ought to use when attempting to retrieve data from a web location. An example usage would be attempting to get information about a specific commit from GitHub. Suppose we wanted commit a050faf on Requests. We would get it like so:

We should confirm that GitHub responded correctly. If it has, we want to work out what type of content it is. Do this like so:

So, GitHub returns JSON. That’s great, we can use the r.json method to parse it into Python objects.

So far, so simple. Well, let’s investigate the GitHub API a little bit. Now, we could look at the documentation, but we might have a little more fun if we use Requests instead. We can take advantage of the Requests OPTIONS verb to see what kinds of HTTP methods are supported on the url we just used.

Uh, what? That’s unhelpful! Turns out GitHub, like many API providers, don’t actually implement the OPTIONS method. This is an annoying oversight, but it’s OK, we can just use the boring documentation. If GitHub had correctly implemented OPTIONS, however, they should return the allowed methods in the headers, e.g.

Turning to the documentation, we see that the only other method allowed for commits is POST, which creates a new commit. As we’re using the Requests repo, we should probably avoid making ham-handed POSTS to it. Instead, let’s play with the Issues feature of GitHub.

This documentation was added in response to Issue #482. Given that this issue already exists, we will use it as an example. Let’s start by getting it.

Cool, we have three comments. Let’s take a look at the last of them.

Well, that seems like a silly place. Let’s post a comment telling the poster that he’s silly. Who is the poster, anyway?

OK, so let’s tell this Kenneth guy that we think this example should go in the quickstart guide instead. According to the GitHub API doc, the way to do this is to POST to the thread. Let’s do it.

Huh, that’s weird. We probably need to authenticate. That’ll be a pain, right? Wrong. Requests makes it easy to use many forms of authentication, including the very common Basic Auth.

Brilliant. Oh, wait, no! I meant to add that it would take me a while, because I had to go feed my cat. If only I could edit this comment! Happily, GitHub allows us to use another HTTP verb, PATCH, to edit this comment. Let’s do that.

Excellent. Now, just to torture this Kenneth guy, I’ve decided to let him sweat and not tell him that I’m working on this. That means I want to delete this comment. GitHub lets us delete comments using the incredibly aptly named DELETE method. Let’s get rid of it.

Excellent. All gone. The last thing I want to know is how much of my ratelimit I’ve used. Let’s find out. GitHub sends that information in the headers, so rather than download the whole page I’ll send a HEAD request to get the headers.

Excellent. Time to write a Python program that abuses the GitHub API in all kinds of exciting ways, 4995 more times.

Custom Verbs¶

From time to time you may be working with a server that, for whatever reason, allows use or even requires use of HTTP verbs not covered above. One example of this would be the MKCOL method some WEBDAV servers use. Do not fret, these can still be used with Requests. These make use of the built-in .request method. For example:

Utilising this, you can make use of any method verb that your server allows.

Many HTTP APIs feature Link headers. They make APIs more self describing and discoverable.

GitHub uses these for pagination in their API, for example:

Requests will automatically parse these link headers and make them easily consumable:

Transport Adapters¶

As of v1.0.0, Requests has moved to a modular internal design. Part of the reason this was done was to implement Transport Adapters, originally described here. Transport Adapters provide a mechanism to define interaction methods for an HTTP service. In particular, they allow you to apply per-service configuration.

Requests ships with a single Transport Adapter, the HTTPAdapter . This adapter provides the default Requests interaction with HTTP and HTTPS using the powerful urllib3 library. Whenever a Requests Session is initialized, one of these is attached to the Session object for HTTP, and one for HTTPS.

Requests enables users to create and use their own Transport Adapters that provide specific functionality. Once created, a Transport Adapter can be mounted to a Session object, along with an indication of which web services it should apply to.

The mount call registers a specific instance of a Transport Adapter to a prefix. Once mounted, any HTTP request made using that session whose URL starts with the given prefix will use the given Transport Adapter.

Many of the details of implementing a Transport Adapter are beyond the scope of this documentation, but take a look at the next example for a simple SSL use- case. For more than that, you might look at subclassing the BaseAdapter .

Example: Specific SSL Version¶

The Requests team has made a specific choice to use whatever SSL version is default in the underlying library (urllib3). Normally this is fine, but from time to time, you might find yourself needing to connect to a service-endpoint that uses a version that isn’t compatible with the default.

You can use Transport Adapters for this by taking most of the existing implementation of HTTPAdapter, and adding a parameter ssl_version that gets passed-through to urllib3 . We’ll make a Transport Adapter that instructs the library to use SSLv3:

Blocking Or Non-Blocking?¶

With the default Transport Adapter in place, Requests does not provide any kind of non-blocking IO. The Response.content property will block until the entire response has been downloaded. If you require more granularity, the streaming features of the library (see Streaming Requests ) allow you to retrieve smaller quantities of the response at a time. However, these calls will still block.

If you are concerned about the use of blocking IO, there are lots of projects out there that combine Requests with one of Python’s asynchronicity frameworks. Some excellent examples are requests-threads, grequests, and requests-futures.

In unusual circumstances you may want to provide headers in an ordered manner. If you pass an OrderedDict to the headers keyword argument, that will provide the headers with an ordering. However, the ordering of the default headers used by Requests will be preferred, which means that if you override default headers in the headers keyword argument, they may appear out of order compared to other headers in that keyword argument.

If this is problematic, users should consider setting the default headers on a Session object, by setting Session to a custom OrderedDict . That ordering will always be preferred.

Timeouts¶

Most requests to external servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more.

The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the connect()) call on the socket. It’s a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.

Once your client has connected to the server and sent the HTTP request, the read timeout is the number of seconds the client will wait for the server to send a response. (Specifically, it’s the number of seconds that the client will wait between bytes sent from the server. In 99.9% of cases, this is the time before the server sends the first byte).

If you specify a single value for the timeout, like this:

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

If the remote server is very slow, you can tell Requests to wait forever for a response, by passing None as a timeout value and then retrieving a cup of coffee.

Requests is an elegant and simple HTTP library for Python, built for human beings. You are currently looking at the documentation of the development release.

Stay Informed

Receive updates on new releases and upcoming projects.

Источник

1. Understanding the Reason Behind SSL Security Checks and Why It Fails
2. Ignore SSL Security Check in Python
3. Conclusion

Accessing a URL without a secure SSL certificate raises exception warnings when HTTP requests are sent to it. A lot of times, the SSL certificate of these URLs expires, thus creating all sorts of security issues.

If the information is not sensitive, these warnings can be subsided when programs use requests in Python. This article will provide multiple ways to disable security certificate checks using requests.

Understanding the Reason Behind SSL Security Checks and Why It Fails

If a program uses Python requests to get requests from a URL whose SSL certificate is expired, it raises two exceptions. The below program displays what those exceptions are.

Scenario 1

This program uses a URL provided by the SSL community with an expired security certificate for testing purposes. It must be noted that the SSL security exceptions only raises with URLs having expired SSL certificates.

The Python requests do not raise any exception with URLs with a valid SSL certificate or a revoked one. So this article will mainly focus on URLs with expired security certificates.

The example below shows a simple program that imports requests in the first line. The second line of the program sends a post request to the URL to modify occurrences of 'bar' as 'baz'.

It is done to send a post request to the URL and holds no other significance inside the program.

import requests

requests.post(url='https://expired-rsa-dv.ssl.com/', data={'bar': 'baz'})

Executing this Python script throws SSLError exceptions.

....
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='expired-rsa-dv.ssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),))

This is the first exception to be considered in learning how to disable security certificate checks using requests.

Scenario 2

This program turns off the SSL certificate verification using verify=False to disable the security certificate check using requests.

import requests

requests.post(url='https://expired-rsa-dv.ssl.com/', data={'bar': 'baz'}, verify=False)

The requests library is built in a way that it can turn off verification for SSL certificates, but the program throws another exception with links having expired SSL certificates.

InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,

These two exceptions are needed to be dealt with to disable security certificate checks using requests.

Ignore SSL Security Check in Python

This section will explain various methods that either disable security certificate check using requests or provides a turnaround with the problem. Every method has its purpose.

Create a Monkey Patch for the requests Library

If a third-party library requires the security checks disabled, the requests library can be monkey patched. A context manager is used for patching to disable security certificate checks using requests.

After the requests is patched, the verify field is given a False value by default, suppressing the warning. Another warning is raised when verify=false is used, as explained in scenario 2 of the previous section.

The patch will then add an exception handling block to disable security certificate check using requests and suppresses the warnings. It will be easier to understand with the following examples.

The program below patches the requests library. Let’s understand what this code does.

Imports:

1. warnings: This library package is a sub-package of the Exceptions library.
2. contextlib: This Python library is used for patching the requests library.
3. requests: The requests library package of Python.
4. urllib3: It is a module that handles HTTP requests and URLs in Python. This library is used to import a sub-module InsecureRequestWarning, which raises an exception for expired SSL certificates.

Patch:

At first, the program saves the default environment settings of the requests library in a variable — old_merge_environment_settings. This variable will be used to bring requests back to their default state after the opened adapters are closed.

old_merge_environment_settings = requests.Session.merge_environment_settings

The method no_ssl_verification is created and decorated with @contextlib.contextmanager. A new variable opened_adapters is created and assigned a set() to it.

A set is a datatype that stores items in an unordered format.

@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

Inside the no_ssl_verification method, another nested method merge_environment_settings is created. This method has six parameters, similar to the merge_environment_settings from the requests module, and it will act as the patch for the original module.

def merge_environment_settings(self, url, proxies, stream, verify, cert):

Inside the method, the opened_adapters variable is updated with matching adapter pair from the parameter url. Every connection is made with some matching adapter pair, which gets returned in this step.

Since verification only occurs once per connection, we must shut all opened adapters after we’re done. If not, the effects of verify=False last after this context manager has ended.

def merge_environment_settings(self, url, proxies, stream, verify, cert):

    opened_adapters.add(self.get_adapter(url))

It is important to remember the first section of the article to understand the next line of code. When the requests.post function was used on the URL with an expired SSL certificate, it threw two exceptions.

The first exception was caused by verify, which is set with a True value. Though the verify field was switchable, it could be given a False value.

The second exception was a non-mutable entity that stopped the program from making a connection.

The below code modifies the verify field to have a False value by default to solve this problem. A new variable settings is created to execute this step, which gets assigned with data from the variable old_merge_environment_settings.

Once the data is stored inside the variable settings, the verify field is turned to False.

This changes the default value of verify. Lastly, this variable is returned.

settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
settings['verify'] = False

return settings

The data from the updated merge_environment_settings method is assigned to the function requests.Session.merge_environment_settings. This will assure that the field verify has a False value by default.

requests.Session.merge_environment_settings = merge_environment_settings

We have dealt with the first exception. The second exception, which is not mutable, will be solved using an exception handling block.

Let’s understand what the code does here.

At first, inside the try block, a with block is created that catches all the warnings raised. Inside this with block, warnings.simplefilter function is used to give an ignore value to the urllib3 sub-module InsecureRequestWarning.

As we learned in the imports section, the InsecureRequestWarning sub-module raises an exception for expiring SSL certificates; assigning an ignore value to it will bypass the second exception caused.

try:
	with warnings.catch_warnings():
		warnings.simplefilter('ignore', InsecureRequestWarning)
		yield

Inside finally block, the original settings stored inside old_merge_environment_settings is assigned back to the function requests.Session.merge_environment_settings.

As all the opened adapters need to be closed before the program exits, a for loop is created that iterates for the number of times opened adapters are inside the program.

All the adapters are closed using the function adapter.close() using a try-except block; inside the except block, the iteration is passed.

for adapter in opened_adapters:
    try:
        adapter.close()
    except:
        pass

The complete code is given below.

import warnings
import contextlib

import requests
from urllib3.exceptions import InsecureRequestWarning

old_merge_environment_settings = requests.Session.merge_environment_settings


@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

    def merge_environment_settings(self, url, proxies, stream, verify, cert):

        opened_adapters.add(self.get_adapter(url))

        settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
        settings['verify'] = False

        return settings

    requests.Session.merge_environment_settings = merge_environment_settings

    try:
        with warnings.catch_warnings():
            warnings.simplefilter('ignore', InsecureRequestWarning)
            yield
    finally:
        requests.Session.merge_environment_settings = old_merge_environment_settings

        for adapter in opened_adapters:
            try:
                adapter.close()
            except:
                pass

Use Monkey Patch in Python

The monkey patch can be invoked using the no_ssl_verification() method. Any request sent under this method will follow the directives of the monkey patch and disable security certificate checks using requests.

Let’s look at an example.

The method no_ssl_verification is called under a with block. This will run the method inside the block and then closes itself when the compiler comes out of the block.

Inside the block, a requests.get call is sent to the URL with an expired SSL certificate. With the default settings, this would have caused an exception throw, but this time the requests.get call is sent successfully.

Another request is sent with the requests, where the verify field is set to True. Unlike the default scenario, this time, no error exception is thrown.

with no_ssl_verification():
    requests.get('https://expired-rsa-dv.ssl.com/')
    print('Modifications working Properly')

    requests.get('https://expired-rsa-dv.ssl.com/', verify=True)
    print('`Verify=true` has its meaning changed')

Output:

Modifications working Properly

`Verify=true` has its meaning changed

It was seen that Verify=False had the directives to reset the patch with default settings. When this field was set inside requests.get(), the monkey patch resets to the default settings, allowing to throw error exceptions.

requests.get('https://expired-rsa-dv.ssl.com/', verify=False)
print('It resets back')

Output:

connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'.
  InsecureRequestWarning,
It resets back

The requests sub-module session can also be used with the monkey patch.

In a variable named session, the function requests.Session() is loaded. The session.verify value is set to True.

Inside a with block, the session.get() function is used to send the get request to the URL, and the verify field is set to True. A print statement prints a message if the connection is made.

session = requests.Session()
session.verify = True

with no_ssl_verification():
    session.get('https://expired-rsa-dv.ssl.com/', verify=True)
    print('Works in Session')

Output:

When the context manager exits, this code closes any open adapters that handle a patched request. Since requests maintains a connection pool for each session and certificate validation only occurs once per connection, unexpected events like the following will occur.

try:
    requests.get('https://expired-rsa-dv.ssl.com/', verify=False)
except requests.exceptions.SSLError:
    print('It breaks')

try:
    session.get('https://expired-rsa-dv.ssl.com/')
except requests.exceptions.SSLError:
    print('It breaks here again')

Output:

C:UsersWin 10venvlibsite-packagesurllib3connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'.
  InsecureRequestWarning,
It breaks here again

Use urllib3 to Disable Warnings in Python

If the program requires disabling security certificate checks without using requests or monkey patching, then using the urllib3 provides a simple solution.

Import requests and urllib3, and from urllib3 import the submodule InsecureRequestWarning. The warning is disabled using the urllib3.disable_warnings function.

The requests.post() statement is placed under the try block, and the verify field is set to False. This will disable the security check for expired security certificates.

Inside the except block, the SSLError is raised with an error message.

import requests
from urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

try:
    requests.post(url='https://expired-rsa-dv.ssl.com/', data={'bar': 'baz'}, verify=False)
    print("Connection made successfully")

except requests.exceptions.SSLError:
    print("Expired Certificate")

Output:

Connection made successfully

Conclusion

This article explains various methods to disable security certificate checks using requests in Python. The reader, through the article, can disable security checks easily.

python requests authentication provides multiple mechanisms for authentication to web service endpoints, including basic auth, X.509 certificate authentication, and authentication with a bearer token (JWT or OAuth2 token). This article will cover the basic examples for authenticating with each of these and using python requests to login to your web service.

python requests basic auth

To authenticate with basic auth using the python requests module, start with the following example python script:

import requests
basicAuthCredentials = ('user', 'pass')
 
response = requests.get('https://example.com/endpoint', auth=basicAuthCredentials)

An alternative to this approach is to just use the python requests authorization header for basic auth:

'Authorization' : 'Basic user:pass'

python requests ignore ssl

To ignore SSL verification of the installed X.509 SSL certificate, set verify=False. For example, in a python requests GET request to ignore ssl:

requests.get('https://example.com', verify=False)

The verify=False parameter in the get method declares that the python requests call must ignore ssl and proceed with the api call. This may be necessary in the event of a self signed certificate in certificate chain on the server being connected to. The cause may also be an expired certificate or some other certificate not trusted by your python application by default.

python requests certificate verify failed

If you get the python requests ssl certificate_verify_failed error, the cause is that the certificate may be expired, revoked, or not trusted by you as the caller. If you are in a test environment then it may be safe to set verify=False on your call, as explained above. If you are in a production environment then you should determine whether or not you should trust the root certificate of the trust chain being sent by the server. You may explicitly set this in your call, for example:

requests.get('https://example.com', verify='truststore.pem')

The ssl certificate_verify_failed error is not an error you should simply ignore with thoroughly thinking through the implications.

python requests certificate

python requests authentication with an X.509 certificate and private key can be performed by specifying the path to the cert and key in your request. An example using python requests client certificate:

requests.get('https://example.com', cert=('/path/client.cert', '/path/client.key'))

The certificate and key may also be combined into the same file. If you receive an SSL error on your python requests call, you have likely given an invalid path, key, or certificate in your request.

python requests authorization bearer

The python requests authorization header for authenticating with a bearer token is the following:

'Authorization': 'Bearer ' + token

For example:

import requests
headers = {'Authorization': 'Bearer ' + token}
response = requests.get('https://example.com', headers=headers)

The bearer token is often either a JWT (Javascript web token) or an OAuth2 token for python requests using oauth2.

To generate and sign a JWT with python and a private key, here is an example. The JWT token generated from this python code snippet may then be passed as the bearer token.

import jwt
import time
# Open private key to sign token with
with open('key.pem', 'r') as f:
  key = f.read()
# Create token that is valid for a given amount of time.
now = time.time();
claims = {
  'iss': '',
  'sub': '',
  'iat': int(now),
  'exp': int(now) + 10 * 60
}
token = jwt.encode(claims, key, algorithm='RS256')

It is also possible to load the private key directly from a Keystore using the pyOpenSSL dependency. Here is a code sample using python requests jwt authentication with a Keystore. Similar to the example above, you can pass in the generated JWT as the bearer token in your python requests REST call.

#!/usr/bin/env python3
import os
import sys
import time
from OpenSSL import crypto
from jwt import JWT
from jwt.jwk import RSAJWK
with open('keystore.p12', 'rb') as f:
  p12 = crypto.load_pkcs12(f.read(), bytes('changeit', 'utf-8'))
key = RSAJWK(p12.get_privatekey().to_cryptography_key())
# Create authentication token
now = int(time.time())
claims = {
    'iss': '',
    'sub': '',
    'iat': int(now),
    'exp': int(now) + 10 * 60
}
jwt = JWT()
token = jwt.encode(claims, key, alg='RS256')
print(token)

If you would like to see more examples of how to authenticate to REST web services with basic auth, bearer tokens (JWTs or OAuth2), or a private key and certificate leave us a comment.

Conclusion

This article has demonstrated how to use python requests with an x509 client certificate, python requests with a cert and key, and general authentication methods. Please leave us a comment if you have any questions on how to authenticate with a cert and key using python requests.

Read more of our content.

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Making HTTP Requests With Python

The requests library is the de facto standard for making HTTP requests in Python. It abstracts the complexities of making requests behind a beautiful, simple API so that you can focus on interacting with services and consuming data in your application.

Throughout this article, you’ll see some of the most useful features that requests has to offer as well as how to customize and optimize those features for different situations you may come across. You’ll also learn how to use requests in an efficient way as well as how to prevent requests to external services from slowing down your application.

In this tutorial, you’ll learn how to:

• Make requests using the most common HTTP methods
• Customize your requests’ headers and data, using the query string and message body
• Inspect data from your requests and responses
• Make authenticated requests
• Configure your requests to help prevent your application from backing up or slowing down

Though I’ve tried to include as much information as you need to understand the features and examples included in this article, I do assume a very basic general knowledge of HTTP. That said, you still may be able to follow along fine anyway.

Now that that is out of the way, let’s dive in and see how you can use requests in your application!

$ pipenv install requests

>>> requests.get('https://api.github.com')
<Response [200]>

>>> response = requests.get('https://api.github.com')

>>> response.status_code
200

if response.status_code == 200:
    print('Success!')
elif response.status_code == 404:
    print('Not Found.')

if response:
    print('Success!')
else:
    print('An error has occurred.')

import requests
from requests.exceptions import HTTPError

for url in ['https://api.github.com', 'https://api.github.com/invalid']:
    try:
        response = requests.get(url)

        # If the response was successful, no Exception will be raised
        response.raise_for_status()
    except HTTPError as http_err:
        print(f'HTTP error occurred: {http_err}')  # Python 3.6
    except Exception as err:
        print(f'Other error occurred: {err}')  # Python 3.6
    else:
        print('Success!')

>>> response = requests.get('https://api.github.com')
>>> response.content
b'{"current_user_url":"https://api.github.com/user","current_user_authorizations_html_url":"https://github.com/settings/connections/applications{/client_id}","authorizations_url":"https://api.github.com/authorizations","code_search_url":"https://api.github.com/search/code?q={query}{&page,per_page,sort,order}","commit_search_url":"https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}","emails_url":"https://api.github.com/user/emails","emojis_url":"https://api.github.com/emojis","events_url":"https://api.github.com/events","feeds_url":"https://api.github.com/feeds","followers_url":"https://api.github.com/user/followers","following_url":"https://api.github.com/user/following{/target}","gists_url":"https://api.github.com/gists{/gist_id}","hub_url":"https://api.github.com/hub","issue_search_url":"https://api.github.com/search/issues?q={query}{&page,per_page,sort,order}","issues_url":"https://api.github.com/issues","keys_url":"https://api.github.com/user/keys","notifications_url":"https://api.github.com/notifications","organization_repositories_url":"https://api.github.com/orgs/{org}/repos{?type,page,per_page,sort}","organization_url":"https://api.github.com/orgs/{org}","public_gists_url":"https://api.github.com/gists/public","rate_limit_url":"https://api.github.com/rate_limit","repository_url":"https://api.github.com/repos/{owner}/{repo}","repository_search_url":"https://api.github.com/search/repositories?q={query}{&page,per_page,sort,order}","current_user_repositories_url":"https://api.github.com/user/repos{?type,page,per_page,sort}","starred_url":"https://api.github.com/user/starred{/owner}{/repo}","starred_gists_url":"https://api.github.com/gists/starred","team_url":"https://api.github.com/teams","user_url":"https://api.github.com/users/{user}","user_organizations_url":"https://api.github.com/user/orgs","user_repositories_url":"https://api.github.com/users/{user}/repos{?type,page,per_page,sort}","user_search_url":"https://api.github.com/search/users?q={query}{&page,per_page,sort,order}"}'

>>> response.text
'{"current_user_url":"https://api.github.com/user","current_user_authorizations_html_url":"https://github.com/settings/connections/applications{/client_id}","authorizations_url":"https://api.github.com/authorizations","code_search_url":"https://api.github.com/search/code?q={query}{&page,per_page,sort,order}","commit_search_url":"https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}","emails_url":"https://api.github.com/user/emails","emojis_url":"https://api.github.com/emojis","events_url":"https://api.github.com/events","feeds_url":"https://api.github.com/feeds","followers_url":"https://api.github.com/user/followers","following_url":"https://api.github.com/user/following{/target}","gists_url":"https://api.github.com/gists{/gist_id}","hub_url":"https://api.github.com/hub","issue_search_url":"https://api.github.com/search/issues?q={query}{&page,per_page,sort,order}","issues_url":"https://api.github.com/issues","keys_url":"https://api.github.com/user/keys","notifications_url":"https://api.github.com/notifications","organization_repositories_url":"https://api.github.com/orgs/{org}/repos{?type,page,per_page,sort}","organization_url":"https://api.github.com/orgs/{org}","public_gists_url":"https://api.github.com/gists/public","rate_limit_url":"https://api.github.com/rate_limit","repository_url":"https://api.github.com/repos/{owner}/{repo}","repository_search_url":"https://api.github.com/search/repositories?q={query}{&page,per_page,sort,order}","current_user_repositories_url":"https://api.github.com/user/repos{?type,page,per_page,sort}","starred_url":"https://api.github.com/user/starred{/owner}{/repo}","starred_gists_url":"https://api.github.com/gists/starred","team_url":"https://api.github.com/teams","user_url":"https://api.github.com/users/{user}","user_organizations_url":"https://api.github.com/user/orgs","user_repositories_url":"https://api.github.com/users/{user}/repos{?type,page,per_page,sort}","user_search_url":"https://api.github.com/search/users?q={query}{&page,per_page,sort,order}"}'

>>> response.encoding = 'utf-8' # Optional: requests infers this internally
>>> response.text
'{"current_user_url":"https://api.github.com/user","current_user_authorizations_html_url":"https://github.com/settings/connections/applications{/client_id}","authorizations_url":"https://api.github.com/authorizations","code_search_url":"https://api.github.com/search/code?q={query}{&page,per_page,sort,order}","commit_search_url":"https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}","emails_url":"https://api.github.com/user/emails","emojis_url":"https://api.github.com/emojis","events_url":"https://api.github.com/events","feeds_url":"https://api.github.com/feeds","followers_url":"https://api.github.com/user/followers","following_url":"https://api.github.com/user/following{/target}","gists_url":"https://api.github.com/gists{/gist_id}","hub_url":"https://api.github.com/hub","issue_search_url":"https://api.github.com/search/issues?q={query}{&page,per_page,sort,order}","issues_url":"https://api.github.com/issues","keys_url":"https://api.github.com/user/keys","notifications_url":"https://api.github.com/notifications","organization_repositories_url":"https://api.github.com/orgs/{org}/repos{?type,page,per_page,sort}","organization_url":"https://api.github.com/orgs/{org}","public_gists_url":"https://api.github.com/gists/public","rate_limit_url":"https://api.github.com/rate_limit","repository_url":"https://api.github.com/repos/{owner}/{repo}","repository_search_url":"https://api.github.com/search/repositories?q={query}{&page,per_page,sort,order}","current_user_repositories_url":"https://api.github.com/user/repos{?type,page,per_page,sort}","starred_url":"https://api.github.com/user/starred{/owner}{/repo}","starred_gists_url":"https://api.github.com/gists/starred","team_url":"https://api.github.com/teams","user_url":"https://api.github.com/users/{user}","user_organizations_url":"https://api.github.com/user/orgs","user_repositories_url":"https://api.github.com/users/{user}/repos{?type,page,per_page,sort}","user_search_url":"https://api.github.com/search/users?q={query}{&page,per_page,sort,order}"}'

>>> response.json()
{'current_user_url': 'https://api.github.com/user', 'current_user_authorizations_html_url': 'https://github.com/settings/connections/applications{/client_id}', 'authorizations_url': 'https://api.github.com/authorizations', 'code_search_url': 'https://api.github.com/search/code?q={query}{&page,per_page,sort,order}', 'commit_search_url': 'https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}', 'emails_url': 'https://api.github.com/user/emails', 'emojis_url': 'https://api.github.com/emojis', 'events_url': 'https://api.github.com/events', 'feeds_url': 'https://api.github.com/feeds', 'followers_url': 'https://api.github.com/user/followers', 'following_url': 'https://api.github.com/user/following{/target}', 'gists_url': 'https://api.github.com/gists{/gist_id}', 'hub_url': 'https://api.github.com/hub', 'issue_search_url': 'https://api.github.com/search/issues?q={query}{&page,per_page,sort,order}', 'issues_url': 'https://api.github.com/issues', 'keys_url': 'https://api.github.com/user/keys', 'notifications_url': 'https://api.github.com/notifications', 'organization_repositories_url': 'https://api.github.com/orgs/{org}/repos{?type,page,per_page,sort}', 'organization_url': 'https://api.github.com/orgs/{org}', 'public_gists_url': 'https://api.github.com/gists/public', 'rate_limit_url': 'https://api.github.com/rate_limit', 'repository_url': 'https://api.github.com/repos/{owner}/{repo}', 'repository_search_url': 'https://api.github.com/search/repositories?q={query}{&page,per_page,sort,order}', 'current_user_repositories_url': 'https://api.github.com/user/repos{?type,page,per_page,sort}', 'starred_url': 'https://api.github.com/user/starred{/owner}{/repo}', 'starred_gists_url': 'https://api.github.com/gists/starred', 'team_url': 'https://api.github.com/teams', 'user_url': 'https://api.github.com/users/{user}', 'user_organizations_url': 'https://api.github.com/user/orgs', 'user_repositories_url': 'https://api.github.com/users/{user}/repos{?type,page,per_page,sort}', 'user_search_url': 'https://api.github.com/search/users?q={query}{&page,per_page,sort,order}'}

import requests

# Search GitHub's repositories for requests
response = requests.get(
    'https://api.github.com/search/repositories',
    params={'q': 'requests+language:python'},
)

# Inspect some attributes of the `requests` repository
json_response = response.json()
repository = json_response['items'][0]
print(f'Repository name: {repository["name"]}')  # Python 3.6+
print(f'Repository description: {repository["description"]}')  # Python 3.6+

>>> requests.get(
...     'https://api.github.com/search/repositories',
...     params=[('q', 'requests+language:python')],
... )
<Response [200]>

>>> requests.get(
...     'https://api.github.com/search/repositories',
...     params=b'q=requests+language:python',
... )
<Response [200]>

>>> requests.post('https://httpbin.org/post', data={'key':'value'})
>>> requests.put('https://httpbin.org/put', data={'key':'value'})
>>> requests.delete('https://httpbin.org/delete')
>>> requests.head('https://httpbin.org/get')
>>> requests.patch('https://httpbin.org/patch', data={'key':'value'})
>>> requests.options('https://httpbin.org/get')

>>> response = requests.head('https://httpbin.org/get')
>>> response.headers['Content-Type']
'application/json'

>>> response = requests.delete('https://httpbin.org/delete')
>>> json_response = response.json()
>>> json_response['args']
{}

>>> requests.post('https://httpbin.org/post', data={'key':'value'})
<Response [200]>

>>> requests.post('https://httpbin.org/post', data=[('key', 'value')])
<Response [200]>

>>> response = requests.post('https://httpbin.org/post', json={'key':'value'})
>>> json_response = response.json()
>>> json_response['data']
'{"key": "value"}'
>>> json_response['headers']['Content-Type']
'application/json'

>>> response = requests.post('https://httpbin.org/post', json={'key':'value'})
>>> response.request.headers['Content-Type']
'application/json'
>>> response.request.url
'https://httpbin.org/post'
>>> response.request.body
b'{"key": "value"}'

>>> from getpass import getpass
>>> requests.get('https://api.github.com/user', auth=('username', getpass()))
<Response [200]>

>>> requests.get('https://api.github.com/user')
<Response [401]>

>>> from requests.auth import HTTPBasicAuth
>>> from getpass import getpass
>>> requests.get(
...     'https://api.github.com/user',
...     auth=HTTPBasicAuth('username', getpass())
... )
<Response [200]>

import requests
from requests.auth import AuthBase

class TokenAuth(AuthBase):
    """Implements a custom authentication scheme."""

    def __init__(self, token):
        self.token = token

    def __call__(self, r):
        """Attach an API token to a custom auth header."""
        r.headers['X-TokenAuth'] = f'{self.token}'  # Python 3.6+
        return r


requests.get('https://httpbin.org/get', auth=TokenAuth('12345abcde-token'))

>>> requests.get('https://api.github.com', verify=False)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>

>>> requests.get('https://api.github.com', timeout=1)
<Response [200]>
>>> requests.get('https://api.github.com', timeout=3.05)
<Response [200]>

>>> requests.get('https://api.github.com', timeout=(2, 5))
<Response [200]>

import requests
from requests.exceptions import Timeout

try:
    response = requests.get('https://api.github.com', timeout=1)
except Timeout:
    print('The request timed out')
else:
    print('The request did not time out')

import requests
from getpass import getpass

# By using a context manager, you can ensure the resources used by
# the session will be released after use
with requests.Session() as session:
    session.auth = ('username', getpass())

    # Instead of requests.get(), you'll use session.get()
    response = session.get('https://api.github.com/user')

# You can inspect the response just like you did before
print(response.headers)
print(response.json())

import requests
from requests.adapters import HTTPAdapter
from requests.exceptions import ConnectionError

github_adapter = HTTPAdapter(max_retries=3)

session = requests.Session()

# Use `github_adapter` for all requests to endpoints that start with this URL
session.mount('https://api.github.com', github_adapter)

try:
    session.get('https://api.github.com')
except ConnectionError as ce:
    print(ce)

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Making HTTP Requests With Python