Certificate error python

I apologize if this is a silly question, but I have been trying to teach myself how to use BeautifulSoup so that I can create a few projects.

I was following this link as a tutorial: https://www.youtube.com/watch?v=5GzVNi0oTxQ

After following the exact same code as him, this is the error that I get:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1240, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1079, in endheaders
self._send_output(message_body)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 911, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 854, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1237, in connect
server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 376, in wrap_socket
_context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 747, in __init__
self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "WorldCup.py", line 3, in <module>
    x = urllib.request.urlopen('https://www.google.com')
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 162, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 465, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 483, in _open
'_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 443, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1283, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1242, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]     certificate verify failed (_ssl.c:645)>

Can someone help me figure out how to fix this?

SSL certificate_verify_failed errors typically occur as a result of outdated Python default certificates or invalid root certificates. We will cover how to fix this issue in 4 ways in this article.

Why certificate_verify_failed  happen?

The SSL connection will be established based on the following process.   We will get errors if any of these steps does not go well.

For this error certificate_verify_failed, it usually happens during step 2 and step 3.

• The client sends a request to the server for a secure session. The server responds by sending its X.509 digital certificate to the client.
• The client receives the server’s X.509 digital certificate.
• The client authenticates the server, using a list of known certificate authorities.
• The client generates a random symmetric key and encrypts it using server’s public key.
• The client and server now both know the symmetric key and can use the SSL encryption process to encrypt and decrypt the information contained in the client request and the server response.

When the client receives the server’s certificate, it begins chaining that certificate back to its root. It will begin by following the chain to the intermediate that has been installed, from there it continues tracing backwards until it arrives at a trusted root certificate.

If the certificate is valid and can be chained back to a trusted root, it will be trusted. If it can’t be chained back to a trusted root, the browser will issue a warning about the certificate.

Related: Check SSL Certificate Chain with OpenSSL Examples

Error info about certificate_verify_failed

We will see the following error.

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

What is SSL certificate

Server certificates are the most popular type of X.509 certificate. SSL/TLS certificates are issued to hostnames (machine names like ‘ABC-SERVER-02’ or domain names like google.com).

A server certificate is a file installed on a website’s origin server. It’s simply a data file containing the public key and the identity of the website owner, along with other information. Without a server certificate, a website’s traffic can’t be encrypted with TLS.

Technically, any website owner can create their own server certificate, and such certificates are called self-signed certificates. However, browsers do not consider self-signed certificates to be as trustworthy as SSL certificates issued by a certificate authority.

Related: 2 Ways to Create self signed certificate with Openssl Command

How to fix certificate_verify_failed?

If you receive the “certificate_verify_failed” error when trying to connect to a website, it means that the certificate on the website is not trusted. There are a few different ways to fix this error.

We will skip the SSL certificate check in the first three solutions.  For the fourth solution, we are going to install the latest CA certificate from certifi.

Create unverified context in SSL

import ssl
context = ssl._create_unverified_context()
urllib.request.urlopen(req,context=context)

Create unverified https context in SSL

import ssl
ssl._create_default_https_context = ssl._create_unverified_context
urllib2.urlopen(“https://google.com”).read()

Use requests module and set ssl verify to false

requests.get(url, headers=Hostreferer,verify=False)

Update SSL certificate with PIP

we can also update our SSL certificate With PIP.  All we would have to do is  to update our SSL certificate directory with the following piece of code: pip install –upgrade certifi

What this command does is update our system’s SSL certificate directory.

Reference:

Understanding SSL certificates

Check SSL Certificate Chain with OpenSSL Examples

5 ways to check SSL Certificate

We know setting up SSL certificates with Python can be confusing — that’s
why we’re here to help

When you’re dealing with Python or any programming language at all, there’s plenty of room for mistakes to be made or technical errors to occur. Among these potential errors is the Python SSL “certificate_verify_failed” error. Getting this error can be frustrating, especially if you’ve done your best to ensure that everything is done right.

When dealing with this error, it’s
important to know that it isn’t hard to solve — but it does require patience.

That being said, before you can
fix the issue, you need to understand why it occurs in the first place. Let’s
dive into the reasons this error occurs, as well as what you can do to address
it.

SSL certificate_verify_failed errors typically occur as a result of outdated Python default certificates or invalid root certificates. If you’re a website owner and you’re receiving this error, it could be because you’re not using a valid SSL certificate. Here’s where you can get one:

Since this error is usually paired
to web page scrapers in Python, let’s assume that this is a typical scenario
where the error happens:

First, imagine you’re trying to
scrape a page. You fire the scraper up, only to be met with an error page.

Don’t worry, though. This issue
can be resolved with a simple command, which we’ll get to shortly.

But what causes the error? The
issue comes from your web browser attempting to download a program that it will
not let it download because of the expired SSL certificates that came
with your version of Python. (Since that version of SSL is no longer deemed
“safe” by Python, your end users receive the warning message.)

How Can I Fix the SSL Certificate_verify_failed Error?

Some people might suggest that you
simply disable the certificate verification function. But this tactic not only
fails to resolve the issue, but also means that you’re no longer verifying the
certificate, which can lead to a variety of other issues.

To fix this this problem, you may
need to upgrade your SSL certificate directory. The most common way to do so is
to use the following PIP code.

PIP,
which stands for “Python Package Installer,” is exactly how it sounds — it’s a
package installer for Python. This command allows for easy installation of
packages — or, in this case, our updated SSL certificates. With PIP, all you
would have to do to update your SSL certificate directory is input the
following piece of code:

pip install --upgrade certifi

What
this command does is update your system’s SSL certificate directory. This allows
you to download the files that were previously being denied as a result of the
lack of an SSL certificate (which, in this case, was the page scraper).

After
executing the code, the error should be gone. 
That wasn’t so hard, was it?

Troubleshooting Guides

• How to Resolve SSL_ERROR_RX_RECORD_TOO_LONG as a Site Visitor
• How to Fix the ERR_SSL_PROTOCOL_ERROR in 8 Easy Steps (2020 Edition)