Urn ietf params acme error unauthorized the client lacks sufficient authorization

Добрый день, не получается сгенерировать сертификат из-за ошибки:

Failed authorization procedure. rukzak3d.ru (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from domain.ru/.well-known/acme-challenge/Iurj9ChNH9JNH… [82.146.55.40]: «rn404 Not Foundrnrn404 Not Foundrnnginx/1.19.0»

Читаю, логи прихожу к выводу, что механизм работы подтверждения домена следующий:

1) Certbot создает конфиг nginx вида:
location /.well-known/acme-challenge/Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0 {
default_type «text/plain»;
return 200 Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0.DyD11cIvsF3VQsyoJH3me_in7zC0mwIq5myzx10Pa3c;
}

2) Подключает его в самое начало блока http в основном конфиге /etc/nginx/nginx.conf

http {
include /etc/letsencrypt/le_http_01_cert_challenge.conf;

3) Идет по адресу domain.ru/.well-known/acme-challenge/Iurj9ChNH9JNH… и проверяет код ответа, если не 200 то выдает ошибку.

Сейчас конфиг /etc/nginx/nginx.conf такой:

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {

    server {
        charset utf-8;

        listen 443;

        keepalive_timeout   120;


        index index.php index.html;

        root /var/www/sites/domain/web;

        server_name domain.ru;

        index index.php;

        location / {
            try_files $uri /index.php?$args;
        }

        location ^~ /.well-known/acme-challenge/ {
            default_type "text/plain";
            root /var/www/sites/domain/web;
        }
    }
}

Если я в секцию server добавляю код:

http {
...
   server {
        ....
        location /.well-known/acme-challenge/Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0 {
           default_type "text/plain";
           return 200 Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0.DyD11cIvsF3VQsyoJH3me_in7zC0mwIq5myzx10Pa3c;   
        }
}

То браузер по адресу domain.ru/.well-known/acme-challenge/Iurj9ChNH9JNH… отвечает 200-м кодом.
Если вставить в том место где подключается конфиг certbot’м, т.е. в начало секции http:

http {
        location /.well-known/acme-challenge/Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0 {
           default_type "text/plain";
           return 200 Iurj9ChNH9JNHUtLUxc_6A0Yo4HS8AoEshvPTx_JXo0.DyD11cIvsF3VQsyoJH3me_in7zC0mwIq5myzx10Pa3c;   
        }
...
   server {
        ....
}

Получаем 404 ошибку

Вопрос, как заставить nginx отвечать 200м кодом при подключенном /etc/letsencrypt/le_http_01_cert_challenge.conf в начало секции http? Или как заставить certbot помещать конфиг в секцию server?

♻️ Updated: November 27, 2022

Context

Here’s some things you need to know before reading any further.

• Purchased to Google Domains
• Correctly configured the Nameserver to point to my VPS
• dig and nslookup commands return correct DNS configuration
• Apache on Ubuntu

Failed authorization procedure

I spend almost 2 days trying to find a solution to Let’s Encrypt certificates not being fully installed under Ubuntu’s with Apache.

I would run the ./certbot-auto and as soon as we got to the Cleaning up challenges step it would output something like this:

From what I could gather there are multiple things that can trigger this error. Permissions, badly configured domain, closed ports.

Apache 403 Forbidden

I tried everything I could find without any luck. Giving permissions to the manually created /.well-known/acme-challenge/ folder or trying to serve a file from the outside. It was working, no problems there.

However, during the process I would still get the irritating 403.

Specify a webroot path

I still don’t know if it was because the domain has this cool «.club» ending instead of the typical «.com» but I needed to specify —webroot-path or its alias -w.

To not reach my rate limit I did add —dry-run to the command to make it sure I got it right before requesting the certificate for real.

I’m very inclined to think it’s the domain itself. I have 3 other domains (ending on .com and .cat) running in the same VPS server, on the same Apache virtualhost configuration and they can be renewed without having to specify the webroot path.

Here’s the magical command:

After running this, the only thing left to do is to add the chain and key file to the site apache virtualhost configuration.

Message from the bot:

Add the following lines:

I hope this might help you out. Good luck!

UPDATE (November 19th): the —apache plugin when running the bot no longer works for any of my domains. I’m now using —webroot The downside is that I have to manually copy and paste the certificate to each virtual host file (even if I just have 1 certificate with all the domains).

UPDATE (April 2019): Be careful if the domain or subdomain has a forwarding from the :80 port to the HTTPS :443
My previous rewrite rules were causing problems (unauthorized).
This rule in my .conf file works:

Principal Frontend Software Engineer at Cognizant Netcentric, from Barcelona, Haidong Gumdo Master (Korean martial art of the sword), street photographer, travel lover, podcaster, TV addict, Boston Red Sox fan, and privacy advocate.

Источник

Common Certbot Errors

Last updated: July 6th 2022

SSL Certificate Guides

Introduction

Let’s Encrypt is a free and open-source Certificate Authority managed by the Internet Security Research Group. It uses Automated Certificate Management Environment (ACME) server to validate the domain and deploy free SSL certificates automatically that are trusted by all major browsers. The main goal of this project to make encrypted connections throughout the Internet.

Let’s Encrypt provides an easy way to install and deploy SSL certificate for your website for free using a command-line tool called Certbot and is fully supported by Webdock natively in our control panel

Let’s Encrypt Certbot sometimes kicks up a fuss however for a variety of reasons. In this article we document the most commonly encountered errors we see on our platform and how to solve them.

What to always try in Webdock

Regardless of the specific error you are encountering, in 99% of all cases the following operations in Webdock clear things up nicely:

1. Make sure Certbot is up to date, if you are on a Bionic stack or older you should make sure that Certbot runs with the new Snap version. Run the Upgrade Certbot to Snap script on your server, see our article on the subject here.
2. Set the Web Root again, in order to make sure your webserver and Certbot config agree on the location of your web root (access this feature by clicking on the small pencil icon next to the web root as displayed in the Webdock dashboard)
3. Run the Server Identity tool in order to make sure your webserver and Certbot know about all your domains
4. Finally, run the SSL Certificates tool again — now Certbot should complete succesfully in almost all cases

If you are still seeing issues — especially timeout/connection issues on older Ubuntu systems — please try the following as well:

1. Update all packages, by running apt update and apt upgrade
2. Reboot the server, As this may help clear problems with connectivity especially on older Xenial based servers.
3. Finally, run the SSL Certificates tool again

If despite all these common steps which usually resolve Certbot issues you are still having problems — please find your error below and follow our suggestions in order to resolve that specific problem.

Make sure your Certbot installation is up to date

As of late 2020 Certbot is no longer maintained as an apt package and you should make sure it is running through Snap on Ubuntu. This procedure is totally safe and easy to do.

Please read our full article on this topic in order to get up to speed.

Checking if your Let’s Encrypt Certificate is working

Typically Certbot runs fine in Webdock the first time you run it and problems crop up over time or when you do changes to your configuration. Often the first indication you get of a problem is when you get an email from Let’s Encrypt with a title similar to «Let’s Encrypt certificate expiration notice for domain xxx»

When you generate certificates with Webdock we automatically add a Cron job which keeps your certificate up to date. If you get this email then this means your certificate may be about to expire and you need to check if renewals are working.

In Webdock you can simply run the «Test Certbot renewal» script on your server on the Server Scripts page.

What this script essentially does is runs the following command on your server and returns the output so you can inspect what is happening

In the following sections we detail various error messages which may show up here and what to do about them.

Invalid response / The client lacks sufficient authorization

If you get a message similar to the following

Try the suggested fix: Check if your DNS records are OK

First thing to check are your DNS records and whether your A and AAAA records are OK and pointing to your server. However, if your certificate has worked before and you reach your website just fine, then it is unlikely this is the culprit.

However, you should keep an eye on whether there are any web forwards configured (some DNS providers allow this) e.g. if you forward www to non-www or vice-versa, this may trip up Certbot. In which case remove the domain you are forwarding using DNS from your certificate. This should resolve the issue.

To remove a domain in Webdock you simply run Server Identity with the domains you want your webserver to actually respond to and then run the SSL Certificates tool once more.

On Apache: Make sure you exclude Certbot from any .htaccess rewrites

If you are on Apache, check if you have a .htaccess file with rewrites in your web root. If you have such a file you should place a rule to exclude Let’s Encrypt Certbot from any rewrites by placing the exception before any RewriteCond statements. For example:

Once you’ve placed the acme-challenge RewriteRule in there, try running Certbot again.

On Apache: Try rolling back completely and nuking any Certbot config

If your DNS records and rewrites are ok and Certbot renew still fails, you should try and issue the certbot rollback command:

If this gives you errors, try removing the Let’s Encrypt SSL configuration file located at (in default Webdock stacks):

Next run Certbot rollback — and if it succeeds keep running it untill it says it hasn’t touched your configuration. You should now be back to a stock configuration and you should be able to run the SSL Certificates tool once more to generate a new certificate.

If you are not using Webdock and following this guide, you should make sure you back up your configuration before doing this and make sure your config is as you want it and working for vanilla http before running Certbot.

On Nginx: Make sure you exclude Certbot from any deny rules

Similar to Apache, if you have custom configuration which excludes access to hidden directories, such as the following:

You need to add the following rule beofre this in order to allow Certbot to validate using the web root method:

Error while running apache2ctl configtest

This error is related to the last fix detailed above and can be solved in the same manner, if you encounter a message like the following:

Simply delete the old config and try again

During the Webdock Let’s Encrypt SSL installation, a Certbot rollback command hasn’t completely cleared an old default config file. You can resolve this by removing the file /etc/apache2/sites-enabled/000-default-le-ssl.conf and run Certbot once more.

No such file or directory error from Nginx

Sometimes during a Certbot rollback operation or when Certbot tries to renew/install a certificate, Nginx cannot start the webserver as there is till old Certbot configuration hanging around in your Nginx vhost config file. You may see an error like the following:

To solve this issue, remove any references to SSL certificates from your Nginx configuration file. The lines typically look like this:

Once you have removed these lines run Certbot rollback untill it says nothing was modified. If you are using Webdock you would then run the Server Identity tool and finally the SSL Certificates tool and you will now have a shiny new (and working) certificate installed on your server.

If you are not using Webdock, you would likely issue another certbot rollback command and see if it succeeds, next review your config and make sure it is as you want it and that it works for vanilla http, finally you would run Certbot again to generate your certifcate.

Timeout Errors

If you get an error similar to the following:

You need to check whether your DNS records are OK and whether your webserver is listening on your server on port 80 and 443. Make sure you check whether you can ping and ping6 your domain. Typically we see this error happening is if AAAA records for a domain have not been updated, or there are old records pointing to a wrong IP hanging around — but a timeout might also indicate a networking issue (for example a firewall) as stated in the message, or that your server is actively refusing connections for some reason.

Another cause for this issue is that Nginx and Certbot are not aligned as to where your web root is located and Certbot is placing its verification files for web root authentication in the wrong place. A quick way to fix this in Webdock is to simply set the Web Root once more (the small pencil icon next to the web root in the Webdock dashboard) and then run renewal again.

The error may also look something like this:

In our experience, this specific message is due to Let’s Encrypt not being able to connect via. IPv6 — so check that IPv6 works if you have AAAA records for your domain, otherwise all the advice above is also applicable, i.e. check your DNS and connection issues in general. On some older Ubuntu systems (Xenial especially) a general package upgrade with apt update and apt upgrade has been seen by us as an easy resolution to some ipv6 connection issues.

Connection Refused Error

If you get a Connection Refused error when doing renewals, this may be due to old-style Certbot configuration not having written the appropriate configuration to your Nginx config in order to allow Nginx to listen on your IPv6 address. You would typically see an error like the following:

What this error really is saying is: Certbot could not connect to your domain on port 443. Typically what we see is that if you now check what is listening on which ports by running netstat you will get something like the following:

Notice how nginx is listening on tcp (ipv4) port 443 but not tcp6 (ipv6) ? This is the problem as the Let’s Encrypt certificate servers prefer to connect through ipv6 if there are AAAA records set up for your domain.

In order to solve this, check that you have the following line in your Nginx config (on Webdock servers the vhostconfig file is typically found at /etc/nginx/sites-enabled/webdock) , and if not, add it:

Next, restart Nginx:

Check that nginx is listening on tcp6 port 443 and try your renewal again. It should work this time — but if not, then make sure nginx is listening on the appropriate interfaces and on both ports 80 and 443 and is reachable from the outside, because if it is not you will get this error message.

Account creation on ACMEv1 is disabled

If you get an error message similar to the following

This simply means that the installed version of Certbot is too old and doesn’t support newer authentication methods. Simply upgrade your system with:

If it prompts you what to do about existing config files just choose the defaults which is to keep the existing configuration. Once the system is upgraded it is a good idea to reboot the system, or at the very least your webserver and php-fpm before trying again.

Please note: During the system upgrade your webserver will likely go down temporarily. Plan accordingly.

The request message was malformed

Similarily an indication your Certbot is too old may be seen from the following Certbot output:

The solution here is, as with the error above, to upgrade your system. The same warning applies: upgrading may bring down your webserver temporarily.

Test Certbot Renewal script hangs for a long time, prompts for a new webroot in output

Sometimes Webdock users experience that the Test Certbot renewal scripts hangs for a long time and then outputs something similar to the following:

This is due to Certbot having «forgotten» what the appropriate web root is for your domain. We use web root authentication in order to avoid certain other issues with Cloudflare and Cloudflare caching. In any case, the solution here is to set the web root once more on your server, after which you should run the SSL Certificates tool once more. If you still get errors, try running the Server Identity tool first.

We have servers in:

Источник

Background

This is a complex issue and the interplay between various technologies such as DNS, HTTP Challenge, website (Apache / NGINX), permissions, and redirections can make the problem incredibly hard to troubleshoot.

Tips

Carefully evaluate the error. Specifically looks for the `Invalid response from` string as you might be getting a completely different reply that you would expect.

Original Article with Amendments

On a Virtualmin server, refreshing a Let’s Encrypt certificate gives the error. We have highlighted in bold the note to watch for in the output as there could be many different failed outcomes for Let’s Encrypt, e.g.

• 404
• Unauthorized

This log file covers Unauthorized

Requesting a certificate for domain.com from Let's Encrypt ..
.. request failed : Web-based validation failed :

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for domain.com
Using the webroot path /home/domain.com/public_html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. domain.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://domain.com/.well-known/acme-challenge/A3oiNXYoxM6Bwze9iTWC0KoxCf-JzF49ozlwJpVzucc [52.215.112.96]: "<!DOCTYPE html>n<html lang="en">n <head>n <meta charset="utf-8">n <meta name="viewport" content="width=device-w"
IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: domain.com
Type: unauthorized
Detail: Invalid response from
http://domain.com/.well-known/acme-challenge/A3oiNXYoxM6Bwze9iTWC0KoxCf-JzF49ozlwJpVzucc
[x.y.112.96]: "<!DOCTYPE html>n<html lang="en">n <head>n
<meta charset="utf-8">n <meta name="viewport"
content="width=device-w"

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

The error was because the NGINX file had directives that wouldn’t let Let’s Encrypt complete. A radical approach worked by refreshing the NGINX configuration but Edit Virtual Server, remove NGINX and SSL NGINX and then re-add. However, the next time this happened another path was chosen:

Revert to Certbot when all else fails

For this specific instance Certbot was chosen instead. One possible reason why Certbot might work better from the command line is this:

Using the webroot path /home/domain.com/public_html for all unmatched domains.

If you’ve modified your webroot, which is often the case for a YII or Laravel application, that path will not be valid anymore.

So instead of using the UI, we used the following command logged in as Root:

sudo certbot --nginx -d domainname.com

To get Certbot to automatically renew, see this article:

https://certbot.eff.org/docs/using.html?highlight=renew#automated-renewals

How Virtualmin’s Let’s Encrypt HTTP Challenge Works

To gain understanding of Let’s Encrypt’s algorithm, it’s broadly like this:

1. Trigger a new SSL install
2. The source server receives the challenge and write a temporary file
3. Web server contacts the Let’s Encrypt Server to initiate request
4. Let’s Encrypt connects back to the source server to see if the challenge is valid find the temporary file
5. If point 4 succeed, request is successful

Where if normally fails is point 4, for whatever reason. The biggest noob mistake is no DNS record pointing to the source server, and then using something such a Cloudflare also needs special treatment because Cloudflare intercepts DNS.

What about Well Known stuff?

Let’s Encrypt HTTP Challenge likes to write to local disk. In that case this should be there:

location ~ /.(?!well-known).* {
        deny all;
    }

Missing index.php

It was a Laravel website, so I also had to add both:

index index.html index.php

and

location / {
   try_files $uri $uri/ /index.php?$query_string;
}

to get the site running.

I believe the location parameter broke the well known location. This error is hard to troubleshoot, see multiple references all with different answers below:

References

• https://www.virtualmin.com/node/58287
• https://www.virtualmin.com/node/56388
• https://laravel.com/docs/deployment#nginx

If you are still having problems with Let’s Encrypt and Virtual, contact us on 27 82 309-6710 to sort out.

I’m trying to use certbot to obtain an SSL certificate for one of my subdomains. However, one of the challenges fails when trying to test .well-known/acme-challenges/<token>. The web server (nginx) returns 404. The precise error is:

Obtaining a new certificate
Performing the following challenges:
http-01 challenge for git.domain.com
http-01 challenge for www.git.domain.com
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.git.domain.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.foo.domain.com/.well-known/acme-challenge/eXpa7Ub3slbohHh0AZZA-aACo70p15KkJS05aYsN2bY [my-ip-addr]: "<html>rn<head><title>404 Not Found</title></head>rn<body bgcolor="white">rn<center><h1>404 Not Found</h1></center>rn<hr><center>", git.domain.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://git.domain.com/.well-known/acme-challenge/WxYfL5t0vLNe7jiIF2TFz1sXyQBH3RcPIVz5de9lQ8M [my-ip-addr]: "<html>rn<head><title>404 Not Found</title></head>rn<body bgcolor="white">rn<center><h1>404 Not Found</h1></center>rn<hr><center>"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: www.git.domain.com
   Type:   unauthorized
   Detail: Invalid response from
   http://www.git.domain.com/.well-known/acme-challenge/eXpa7Ub3slbohHh0AZZA-aACo70p15KkJS05aYsN2bY
   [my-ip-addr]:
   "<html>rn<head><title>404 Not Found</title></head>rn<body
   bgcolor="white">rn<center><h1>404 Not
   Found</h1></center>rn<hr><center>"

   Domain: git.domain.com
   Type:   unauthorized
   Detail: Invalid response from
   http://git.domain.com/.well-known/acme-challenge/WxYfL5t0vLNe7jiIF2TFz1sXyQBH3RcPIVz5de9lQ8M
   [my-ip-addr]:
   "<html>rn<head><title>404 Not Found</title></head>rn<body
   bgcolor="white">rn<center><h1>404 Not
   Found</h1></center>rn<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

I have added into my config file:

location ^~ '/.well-known/acme-challenge' {
    allow all;
}

But this does nothing. The fact that it is getting a 404 is whats throwing me off. If it was a problem of nginx not allowing access to the file, then wouldn’t it throw a 403?

nginx -t shows no errors in my config. I have ensured that my DNS info is set up correctly.

Full config (nginx -T): https://pastebin.com/a6GcbXai

Last 25 lines of access.log after running certbot: https://pastebin.com/Lp72LWxC

Another thing that puzzles me is that I have 3 other subdomains running on this server, none of which I have had this problem with.

What is happening here, and how do I allow certbot to see this file, so I can get the certificate?