What to do if you fail to get a Let’s Encrypt certificate with Certbot

What to do if you fail to get a Let’s Encrypt certificate with Certbot

The main point of the problem

Error details:

• Error content: HTTP-01 challenge failed.
• Reason:Unauthorized: Invalid response from http://example.com/.well-known/acme-challenge/XXXXXX: 404 Let’s Encrypt’s server .well-known/acme-challenge/ was unable to verify the correct token for and returned an HTTP 404 error.

Check the list of Certbot certificates

Check the list of installed certificates.

sudo certbot certificates

If the installation is successful, the following results will be returned:
If the correct result cannot be obtained, investigate the cause.

Certificate Name: example.com
    Domains: example.com sub.example.com
    Expiry Date: 2024-02-26 12:00:00+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem

Possible causes and confirmation points

Incorrect DNS settings for the domain: The domain example.com may not resolve to the correct server’s IP address. Make sure that the IP address matches the IP of the server that Certbot is running on.
How to check:

dig example.com A

Port 80 unavailable:

sudo netstat -tuln | grep :80

What to do: Stop Apache or Nginx
Stop Apache

sudo systemctl stop apache2 

Stop Nginx

sudo systemctl stop nginx

Firewall or Network Restrictions:

Check the status of the firewall:

sudo ufw status

If necessary, allow port 80:

sudo ufw allow 80

IPv6 Address Conflicts:

certbot certonly --standalone --preferred-challenges http -d example.com --agree-tos --email <メールアドレス> --http-01-port 80

Resolution step 1

Check DNS Settings:

Verify that the domain resolves to the IP address of the server running Certbot.

Check Port 80 Usage:

Stop or reconfigure other processes to ensure that port 80 is available for Certbot’s temporary servers.

Firewall Settings:

Allow external connections to port 80.

Retry the Certbot command:

After resolving the above issues, run Certbot again:

certbot certonly --standalone -d example.com --agree-tos --email ＜メールアドレス＞

Use of alternative challenge methods (last resort):

If the HTTP-01 challenge doesn’t work, use the DNS-01 challenge. This requires adding a specific TXT record to your DNS record.

Resolution Step 2 (DNS-01 Challenge):

summary

DNS challenge(dns-01) proves domain ownership by creating a DNS TXT record that Let’s Encrypt specifies.

merit

1. No ports required:
   • It does not use ports 80 or 443, so you can bypass network and firewall constraints.
2. Compatible with internal servers and private domains:
   • Even if the domain is not public, it can be used if DNS is operational.
3. Wildcard certificates are supported:
   • Wildcard (e.g. *.example.com) certificates can only be obtained by DNS challenges.

demerit

1. DNS configuration hassle:
   • You need to set up TXT records manually or automatically with your DNS provider.
2. Complexity of Automation:
   • To automate, you need to configure the corresponding APIs and tools.

Suitable environment

• If your network configuration is complex and ports 80 or 443 are not available.
• If you want to get a wildcard certificate.
• If you can control the DNS of your domain.

Start the DNS-01 Challenge:

certbot certonly --manual --preferred-challenges dns -d example.com

Follow the instructions to set up a TXT record in your DNS.

1. What is adding a TXT record?

A TXT record is a type of DNS record that stores arbitrary text data associated with a domain name. Let’s Encrypt’s DNS-01 challenge adds the token specified to issue the certificate as a TXT record.

2. DNS-01 Challenge Flow

Run the Certbot command:

certbot certonly --manual --preferred-challenges dns -d example.com

Check the instructions that appear:

Certbot displays a token (example) like this:

Please deploy a DNS TXT record under the name:
_acme-challenge.example.com

With the following value:
ABCXXXXXXXXXXXXXXXXX

where is_acme-challenge.example.com the name of the TXT record,ABCXXXXXXXXXXXXXXXXX is the value.

Add a TXT record in your DNS settings:

Log in to the control panel of the DNS provider that manages your domain (e.g., Google Domains, AWS Route 53, Cloudflare, etc.).

Go to the page where you want to manage your DNS records.

Add a new record:

Type: TXT

Name: _acme-challenge(or _acme-challenge.example.com)

Value:ABCXXXXXXXXXXXXXXXXX (Value provided by Certbot)

TTL: 300 seconds (example)

Return to Certbot and continue:

Wait a few minutes ~ hours for the settings to be reflected in your DNS.

Once you’re set up, proceed with the Certbot process. Certbot will check your DNS records and issue a certificate.

3. Can I use it with A Record?

• Can be used together.
• TXT records are records of different purposes from A records and do not interfere with each other.
• You can set up multiple records for the same domain (example.com) as follows:
  • A-record: Specify an IPv4 address (e.g. xxx.xxx.xxx.xxx)
  • TXT record: _acme-challenge Specify the token value of the

Example configuration

4. Check the reflection status of TXT records

To check if the reflection is complete, use the following command:

For Linux/MacOS (dig Commands)

dig _acme-challenge.example.com TXT

If the result contains the following values, it is a success:

_acme-challenge.example.com. 300 IN TXT "ABC00000000000000000"

Use online tools

• You can use the DNS Propagation Checker to check the status of reflections on DNS servers around the world.
  

External: DNS Propagation Checker Example

あわせて読みたい

DNS Propagation Checker – Global DNS Checker Tool Instant DNS Propagation Check. Global DNS Propagation Checker – Check DNS records around the world.

5. Notes

1. Reflection time:
   • DNS changes can take minutes ~ hours depending on TTL (Time to Live) settings and DNS providers.
2. Multiple TXT records:
   • If you want to set up multiple TXT records for the same domain, you’re good to go. However, be careful if you have set up other TXT records_acme-challenge with the name of the name, as this may cause conflicts.
3. Manual authentication effort:
   • The DNS-01 challenge of manually adding a TXT record is not suitable for automatic updates. If you want to renew your certificate regularly, consider setting it up to automate (for example, HTTP-01 challenge).

Resolution Step 3 (ALPN-01 Challenge):

summary

• Use HTTPS (port 443) to verify domain ownership.
• Utilize Application Layer Protocol Negotiation (ALPN) to verify certificate ownership during TLS handshakes.

merit

1. No Port 80 Required:
   • It would be nice if port 443 could be used.
2. Expedited Certificate Issuance:
   • Like HTTP-01, no manual operation is required.
3. Easy to automate:
   • Relatively easy to configure with tools like Certbot and acme.sh.

demerit

1. Port 443 is required:
   • Difficult to configure if port 443 is occupied by other services.
2. Wildcard certificates are not supported:
   • You can only issue certificates for a single domain or subdomain.

Suitable applications

• If port 80 is closed but port 443 is available.
• When a single domain or subdomain is sufficient.
• If your server is public and you want to choose HTTPS verification.

precondition

Installing acme.sh

You must have a acme.sh installed that supports ALPN challenges.
If not, install it with the following command:

curl https://get.acme.sh | sh

Port 443 available
The ALPN challenge uses port 443, so make sure that this port is not occupied by other services.

Your domain must be suitable for your server.
example.com points to the server’s IP address.

procedure

Issuing certificates
acme.sh to issue certificates in ALPN challenges.

command

~/.acme.sh/acme.sh --issue --alpn -d example.com

explanation

• --issue: Option to issue a certificate.
• --alpn: Using the ALPN Challenge.
• -d test.brain-ai-lab.com: Specify the domain.

Conclusion – Comparison Table

Why choose HTTP-01?

• There is a public web server and port 80 is available.
• It is enough to apply SSL to a single domain or subdomain.

Why choose DNS-01?

• Requires a wildcard certificate (e.g., *.example.com).
• The server is running privately (in the internal network).
• If port 80/443 is not available.

Why choose ALPN-01?

• If port 443 is available, but port 80 is closed.
• You want to issue certificates for a single domain or subdomain on a public server.