Saturday, March 25, 2023

How To Auto Renew SSL Certificates With Certbot Using DNS Challenge

If you have used certbot for automatic renewal of SSL certificates for your website using the HTTP challenge and are also running Technitium DNS Server to host your domain names then you can use certbot with DNS challenge to auto renew your SSL certificates. DNS challenge for certificate renewal has many advantages over HTTP challenge:

  • DNS challenge allows renewing wildcard certificate for your domain like *.example.com. This allows hiding your subdomain names such that they are not listed in publicly available Certificate Transparency logs. If you are not familier with this then do check your domain name on crt.sh website.
  • It allows renewing certificates for your internal servers i.e. servers in private LAN that are not accessible over the Internet to use the HTTP challenge. In fact, you do not even need to have a web server running to get the certificate issued.

The only constraint with using DNS challenge is to have either API access or Dynamic Updates (RFC 2136) access to your DNS provider. If you are using Technitium DNS Server to host your domain names, you have access to both. In this post we will discuss both the options you have and their pros and cons.

This post is written for Ubuntu Linux but, you can easily follow similar steps on your favorite distro.

Using HTTP API

Technitium DNS Server provides HTTP API which can be used to perform all the tasks that you can perform using the DNS web console. In fact, the DNS web console itself uses the same HTTP API. Certbot provides manual option for certificate renewal which also includes auth hook and cleanup hook options that you can utilize to create DNS records and remove them during the renewal process. We will be using these options to run a small bash script that will use the DNS server's HTTP API.

Follow the steps below to setup certbot that will use the DNS HTTP API to handle DNS challenge:

  1. Generate an API Token for the DNS Server HTTP API from the web console. To do that, login to the web console, click on the top right user menu and click the Create API Token menu item. Enter the user's password, give an appropriate token name to help you identify it later, and click Create button. The API token will be generated and displayed immediately.

    Note! The API token is displayed only once and not available later. Thus, copy the token string somewhere temporarily to use in the later steps.

    Warning! It is recommended that you do not generate the API token for the DNS web console's "admin" user. Instead, create a separate user and edit the zone's permissions to give View and Modify access to it. This is necessary since in an event of the API token getting compromised the impact will be limited.
  2. Login using SSH on your web server (for which you wish to setup certbot) as the root user or use sudo su to get root user access before proceeding.
  3. Create a /root/certbot-auth.sh bash script using your favourite text editor with contents as shown below:
    #!/bin/bash
    
    # Generate API token from DNS web console
    API_TOKEN="your-api-token"
    
    # Create challenge TXT record
    curl "http://dns-server:5380/api/zones/records/add?token=$API_TOKEN&domain=_acme-challenge.$CERTBOT_DOMAIN&type=TXT&ttl=60&text=$CERTBOT_VALIDATION"
    
    # Sleep to make sure the change has time to propagate from primary to secondary name servers
    sleep 25
  4. Similarly create a /root/certbot-cleanup.sh bash script using your favourite text editor with contents as shown below:
    #!/bin/bash
    
    # Generate API token from DNS web console
    API_TOKEN="your-api-token"
    
    # Delete challenge TXT record
    curl "http://dns-server:5380/api/zones/records/delete?token=$API_TOKEN&domain=_acme-challenge.$CERTBOT_DOMAIN&type=TXT&text=$CERTBOT_VALIDATION"
  5. Make both the bash scripts executable and only accessible to the root user as shown below:
    chmod 700 /root/certbot-auth.sh
    chmod 700 /root/certbot-cleanup.sh
  6. Install certbot if you do not have it already installed.
    apt update
    apt install certbot
  7. Run the certbot command as shown below to start the initial certificate request for your domain name.
    certbot certonly \
        --manual \
        --preferred-challenges=dns \
        --manual-auth-hook /root/certbot-auth.sh \
        --manual-cleanup-hook /root/certbot-cleanup.sh \
        -d example.com \
        -d *.example.com
  8. Once the certbot command completes successfully, you will find the generated certificates in /etc/letsencrypt/live/example.com folder.

The HTTP API provides powerful ways to automate tasks as shown above but it has some issues that needs to be considered:

  • Using the HTTP API requires that the DNS server's web console needs to be accessible so that it can be used with automation scripts which may not be desirable in all scenarios.
  • The sample certbot auth/cleanup bash scripts that use the HTTP API uses HTTP links which are not secure. Thus you need to setup SSL certificate for the DNS server's web console before deploying certbot as described above in production.
  • If the API token saved in the bash scripts is compromised, an attacker will be able to perform all HTTP API calls with the priviledge of the API token's user. Thus its needed to be made sure that the user has limited access to only the zones that are required.
  • The API token allows modifying all the records in the zone since there is no granular permission available for each record. So, any compromise of the token can cause issues for the entire zone.

Using Dynamic Updates (RFC 2136)

Technitium DNS Server supports Dynamic Updates (RFC 2136) for primary zones which can be used with Certbot's certbot-dns-rfc2136 plugin.

Dymamic Updates is a DNS standard which means it uses the DNS protocol itself to work and thus there is no requirement for configuring SSL certificate for your DNS server's web console like in the case of HTTP API.

Also a special thing about Dynamic Updates is that it allows authentication using TSIG standard and allows specifying a Security Policy wherein you can explicitly configure which record and record type can be created/modified/deleted using the TSIG key.

This means that Dynamic Updates option is much better compared to the HTTP API option from security perspective. Thus its recommended to be used in production over the HTTP API option described earlier.

Follow the steps below to setup certbot to use certbot-dns-rfc2136 plugin to handle DNS challenge:

  1. Login using SSH on your web server (for which you wish to setup certbot) as the root user or use sudo su to get root user access before proceeding.
  2. Install certbot and python3-pip if you do not have it already installed.
    apt update
    apt install certbot python3-pip -y
  3. Install the certbot-dns-rfc2136 plugin as shown below.
    python3 -m pip install certbot-dns-rfc2136
  4. Login to the DNS server's web console and navigate to Settings > TSIG section. Click on the Add button on the top right side to add a new entry. Enter an appropriate TSIG key name to help you identify it later, keep the Shared Secret empty, select the algorithm as HMAC-SHA256, and click Save Settings button at the bottom left corner. This will generate a TSIG key with a randomly generated Shared Secret which will be shown after you have saved the settings. Use the TSIG key name and the generated Shared Secret in the next step.
  5. Create a /root/certbot-rfc2136.ini text file using your favourite text editor with contents as shown below:
    # Target DNS server (IPv4 or IPv6 address, not a hostname)
    dns_rfc2136_server = 192.168.10.2
    # Target DNS port
    dns_rfc2136_port = 53
    # TSIG key name
    dns_rfc2136_name = YOUR-TSIG-KEY-NAME.
    # TSIG key secret
    dns_rfc2136_secret = YOUR-TSIG-SHARED-SECRET
    # TSIG key algorithm
    dns_rfc2136_algorithm = HMAC-SHA256
  6. Set file permissions so that the ini file is only accessible to the root user.
    chmod 600 /root/certbot-rfc2136.ini
  7. Configure the zone, for which you wish to generate the SSL certificate, to enable Dynamic Updates. To do so, switch to the DNS server's web console and navigate to the Zones section. Find your zone and click on it to open the zone edit view. Find the Options button at the top and click it to open the Zone Options dialog. Navigate to the Dynamic Updates (RFC 2136) tab to configure it. In there select the "Allow" option to allow Dynamic Updates then scroll down to create a Security Policy. Click on the Add button to add a new security policy entry. Here select the TSIG key name that you had created earlier, enter "_acme-challenge.example.com" as the domain name, enter "TXT" as the Allowed Record Types, and click Save button to complete the config.
  8. Run the certbot command as shown below to start the initial certificate request for your domain name.
    certbot certonly \
      --dns-rfc2136 \
      --dns-rfc2136-credentials /root/certbot-rfc2136.ini \
      --dns-rfc2136-propagation-seconds 25 \
      -d example.com \
      -d *.example.com
  9. Once the certbot command completes successfully, you will find the generated certificates in /etc/letsencrypt/live/example.com folder.

Testing Auto Renewal

Once you have the SSL certificate generated with certbot, it will be automatically renewed using the same config that you used to request the initial certificate. To test the certbot renewal process, you can try the dry run command shown below. If there are no errors reported during the dry run then it means the renewal mechanism is working as expected.

sudo certbot renew --dry-run

Configuring Your Web Server

You can now configure your web server or any other server that requires the SSL certificate to complete the setup. When the SSL certificate is renewed, your web server may need to be reloaded to allow it to load the new certificate. To do that you can create a /etc/letsencrypt/renewal-hooks/post/reload.sh bash script using your favourite text editor with the sample script below that will reload nginx web server in this example.

#!/bin/sh

systemctl reload nginx
echo "nginx reloaded!"

Make the reload script executable as shown below.

chmod +x /etc/letsencrypt/renewal-hooks/post/reload.sh

The above script will be automatically executed by certbot when the SSL certificate is renewed.

If you have any queries do let me know in the comments section below or send an email to support@technitium.com.

7 comments:

  1. Hello Technitium team,
    thanks for the functionality. The instructions do not have enough recommendations for setting up a domain. There should probably be a CNAME record "_acme-challenge" in example.com? Perhaps somehow differently? If we follow the “Using Dynamic Updates (RFC 2136)” instructions directly, then when generating certificates we may see the following error:

    Certbot failed to authenticate some domains (authenticator: dns-rfc2136). The Certificate Authority reported these problems:
    Domain: your_domain
    Type: dns
    Detail: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.your_domain - check that a DNS record exists for this domain

    ReplyDelete
    Replies
    1. Thanks for asking. The "_acme-challenge." TXT record is added automatically by the certbot rfc2136 plugin. The plugin uses the Dynamic Updates RFC 2136 protocol to add that record in your zone. You do not need to add any CNAME or other type of record manually.

      If its still not working then just check if the zone has the TXT record populated from the DNS admin panel when the cert renewal begins. If you do not see TXT record then it could be that the certbot plugin is failing to add the TXT record for some reason.

      Check the DNS server logs from the admin panel and see if there are any errors related to dynamic updates. The error message will explain what went wrong.

      If you see any error related to TSIG then verify that you have correctly configured the certbot plugin by checking if the TSIG key name, algorithm and secret match exactly as you have configured in the DNS server.

      Delete
    2. Hello Shreyas Zare!

      Tell me, besides the zone settings in technitiumdns itself, what settings should I make at the level of the DNS server that manages the domain (NS record) so that certbot understands that I am the owner of the domain?

      Perhaps I'm missing some stupid point, so I apologize for a possibly obvious question.

      thank you in advance!

      Delete
    3. There is nothing to configure apart from what the blog post already describes. The "dns_rfc2136_server" option in the certbot plugin config tells the IP address of your DNS server which the certbot plugin will use to send the dynamic update request for adding the TXT record.

      You need to check the DNS logs to understand if the certbot plugin is succeeding in its task to add the TXT record using dynamic updates.

      Delete
    4. Please do not hesitate to ask questions. If you need more help with the setup then you can email to support@technitium.com.

      From the error messages you posted, it looks like both the domain names in command in the the error message do not match since one is ".io" and other ".ru". Not sure why this should happen.

      Since you see the TXT records in your zone, the certbot plugin is working as expected. So no issues with your certbot config at least.

      Have you setup your DNS server and zone correctly? I mean, if you query from the Internet (not from LAN), does your DNS server answer them? Test this using the dnsclient.net website where you can enter your DNS server's public IP address and query for your zone and see if it responds as expected.

      Delete
    5. Thank you)

      In both cases, ".io" is installed, not ".ru". The output from the console was sent with an error, everything is fine here)

      I am making a request from the local network via nat. The domain “myhomedns.io” itself does not belong to me. I configured the zone according to the recommendations. I could hardly make a mistake here. I don't understand a little about setting up a DNS server? I have another domain A whose record I can manage and more. If RFC 2136 requires a real domain and DNS settings, then apparently I missed this point...

      Delete
    6. For the DNS Challenge to work, the zone you have must be publicly accessible. The TXT record verification is done by Let's Encrypt servers (not local certbot) to verify ownership of the domain name by testing if you have access to the domain to add those TXT records. So yes, you need a real domain and run your own DNS server on public IP address to host it for this blog post to work.

      Delete