The Linux Page

Setting up BIND to get the letsencrypt wildcards to work on your system using RFC 2136

Which destination has to be used?

Wildcard Certificate with letsencrypt

I have my own DNS, so I need to set it up myself to get letsencrypt to work as expected and generate a wildcard certificate for my websites.

They decided to test the DNS because that way they know you are in control of the domain and its sub-domains (only the owner  of a domain name would be able to allow such a test to work.) When creating a certificate for just one website they can ask you to place a file there, which is very easy, but for an entire domain, that wouldn't be quite enough, especially since some business endeavors actually use such a technique and either offer for free or sell sub-domains of a highly priced domain name.

About the Nameserver (BIND9)

So, I had to setup my DNS to let letencrypts tweak a parameter to test that I do indeed own that domain name. I use BIND9 as the name server. I looked around and found a very good post about how to do it the hard way... or maybe it was how to do it when the letsencrypt wildcard feature came out. The fact is that it's actually very simple if you know what is required.

With BIND9 you define two types of files:

1) Configuration files, at least one of which has a list of zones

2) Zone files, one zone file per domain is required for this one

The BIND9 zone file format follows semantics that are very close to the technical aspect of the nameserver protocol.

The configuration files are easier. Somewhat based on a C-like structure definition.

What needs to be updated

In order to get a letsencrypt certificate, there are only two things you need to update in your configuration files. First the zone definition is given and update-policy entry as follow:

zone "restarchitect.com" IN {
  type "master";
  file "/var/lib/bind/restarchitect.com.zone";
  allow-transfer { trusted-servers; };
  check-names warn;
  update-policy {
    grant letsencrypt_wildcard. name _acme-challenge.restarchitect.com. txt;
  };
};

In that definition, I also added the check-names option because letsencrypt decided to use a name with an underscore (see below for more info.)

The rest is pretty much standard, but here I will define each of the instructions I use in my example here:

  • type "master";

The type defines whether this domain name is being served by the master DNS or a slave. In my example here, this zone is from the master DNS.

  • file "/var/lib/bind/restarchitect.com.zone";

The path and filename to the actual zone file. I have a sample below where you can see how it needs to be setup. Not much going on though. However, the important part is that this file can't reside under /etc.

IMPORTANT INFORMATION: The named service checks where the zone file is found. If it is defined under /etc/... then it does not get updated and that means updates to a zone are accepted but lost right away (I'm not too sure why it is accepted if it can't be saved, though.)

This is why I use a path under /var/lib/... where files are expected to be writable. Under Ubuntu, you can set the ownership to bind.

chown -R bind:bind /var/lib/bind

Although from my own experience, it works with files owned by root. But if you have a problem, your setup may be such that the named service may be running in a jail and not have write access to those files. You may also have issues with the writing of files when SE Linux is running.

You may have copies of your zones under /etc/bind/... if you only backup /etc and not /var/lib/.... That way you'll have the original available.

  • allow-transfer { trusted-servers; };

I use this line to authorize full access to my secondary DNS systems. Especially, that gives my other DNS to retrieve the complete list of known sub-domain names for each domain I have.

This works because I have static IP addresses and the addresses of my other computers are specified in a named.conf option file:

options {
  ...
  allow-transfer { 10.0.0.1; 10.0.0.2; ... };
  ...
};

This example says that computers with IP address 10.0.0.1 and 10.0.0.2 gave permissions to do fast transfers from the DNS. (see about the axfr option below)

  • check-names warn;

The check-names option is required in case the name letencrypt adds _acme-challenge to your list of known sub-domains. The underscore character is not liked by BIND9. This is because it is not part of the domain name specification. It is not allowed at all. By default BIND will generate an error and log it and skip over that entry entirely (i.e. it will not serve that zone at all, albeit all the other zones will work just fine.)

You can also set this parameter to ignore. In that case, no warning is emitted in your logs.

Here is the error you get ("bad owner name") when a name uses characters that are not supposed to be used in a domain name:

09-Feb-2019 03:02:31.988 general: error:
           /var/lib/bind/restarchitect.com.zone:31:
          _acme-challenge.restarchitect.com:
          bad owner name (check-names)

The check-names option is currently the only way to fix this problem (i.e. you can't use an escape for that one specific letter.)

  • update-policy { ... }

This line grants one specific permission to anyone with the HMAC secret key.

grant letsencrypt_wildcard.
      name _acme-challenge.restarchitect.com.
      txt;

Here is the line dissected:

1. The policy is a "grant"

This means we are going to allow a certain command to modify our DNS. The granularity is small enough that we can offer such with enough assurance that it won't allow hackers to completely destroy (take over) your DNS.

2. The name of the key: "letencrypt_wildcard."

This is a reference to your encryption key. It is used to communicate between the client and server in such a way that it proves that the client knows us (the client has to have a copy of the key to be able to communicate with us.)

3. The name "_acme-challenge.restarchitect.com"

The sub-domain name which the client can temper with. This means only that one name is going to be updated by clients. All the other names can't be changed at all.

4. The type "txt"

Further, we specify the exact type of information can be modified. By using the "txt" type, letsencrypt limits permissions to the name which does not allow them to transformed that name with a valid IP address.

In other words, there is pretty much no hijacking possible with such a grant.

Create and Setup an HMAC Key

Now you want to define the HMAC key to be used with the bind setup.

letsencrypt expects such a key when you run their certbot command.

To generate the key, run the following commands:

$ sudo su -
(enter password when prompted)
# cd /etc/bind
# mkdir letsencrypt_keys
# chmod 700 letsencrypt_keys
# cd letsencrypt_keys
# dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST letsencrypt_wildcard.
# sudo chmod 600 *

This is a relatively secure way of creating the key.

The output of the dnssec-keygen command are two files:

Kddns_update.+165+35602.key
Kddns_update.+165+35602.private

We are interested by the .private file. That file includes the key encoded using base64.

Private-key-format: v1.3
Algorithm: 165 (HMAC_SHA512)
Key: <here is a bunch of letters/digits/etc.>
Bits: AAA=
Created: 20190209013350
Publish: 20190209013350
Activate: 20190209013350

I show in blue what we are interested by. Copy that value from this file and paste it in a .conf file in BIND that you will name something like letsencrypt_wildcard_key.conf.

key "letsencrypt_wildcard" {
    algorithm hmac-sha512;
    secret "<here is a bunch of letters/digits/etc.>";
};

Now you need to include this key in your named.conf so it can be accessed by the zone declaration we mentioned earlier (and it has to appear before that zone declaration.) You can also copy/paste the whole key in your options, however, the key file should be in a safe folder as shown above (i.e. owned by root with read/write permissions only given to root. If you have SE Linux, you may instead need to use bind:bind for the ownership.)

IMPORTANT NOTE:

Here the key name is just letters and an underscore. No period (.) at the end of the name. Notice that in the grant definition above we have a period at the end.

To include the file, edit one of your configuration files and add an include command:

include "/etc/bind/letencrypt_keys/letencrypt_wildcard_key.conf";

Restart BIND9

To make sure that your settings are still valid, run a check:

named-checkconf /etc/named.conf

When no errors are discovered, the command returns as is (no output.) When errors are displayed, you may want to also look at the logs for additional hints about the potential problems.

Now is time for you to restart your BIND service. This is done with:

systemctl restart bind9

If you just added a new domain name, you can test the validity with dig:

dig @ns1.m2osw.com restarchitect.com

With the @... I specify the domain name server (DNS) to query. This way I directly query the source. The second parameter to dig is the name of the domain I'm testing.

Testing The Grant

To make sure that the grant we just defined in BIND9 works, we can use the nsupdate tool. This tool gives you a way to send commands to your DNS.

WARNING: to test properly, you want to test from a computer which is not the one with the DNS nor a secondary server (i.e. do not use a computer which IP address appears in the list of allow-transfer { ... } computers. Testing with one of these computers is likely to not check the grant itself.

The following are the instructions I suggest you use to test the grant:

$ nsupdate -k /path/to/letsencrypt_wildcard_key.conf -v
> server restarchitect.com
> debug yes
> zone _acme-challenge.restarchitect.com
> update add _acme-challenge.restarchitect.com. 86400 TXT "test"
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; ZONE SECTION:
;restarchitect.com.        IN    SOA

;; UPDATE SECTION:
_acme-challenge.restarchitect.com. 86400 IN TXT    "test"

> send
Sending update to 10.0.0.1#53
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:  60546
;; flags:; ZONE: 1, PREREQ: 0, UPDATE: 1, ADDITIONAL: 1
;; ZONE SECTION:
;restarchitect.com.        IN    SOA

;; UPDATE SECTION:
_acme-challenge.restarchitect.com. 86400 IN TXT    "test"

;; TSIG PSEUDOSECTION:
letsencrypt_wildcard.    0    ANY    TSIG    hmac-sha512.
                   1549778797 300 64 <key> 60546 NOERROR 0


Reply from update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:  60546
;; flags: qr; ZONE: 1, PREREQ: 0, UPDATE: 0, ADDITIONAL: 1
;; ZONE SECTION:
;restarchitect.com.        IN    SOA

;; TSIG PSEUDOSECTION:
letsencrypt_wildcard.    0    ANY    TSIG    hmac-sha512.
                    1549778797 300 64 <key> 60546 NOERROR 0

> quit

First, notice that I use the key defined in the configuration file I created above for BIND9. I changed the path because I'm on a different computer which is not a nameserver (otherwise the test may succeed even though the grant would not allow the user to otherwise change the zone settings.)

The update add ... command is the one defining what will be updated on the zone (which was specified on the line before.) Here we set a TXT field to the value "test".

The show command is not necessary. It gives you a way to see whether you type things right (if you can understand the output...)

The send command is what sends the UDP request to your server. The result shows "NOERROR" when it worked. It will say "FAILED" otherwise. For example, if you use the wrong key, it will fail. But assuming that you get everything right on your client computer, the command will fail only if the DNS does not authorize you to make the update. [NOTE: I broke up a couple of lines in two to make it easier to read on the website.]

Once the nsupdate says it worked (i.e. it returned a NOERROR message,) you can test with the dig command to see that it indeed was regitered:

$ dig @ns1.m2osw.com restarchitect.com axfr

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.m2osw.com restarchitect.com axfr
; (1 server found)
;; global options: +cmd
restarchitect.com.    86400    IN    SOA    ns1.m2osw.com. hostmaster.m2osw.com. 1309082308 10800 180 1209600 300
restarchitect.com.    86400    IN    NS    ns1.m2osw.com.
restarchitect.com.    86400    IN    NS    ns2.m2osw.com.
restarchitect.com.    86400    IN    A    10.0.0.1
_acme-challenge.restarchitect.com. 86400 IN TXT    "test"
w.restarchitect.com.    86400    IN    A    10.0.0.1
ww.restarchitect.com.    86400    IN    A    10.0.0.1
www.restarchitect.com.    86400    IN    A    10.0.0.1
wwww.restarchitect.com.    86400    IN    A    10.0.0.1
restarchitect.com.    86400    IN    SOA    ns1.m2osw.com. hostmaster.m2osw.com. 1309082308 10800 180 1209600 300
;; Query time: 20 msec
;; SERVER: 10.0.0.1#53(10.0.0.1)
;; WHEN: Sat Feb 09 22:11:27 PST 2019
;; XFR size: 11 records (messages 1, bytes 313)

Note that the axfr requests a full transfer of the zone. For unknown clients, this feature is turned off by default. For this command to work you actually want to run it on one of your DNS servers (primary or secondary.)

As we can see in the output (highlighted in blue) the TXT field was added as expected. If not added, you're on your own buddy! Maybe re-read my document here and see whether something could have gone wrong. The logs often tell you why BIND9 refused the update transaction.

Once this works, the certbot will also be able to make such changes and that's sufficient for your system to receive a wildcard type of certificate.

Passing the Key to certbot

In order for certbot to access your DNS, it needs to have access to your key. The key will be transmitted to the letsencrypt servers which in turn verify that you own your domain before issue the certificate.

Create a file, for example, /etc/bind/letsencrypt_keys/certbot.ini and copy the following, fixing a few parameters as required by your system:

# Target DNS server
dns_rfc2136_server = 10.0.0.1
# Target DNS port
dns_rfc2136_port = 53
# TSIG key name
dns_rfc2136_name = letencrypt_wildcard.
# TSIG key secret
dns_rfc2136_secret = <here is a bunch of letters/digits/etc.>
# TSIG key algorithm
dns_rfc2136_algorithm = HMAC-SHA512

The dns_rfc2136_server parameter is the public IP address of your DNS server. (The example shows a private IP.)

The dns_rfc2136_name parameter defines the name of the key. I used "letsencrypt_wildcard" in my prior examples, this is that name.

The dns_rfc1236_secret parameter is the private key. The same we put in the letsencrypt_wildcard_key.conf file.

Generating the Wildcard Certificate

Now we are ready to generate a wildcard certificate with certbot:

sudo certbot certonly \
  --dns-rfc2136 \
  --dns-rfc2136-credentials /etc/bind/letsencrypt_keys/certbot.ini \
  -d '*.restarchitect.com'
  -d restarchitect.com

As we can see, the command references the certbot.ini file we just created.

The --dns-rfc2136 command line option tells certbot how to handle the domain name verification: directly with your DNS information as defined in the certbot.ini file.

The -d option specifies the name of the domain for which you want a certificate. Notice that to get a wildcard certificate, you want to use an asterisk. That way all the sub-domains created for restarchitect.com (in my example) will all benefit from the same certificate. However, you also want the name by itself because the wildcard only doesn't match the name without a subdomain.

If you want to protect multiple domain names with the same certificate, you can do so using additional -d command line options. For example, in my case I had the .com and .org so I could write:

... -d '*.restarchitect.com' -d restarchitect.com
    -d '*.restarchitect.org' -d restarchitect.org

There is a limit to the number of -d options you can use. Last time I checked it was 100, which I think is plenty, especially if you use a wildcard certificate.

Note: If you forget to include a certain domain, you can later add it using the --expand option. The one strange thing about the --expand is that you must keep a complete list of all the -d options that you first included and then add more of them. So for example, if you first had the .com and later you want to add the .org, you would still include the .com domains in the list of -d options.

Zone File

Here is a simple example of a minimal zone file.

$ORIGIN .
restarchitect.com       IN SOA  ns1.m2osw.com. hostmaster.m2osw.com. (
                                1309082307 ; serial
                                10800      ; refresh (3 hours)
                                180        ; retry (3 minutes)
                                1209600    ; expire (2 weeks)
                                300        ; minimum (5 minutes)
                                )
                        NS      ns1.m2osw.com.
                        NS      ns2.m2osw.com.
                        A       138.197.205.139
$ORIGIN restarchitect.com.
$TTL 86400      ; 1 day
w                       A       138.197.205.139
ww                      A       138.197.205.139
www                     A       138.197.205.139
wwww                    A       138.197.205.139

The file defines the 5 sub-domains I allow for pretty much all of my websites:

1. (nothing) — the A entry without a name

2. w, ww, wwww — for people who can't type

3. www — for the usual World Wide Web sub-domain (although many of my websites use case (1) and the "www" redirects there.)

You could add more entries such as a mail address or specialized sub-domains such as api for a REST access point.

This is all you need to have in your zone to make the letsencrypt wildcard system work as expected.

If the server decides it can't write to the file, you get (on one line, broken up to fit on this website):

09-Feb-2019 02:05:47.766 general: error:
     /etc/bind/zones/restarchitect.com.zone.jnl:
     create: permission denied

As we can see, we get a "create: permission denied", and that when the file is owned by BIND and rw in modification set.

Potential Errors

When the DNS challenge fails, you may get the following error:

Unable to determine base domain for _acme-challenge.ordermade.com
   using names: ['_acme-challenge.ordermade.com',
   'ordermade.com', 'com'].

More or less, this means that letsencrypt was not able to guarantee that you were the owner of the domain name (ordermade.com in my example.) It means your bind9 setup is not correct and it did not allow for adding a TXT field to test that you could allow letsencrypt to verify that you own that domain name.

Additional Information

Here are the pages I used to setup BIND9 and get my wildcard certificates:

certbot RFC 2136 documentation

Let's Encrypt Wildcard Certificate (with the acme.sh script)

Check BIND Server Configuration

List all DNS records


Re: Setting up BIND to get the letsencrypt wildcards to work ...

When you say you tried to run

dig @ns1.m2osw.com restarchitect.com axfr

Did you actually try with that command exactly or did you properly replace the domain names to yours?

ns1.m2osw.com is my DNS server URL. You need to use yours there.

restarchitect.com is the domain for which I want to generate the letsencrypt certificate.

Also, just in case, this is required only if you want to create a certificate that works for all your subdomains (such as www.restarchitect.com and api.restarchitect.com and feed.restarchitect.com etc.)

If you only need one domain name, then this DNS work is not required.

Re: Setting up BIND to get the letsencrypt wildcards to work ...

i have setup my dns bind in ubuntu18.04 . I want to generate a ssl certificate for my subdomain xxx1234.com it resolves perfectly by dns server. My dns server is in local environment .I am using IIS web server in windows10 pro. How i can green pad lock https from my local dns. I have tried all this steps but i got stuck at one place where you say a command
dig @ns1.m2osw.com restarchitect.com axfr. It failed to connect my web server. What may be the reason . I will Thank full if some tell me all the process to get all this work for a particular subdomain.

Thank you.

Re: Setting up BIND to get the letsencrypt wildcards to work ...

Oh I'm sure it's possible.

One idea for many servers is to make all folders, especially /etc, read-only. So having the updated DNS files under /var/lib/bind/... is not a bad thing.

If you look at a tool such as tripwire, you'll see that the least changes you have in /etc the less reports of possible hacks you get. Such a tool is important if you run a website which accepts credit card or collect user information (such as an address, phone number, etc.)

But for a website which does not have such constraint, it's certainly nice to have all the files in one place.

Re: Setting up BIND to get the letsencrypt wildcards to work ...

Hi,

In order to use the regular 'etc' directory instead of /var/lib/bind, just chown both the zone file + containing directory to the user running bind instance.

I did it on my FreeBSD / bind914-9.14.6 and it worked well. So I could leave my zone files at their right place instead of /var/lib/bind/ as advised here.

Cheers and thank you for this howto !

Kind regards,
apn.