Replacing Unbound and Cloudflare with DNS RPZ_

Back in 2018 I deployed a two-tier DNS server architecture on my network to provide filtering to recursive clients. I wrote about it in more detail in a blog post but the short summary is that I was using BIND which conditionally forwarded to Unbound which forwarded to Cloudflare's anycast DNS resolvers. Over the years I grew unhappy with the complexity, performance and reliability of this setup and decided to take a step back and re-evaluate the solution. I evaluated my goals to start with and determined that I need to provide the following DNS services.

1. Authoritative DNS to the Internet for my public zones.
2. Authoritative DNS to my private network for both my public and private zones.
3. Caching DNS resolution for clients and servers with filtering.
4. Caching DNS resolution for clients without filtering.

Clearly there is a lot of overlap in these services and in the name of simplicity it is desirable to maintain a single pool of systems with a common configuration to perform these functions. The previous architecture grew out of a hack I had in place using Unbound to proxy external lookups over DNS-over-TLS (DoT) since that was not suppored in BIND. The PiHole project provided block lists for Unbound so it was simple to just bolt that on and call it a day. If I want to simplify things I need to figure out a way to replicate that functionality with BIND and I needed to make a decision on DoT.

Why Give Up Encryption?

The most obvious question at this point is why am I thinking of giving up encrypted DNS? In the end the tradeoff must be weighed in each case.
Encrypting DNS adds quite a lot of setup time to the connection, potentially adding a huge amount of latency to all sorts of applications for difficult to diagnose reasons. It's true that DNS traffic can be used to profile, track, and even fingerprint users and encryption can provide privacy if it is your ISP snooping on your traffic. In the case of my previous setup though I was simply moving the point of snooping to Cloudflare and lets face it, they're no more trustworthy than my ISP. By running a caching, recursive resolver the actual DNS traffic that leaves my network is going to traverse the normal resolution process, eg: . -> com -> going-flying and then be cached. In the end, the more clients that I have using my resolvers the better, as the ability to distinguish my traffic from other clients is diminished. I have a /23 across 4 physical locations using 2 separate resolvers so any given request could be on behalf of an automated process on a device like a Roku or another person here, or my web browser. That request could leave through any number of IP or IPv6 addresses across two or three different ISPs, further confounding would-be-snoopers. Thinking through all of that, transport encryption seems like a very small gain for the increase in latency. And to be clear I was seeing un-cached request times in the 600 - 800 millisecond range via DoT versus 40 - 100 milliseconds for the same requests over unencrypted DNS. It's worth noting that I've supported DNSSEC (both signing my zones and validating in my resolvers) for years now so the actual authenticity of DNS replies is already cryptographically secured.

Do I Have To Though?

Well that depends. I run Debian Linux's stable distribution wherever I can and the currently shipping version of BIND is in the 9.16 branch, while BIND didn't gain DoT support until 9.17.7. Certainly you can use the upgraded forwarders syntax to use DoT for transport security if your Linux distribution ships a newer BIND version (or if you compile it yourself) but as you can probably tell from the previous section I didn't feel it was worth chasing.

DNS RPZ

Ok, so how do I do filtering in BIND? DNS RPZ or DNS Response Policy Zones were introduced in BIND 9.8 and provide a native method to construct a DNS firewall using zones. These are then usable by views so I can apply separate policies (or sets of policies) for trusted and untrusted clients on different parts of my network. Perfect! It also turned out that this is much less memory intensive than the Unbound method, which created stub zones for each block list entry causing Unbound to consume several GB of memory on my DNS servers and take several seconds to start or stop. With DNS RPZ BIND consumes a svelte 512MB of memory and doesn't take a noticably different amount of time to load or reload than it did without a 216,000+ line zone loaded.

To move from Unbound to DNS RPZ I went back to the source data, which turned out to be several hosts.txt style files containing lists of domain names to block. I rewrote the script that I had written to fetch the PiHole files to instead converting these source lists into a zone file suitable for BIND.

#!/bin/sh
# generate-bind-blocklist
# (c) 2018-2022 Matthew J Ernisse <matt@going-flying.com>
# All Rights Reserved.
#
# Create a bind9 style zonefile of domains to block using RPZ.

set -e

OUTPUT="/var/cache/bind/adblock"

cleanup()
{
    if [ -d "$TMPDIR" ]; then
        rm -r -- "$TMPDIR"
    fi
}

combine_lists()
{
    local outfile="$TMPDIR/$(basename $OUTPUT)"
    local zonefile="$TMPDIR/$(basename $OUTPUT).zonefile"

    for fn in $@; do
        cat "$TMPDIR/$fn" >> "${outfile}"
    done

    sort -u "${outfile}" > "${outfile}.sorted"
    cat - > "$zonefile" << EOF
; This file wass generated by generate-bind-blocklist at
; $(date)
\$TTL 1209600
@           IN SOA localhost. hostmaster.localhost. (
                1
                1209600
                72000
                8640000
                1209600
)
            IN NS localhost.

EOF
    for zone in $(cat "${outfile}.sorted"); do
        echo "${zone}           IN CNAME ." >> "$zonefile"
    done

    mv -- "$zonefile" "$outfile"
}

fetch_list()
{
    if [ -z "$1" ] || [ -z "$2" ]; then
        echo "usage fetch_list url file"
        exit 1
    fi

    local url="$1"
    local file="$2"
    wget -qt 3 -O "$TMPDIR/$file" "$url"
}

flatten_hosts()
{
    if [ -z "$1" ]; then
        echo "usage: flatten_hosts file"
        return 1
    fi

    local src="$TMPDIR/$1"
    awk '/^(0\.0\.0\.0)/ {
        if ($2 == "localhost") {
            next
        }

        if ($2 == "0.0.0.0") {
            next
        }

        gsub("[\r\n]", "", $2)
        print $2
    }' "$src" > "${src}.new"
    mv -- "${src}.new" "${src}"
}

process_excludes()
{
    local workfile="$TMPDIR/$(basename $OUTPUT)"
    for zone in $(cat /usr/local/share/adblock/local_exceptions); do
        sed "/^${zone}/d" "$workfile" > "${workfile}.new"
        mv -- "${workfile}.new" "${workfile}"
    done
}

trap cleanup EXIT
TMPDIR="$(mktemp -qdt generate-bind-blocklist.XXXXXX)"
if [ ! -d "$TMPDIR" ]; then
    echo "Failed to create temp dir"
    exit 1
fi

# looks like hosts-file.net got bought by malwarebytes and is now a dumpster
# fire.  Yaaay.

# potentially a replacement for hosts-file.net
fetch_list "https://v.firebog.net/hosts/AdguardDNS.txt" \
    "ats.txt"
fetch_list "https://v.firebog.net/hosts/JoeyLane.txt" \
    "joeylane.txt"
fetch_list "https://v.firebog.net/hosts/Prigent-Malware.txt" \
    "prigent-malware.txt"
fetch_list "https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts" \
    "hosts"

cp /usr/local/share/adblock/local_ad_hosts "$TMPDIR/local_ad_hosts"

# We just want a list of names, we don't need the hosts format.
flatten_hosts "hosts"

combine_lists "ats.txt" \
    "joeylane.txt" \
    "prigent-malware.txt" \
    "hosts" \
    "local_ad_hosts"

process_excludes
mv -- "$TMPDIR/$(basename $OUTPUT)" "$OUTPUT"

I also created a separate zone file to replaces the Unbound configuration I used to block the Firefox DNS-over-HTTPS canary domain as I described in a separate blog post. While I was at it I also blocked the iCloud Private Relay canaries. That zone file is included below, a more complete example is also provided in the BIND documentation.

$TTL 1209600
@               IN      SOA     localhost.      hostmaster.localhost. (
                                    1
                        1209600
                                    72000
                                    8640000
                                    1209600
)
                        IN  NS      localhost.

; Disable iCloud Private Relay.
; https://developer.apple.com/support/prepare-your-network-for-icloud-private-relay/
mask.icloud.com             CNAME .
mask-h2.icloud.com          CNAME .

; Disable Mozilla DoH forwarder TYVM.
; https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https
; https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet
use-application-dns.net         CNAME .

This lets me choose on a per-view basis what, if anything, to block. You simply load the zones in your view (or in the root level of the config if you aren't using views), then refer to that view or view in a response-policy stanza. For example I have a guests view that gets the the DoH and iCloud canary block and an internal view that gets everything. As you might expect the public views get no blocks at all. A chunk of my named.conf looks like this.

view "guests" {
        allow-query { guests; };
        allow-recursion { guests; };
        match-clients { guests; };
        response-policy { zone "blocklist"; };

        zone "blocklist" {
                type master;
                file "blocklist";
        };

        include "/etc/bind/named.conf.default-zones";
        include "/etc/bind/named.conf.public-zones";
};

view "internal" {
        allow-query { vpn; };
        allow-recursion { vpn; };
        match-clients { vpn; };
        response-policy {
                zone "adblock";
                zone "blocklist";
        };

        zone "adblock" {
                type master;
                file "adblock";
        };

        zone "blocklist" {
                type master;
                file "blocklist";
        };

        include "/etc/bind/named.conf.default-zones";
        include "/etc/bind/named.conf.internal-zones";

};

Conclusion

Unfortunatly, and in no small part thanks to the length of my ~/TODO I have been running this for several years now. In fact I have likely been runnig this for longer than I ran the previous two-tier setup described in the original article. 😊 The results are all that I could have asked for, though. This way is more performant and more reliable than the previous system. I used to regularly fiddle with the DNS server assignments given out by my DHCP server to placate various guests and family members who would have difficult to diagnose issues, bypassing the protection and sending them to an off-net service, but no more. It allows me to have a single DNS server configuration that works for all of my scenarios (I previously had separate public DNS server configurations that didn't include the Unbound layer), serving my private network and clients on the Internet which allowed me to reduce my cloud server spend. As a privacy bonus, it keeps untrustworthy Internet mega-providers like Cloudflare or Google from being a choke point for all of my DNS resolution traffic. Since my DNS needs are way too complex to run consumer-grade appliances, this is refreshingly simple solution that has been serving me well for several years now.