Full-Network Adblocking with Dnsmasq

linux ubuntu raspberrypi ansible ipv6

Every few months, I contemplate setting up a pi-hole to save bandwidth and make life easier for any devices at home that can't run ublock. Pi-hole is a great project that helps a lot of people, but every time I go to install it, I see their curl-piped installer and cringe.

Nowadays, they have alternate install instructions that are a little bit safer, so I downloaded it and started going through the 2000+ line shell script. About halfway through, I realized that most of what it's doing, I don't need!

The main things that pi-hole provides:

blocklisting of domain names
nice web UI for allowing specific domains
automatic download of blocklists
stats and graphs
automatic setup of the various network bits to make it work

The only one of these that I actually need is the first one! For the rest, I have the skills and infrastructure already to make it happen with only a little bit of glue.

Quick Background

I'm already using ansible to install and update my raspberry pi as a NAS. Since that's all working already, adding an extra role for serving DNS is pretty easy!

The Plan

Install Dnsmasq
Set up a blocklist from a public source
Set up DHCP to point devices at the new DNS server

It turns out number 3 was harder than I expected, but I'll get to that later!

Dnsmasq server

Installing Dnsmasq is basically trivial. And there's only a teensy bit of config needed to get it up and running.

One useful thing that raspbian has set up by default is loading files from /etc/dnsmasq.d/ automatically. I'll be using that to load in the blocklist.

Fetching a blocklist

I'm using the unified blocklist from StevenBlack. It's fairly comprehensive, and shouldn't need much tweaking as time goes on.

The most obvious way would be to use wget or curl to pull down the file and parse it. But ansible has a built-in way to fetch a URL and load it into a variable - I'll use that and save it into a dnsmasq config file.

Wait, what about /etc/hosts?

I actually tried this at first, but I feel like it's too likely to cause dpkg upgrade conflicts and block something. It's also likely that dnsmasq is going to be more performant when loading a file under its control.

Reformatting the list for dnsmasq

Formatting the data for dnsmasq is a little bit tricky though.

The raw list is in hostfile format:

0.0.0.0 doubleclick.net  # and there's often comments

But it needs to be reformatted to be address lines:

address=/doubleclick.net/0.0.0.0

Plus, it's a 10,000 line file!

The steps I'll put it through:

fetch the URL as a list of lines
filter out localhost lines and blank lines
get rid of comments at the end of lines (they make reformatting hard)
rearrange it and save the whole thing where ansible can use it

Here's what that ends up looking like:

- block_list_lines: "{ { lookup('url', 'https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts', wantlist=True)
     | select('match', '^0.0.0.0.*')
     | map('regex_replace', '\\s*#.*$', '')
     | map('regex_replace', '^0.0.0.0 (?P<host>.*)$', 'address=/\\g<host>/#')
     | join('\n') } }"

One awesome feature that dnsmasq supports in this format is mapping the address to #. Using this is equivalent to having mappings to BOTH 0.0.0.0 and the ipv6 equivalent ::, so we're saving 10,000 lines of config this way. :D

roles/dns-adblock/templates/blocklist_hosts.conf

Armed with a super-long variable representing our blocklist, it's getting templated into an extremely simple blocklist template file.

# blocklist file - adblocking and malware blocking
# this file is managed by ansible.

# Add domains which you want to force to an IP address here.
# using an upstream list modified for dnsmasq - should look like this:
# address=/doubleclick.net/0.0.0.0
##############################################

{ { block_list_lines } }

# manually add in any extra domains to block here
address=/bsimb.cn/#

roles/dns-adblock/templates/dnsmasq.conf

I have this set up as a template in my ansible config. No actual templating needed though, it's used verbatim.

# Never forward plain names (without a dot or domain part)
domain-needed
# Never forward addresses in the non-routed address spaces.
bogus-priv

# where to send upstream requests. using cloudflare.
# doing this to avoid pi sending requests to itself
server=1.1.1.1
server=1.0.0.1

# log queries for debugging. goes to /var/log/syslog by default.
log-queries

That's it!

roles/dns-adblock/tasks/main.yml

Putting it all together - this is the entirety of the ansible setup!

---
# PACKAGES
- name: Install packages and dependencies for DNS adblocking
  become: yes
  apt:
    state: present
    pkg:
      - dnsmasq

# CONFIG
- name: Update Dnsmasq main config
  become: yes
  template: src=dnsmasq.conf dest=/etc/dnsmasq.conf owner=root group=root

# load in StevenBlack's unified blocklist (adware + malware)
# alternate blocklist source location: http://sbc.io/hosts/hosts
- name: Update DNS blocklist
  become: yes
  template: src=blocklist_hosts.conf dest=/etc/dnsmasq.d/blocklist_hosts.conf owner=root group=root
  vars:
    # pull down the file and reformat it for dnsmasq in one step
    - block_list_lines: "{ { lookup('url', 'https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts', wantlist=True) | select('match', '^0.0.0.0.*') | map('regex_replace', '\\s*#.*$', '') | map('regex_replace', '^0.0.0.0 (?P<host>.*)$', 'address=/\\g<host>/#') | join('\n') } }"

- name: Restart dnsmasq service
  become: yes
  service: name=dnsmasq state=restarted

Connecting to the new DNS Server

While this sounds like the easiest part of the whole process, it ended up being a bit of a journey.

On my router, I went into the DHCP settings and set the raspberry pi's IP address as the primary DNS server. I reconnected my android phone to the wifi and confirmed that it got the new DNS settings, off to a great start!

But when I went to load an ad-filled news site on the ipad, prepared for victory… there were still ads. Noo! The same thing happened on the OSX laptop. Okay, maybe it's a mac thing? But no, it's also still happening on my PopOS linux laptop. They're all picking up the IP address of the Pi fron DHCP, so that's working. When looking closer, they're all picking up 2 extra DNS servers with ipv6 addresses. Time to figure out where those are coming from!

After more investigation than it should have taken, and a fair bit of a refresher on how ipv6 works, I found that they belonged to my ISP, and were being provided automatically by my DSL modem/gateway. I had to go into the IPv6 WAN settings and paste in the Pi's IPv6 address.

Getting the Pi's ipv6 address is quite easy:

ip -6 addr

Updating this in the router config, and reconnecting devices to wifi showed the ipv6 config coming through - so far so good.

One Last thing to get it working on linux

PopOS and it's parent distro Ubuntu both run a local DNS resolver - systemd-resolved. It clearly does some amount of caching, and to get it to pick up the ipv6 config I had to restart it.

sudo service systemd-resolved restart

Conclusions

At the end of the day, for the cost of really not that much setup code, I have a network-wide DNS blocklist, which I can control and redeploy with ansible any time. To get the blocklist updated, I just have to re-run the ansible tasks. I avoided all the risk of having the pihole installer stomp on something else I'd configured, and I learned some things about ipv6 and modern linux networking along the way. Well-worth it!