DNS over HTTPS on Ubiquity USG

If you wish to run your DNS queries on  over HTTPS to Cloudflare’s using Ubiquity USG,  then read on.

The first step is to compile and install cloudflared. I used a mac and and docker to compile the binary from github.

docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp -e GOOS=linux -e GOARCH=mips golang bash -c "go get -v github.com/cloudflare/cloudflared/cmd/cloudflared; GOOS=linux GOARCH=mips go build -v -x github.com/cloudflare/cloudflared/cmd/cloudflared"

After compiling – you’ll end up with a file called cloudflared on your local machine. Copy this over to the USG in the /confi/scripts/ folder. You could use SCP, or a GUI based client for Mac like Transmit.

Login to the USG via shell, then change sudo to su . Change permission to make the file executable.

sudo su
chmod +x /config/scripts/cloudflared

Create an auto startup script and place in config/scripts/post-config.d

vi /config/scripts/post-config.d/cloudflare-dns.sh



# start DNS proxy to Cloud Flare

/usr/bin/pkill cloudflared

nohup /config/scripts/cloudflared --no-autoupdate --proxy-dns --proxy-dns-port 5053 &>/var/log/cloudflared.log &

Change permission to make the file executable.

chmod +x /config/scripts/post-config.d/cloudflare-dns.sh

Create a new file using a text editor such as TextEdit or Atom on your mac and create the following json file. The structure of a json file is just as important as the words themselves. Incorrect placement of brackets, indentations, line breaks or any other structural element will make the json file invalid. Save it by naming it config.gateway.json


        "service": {
                "dns": {
                        "forwarding": {
                                "options": [

Now copy this file to your Controller under the following folder (You could use SCP, or a GUI based client for Mac like Transmit. )  – On Cloud Key install the path for the .json file is: /srv/unifi/data/sites/[site name/default]/

Read here for more information on the config.gateway.json file -https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-USG-Advanced-Configuration

Once done, head over to your controller dashboard , select devices, select USG, select config, select manage device, select force provision.

Once the provisioning completes, validate that the DNS on your machine is pointing to the USG IP. Test your DNS by visiting the following IP on your mac.



Screenshot 2018-10-22 at 9.11.24 PM

If doesn’t work delete the 2 lines in red in config.gateway.json and force reprovision. 


So apparently, after a reboot of the USG, I started seeing errors in the logs stating that a secure connection to cannot be made. I first thought the ISP is bloking the connection but later realized that my WAN IP perisited after a reboot.  I tried mutiple combinations, but the following worked

<Kill the process>
ps -ef| grep cloudflared
kill -9 <Process ID of cloudflared from command above>

<remove startup script>
rm -rf /config/scripts/post-config.d/cloudflare-dns.sh

<reboot USG>
<start cloudflared>
nohup /config/scripts/cloudflared --no-autoupdate --proxy-dns --proxy-dns-port 5053 &>/var/log/cloudflared.log &
<validate no errors in /var/log/cloudflared>

<Reset contents of config.gateway.json to the one above - which means if you deleted the text in RED put it back>
<Force provision USG again>
<Test - https://www.cloudflare.com/ssl/encrypted-sni/>
<If it still doenst work, delete 2 RED lines in config.gateway.json>






DIY Home Firewall – Part 1 Sophos XG

I have been a  ASUS fan for a long time. Had been running AC-87U for a while at my house. I had the Merlin firmware on it with a Openvpn client to my VPN provider. Served my needs for a few years before I realized that  I wasn’t  able to run a secure VPN server on it/ behind it.  I used a Synology NAS behind the router to host a VPN server and connect to it when I was outside the network. And in no time the NAS was hacked 🙂 – how did I know ? Well the light’s wont stop blinking even when I’d shut down all packages. I Ran a packet capture  and sure enough it was running a crypto mining malware. Anyways more on that later.

I looked around and a few options presented themselves for a home firewall

  1. Sophos XG UTM – An Enterprise grade true UTM with features like IPS, AV, Firewall, Anti-Spam etc. A very nice GUI and a forever Free Home License. (https://demo.sophos.com/webconsole/webpages/login.jsp – user:demo pass:demo)
  2. Pfsense – An open source firewall with ‘packages’ that you can configure to get near to a UTM performance. Widely supported and a range of tutorials available on the web.
  3. Opnsense – A fork from Pfsense  with a slicker GUI and some improvements/enhancements from Pfsense
  4. Untangle – Marketed towards SME’s with extensive reporting features, a home license is 50$ a year. You could also get a community version without a charge. (http://demo.untangle.com/admin/index.do)
  5. Clear OS – While not exactly a firewall, ClearOS is an operating system for servers, gateways and firewalls. It delivers applications through a marketplace to enable SMB’s to consume services without the need for multiple hardware. They offer an online demo here – https://www.clearos.com/clearos-live-demo
  6. Ubiquiti USG – An appliance based firewall, delivers excellent value for money and can be controlled via an intutive portal. Offers fancy dashboard for reporting based on Deep Packet Inspection (DPI)

My internet connection at home is a 1G Fiber broadband. The fiber terminates on an ISP provided ONT box, which acts as a layer 2 device. The ONT gives a RJ-45 handoff.

So with my needs clear, 1G throughput and a couple of RJ45 ports (one for LAN and one for WAN), SSLVPN server – i started searching.

I eliminated the Ubiquiti USG because it doesn’t offer an inbuilt SSL VPN server and the people on the forums reported throughput issues on the smaller USG-3P device. The 4P device looked too big for my house and so i skipped it.

Untangle was easy to eliminate as I wasn’t too fond of the GUI (personal choice) . ClearOS was also eliminated as it looked too heavy for my requirements. So choice was down to Sophos XG, Pfsense and Opnsense.

I ended up selecting Sophos XG as the UI looked polished and components looked easy to setup (this would change later, as I moved to Opnsense ultimately). Part of it was also because I wanted somethign that would just work, without having to spend a great deal of time on it (trying to be lazy).

For the hardware, i zeroed it down to a Qotom 335G4 box from China. It has an i5 processor with 4 cores , upto 8 GB RAM, support for AES-NI and most importantly all Intel NIC’s (apparently all platforms XG, pfsense and opnsense reported issues with RealTek NIC’s). I ordered it with 8GB RAM and a 60GB SSD (came installed as a msata drive) . It also has a SATA connecter – so you can hook up a SATA SSD to the box later.

Ordered the box from Alibaba and it was with me in a couple of days.

Q300G4 Ports 800

The install was a breeze.I got the ISO from the Sophos XG site after registering. I used Etcher to burn the ISO to a USB drive. Connected the box to power and an HDMI monitor. To boot from the USB, hit DEL during startup (when you see the boot logo). Continue with the onscreen instructions, till you device reboots from the inbuilt storage.

The only confusing part is that XG doest not recognise the ports in order (same for OPNsense while PfSense recognised them in order)

For XG (default config) - The LAN Port is the one on the extreme left and WAN is the 3rd port from left.
For OPNsense - The LAN Port is the one one extreme left and WAN is the one next to it (or second from the left).
For pfswense - The first port from left is WAN and the one next to it is LAN (or the second from left)

So you would want to connect your PC to the extreme left port and wait for the DHCP to assign you an IP. Once connected the default IP for the web admin is You can change this after you run the initial setup. The initial setup is nothing too complicated and will get you up and running in about 5 minutes. The initial config also has a default rule allowing all outbound LAN communication and blocking all inbound.

I reconfigured the ASUS router under AP mode and in no time, I had a functional UTM on. I suppose the entire thing took about 20 minutes and I was back on the internet – sweet !.

Once you are on and connected, here are few things you can do to complete your setup

  1. Define DHCP pool and create static MAC to DHCP bindings
  2. Create Users or clienteles users
  3. Enable user authentication ( I do this only for the guest network). All other devices have a static MAC to IP binding.
  4. Enable SSL VPN
  5. Enable IPS policies and AV
  6. Create custom firewall rules
  7. Create schedules for internet access
  8. Enable content filtering
  9. Create a VLAN for separating IOT traffic
  10. Add more firewall policies for IOT to WAN communication; LAN to IOT communication
  11. Enable multicast broadcast across VLAN’s for DLNA access from one VLAN to the other.

Sophos has some pretty good documentation on this and the forums are very helpful ( the guys are awesome !)

With IPS and and content filtering on – I get about 300Mbps on the wireless network (more than enough for my needs). Sophos uses snort which is single threaded which explains that it doesn’t go beyond 300Mbps on a single connection. If you have a few machines running in parallel or if you iperf3 to a public perf server with multi-thread you can go upto 700Mbps in my case. Under normal loads with about 20 devices the CPU stays under 5%.

To conclude , if you are looking at a home firewall solution to replace your router and you want things to just work (limited customization) SophosXG offers a very compelling UTM in a box solution – worth a try !

End – for now

P:S – I’ll probably do a Sophos vs Pfsense vs Opnsesne post later.