Wireguard on a USG Pro 4

I ran into some issues when trying to configure an OpenVPN tunnel between my Ubiquiti USG Pro 4 and a Debian VPS. I was very disappointed to discover that the version of OpenVPN on the USG only supports TLS 1.0! My issue was the Debian side rejecting that as insecure.

Thankfully, it was fairly painless to get Wireguard configured on the USG Pro 4. I was hesitant to do so at first because I knew every time my USG was updated I would lose the wireguard pacakge. Fortunately that can be resolved by configuring a post-install script. Thanks to ilar.in and calypte.cc and this github gist for the steps on how to do so.

curl -O https://raw.githubusercontent.com/britannic/install-edgeos-packages/master/install-pkgs
sudo install -o root -g root -m 0755 install-pkgs /config/scripts/post-config.d/install-pkgs
  • Add wireguard DEB package to persistent storage that the script will look for:
sudo mkdir -p /config/data/install-packages
cd /config/data/install-packages
curl -fLSs https://github.com/WireGuard/wireguard-vyatta-ubnt/releases/download/1.0.20200729-1/ugw4-v1-v1.0.20200729-v1.0.20200513.deb
  • Generate a public/private keypair for USG use
cd /config/auth
umask 077
wg genkey > wg_private.key
wg pubkey < wg_private.key > wg_public.key
  • Generate config.gateway.json config to use wireguard
    "interfaces": {
 ...
        "wireguard": {
            "wg0": {
                "address": "<IP_OF_USG_ON_WG_CLIENT_SUBNET>",
                "listen-port": "<WG_LISTEN_PORT>",
                "peer": {
                    "<ENDPOINT_CLIENT_PUBLIC_KEY>": {
                        "allowed-ips": "0.0.0.0/0",
                        "endpoint":  "<ENDPOINT_CLIENT_ADDRESS>:<ENDPOINT_CLIENT_PORT>
"
                    }
                },
                "private-key": "/config/auth/priv.key",
                "route-allowed-ips": false
            },
            "wg1": {
                "address": "<IP_OF_USG_ON_WG_SERVER_SUBNET",
                "firewall": {
                    "in": {
                        "name": "LAN_IN"
                    },
                    "local": {
                        "name": "LAN_LOCAL"
                    },
                    "out": {
                        "name": "LAN_OUT"
                    }
                },
                "listen-port": "<USG_WG_SERVER_LISTEN_PORT>",
                "mtu": "1352",
                "peer": {
                    "<PUBLIC_KEY_OF_WG_CONNECTING_CLIENT": {
                        "allowed-ips": "<SUBNETS_ON_REMOTE_HOST>"
                    }
                },
                "private-key": "/config/auth/wg-server.priv",
                "route-allowed-ips": true
            }
        }
    },

I have two different wireguard interfaces configured – wg0 to be a client to another server, and wg1 to be a server accepting other clients (site-to-site VPN.)

If you want to have multiple peers defined on a single wireguard interface, encapsulate the peers with brackets like so:

"peer": [{
						"--pubkey--": {
							"allowed-ips": [
								"172.255.252.2/32"
							],
							"persistent-keepalive": 60
						}
					},
					{
						"--pubkey--": {
							"allowed-ips": [
								"172.255.252.3/32"
							],
							"persistent-keepalive": 60
						}
					}
				],

Test configuration first

Before committing your config.gateway.json code, test it line by line by SSHing into the USG-Pro 4 and entering config mode. Then type out your JSON lines one at a time, with each key being a new argument separated by a space. The first section above would look like this:

configure
edit interfaces
set wireguard wg0 address WIREGUARD_ADDRESS
set wireguard wg0 listen-port WG_LISTEN_PORT
set wireguard wg0 peer ENDPOINT_CLIENT_PUBLIC_KEY allowed-ips 0.0.0.0/0
set wireguard wg0 peer ENDPOINT_CLIENT_PUBLIC_KEY endpoint ENDPOINT_ADDRESS:ENDPOINT_PORT
set wireguard wg0 private-key /config/auth/priv.key
set wireguard wg0 route-allowed-ips false
commit

If the commit works without error, you can then drop out of the configure section and look at your wireguard config:

exit
sudo wg show

If all looks well, then copy your config.gateway.json to your controller and trigger a reprovision.

Verify after provisioning: 

sudo netstat -npl | grep <WIREGUARD_PORT> | grep udp

Troubleshooting

USG not connecting to changed endpoint address

If you change the address of the wireguard endpoint, USG pro will not connect to that new address. You have to delete and re-create the interface (thanks to https://github.com/Lochnair/vyatta-wireguard/issues/72#issuecomment-423840448 for the information)

Fix this by deleting the wireguard interface

admin@Firewall:~$ configure
[edit]
admin@Firewall# delete interfaces wireguard
[edit]
admin@Firewall# commit


Then reprovision by making a small change, force provision, then change back, and force another provision (annoying) or alternatively reboot firewall.

Wireguard shows established but ping doesn’t work

Example error:

From 10.99.13.1 icmp_seq=5 Destination Host Unreachable
ping: sendmsg: Required key not available

To figure out what’s going on, enable logging to kernel buffer (dmesg) Thanks to procustodibus.com for the info.

echo module wireguard +p > /sys/kernel/debug/dynamic_debug/control

With debug on, ping again and check kernel messages (dmesg)

[Tue Dec 21 22:16:12 2021] wireguard: wg0: No peer has allowed IPs matching 10.99.13.2

This showed I didn’t have my access control properly configured. Modify /etc/wireguard/wg0.conf on your client config and make sure your AllowedIPs are properly letting traffic through.

AllowedIPs = 10.99.13.0/24

USG not allowing connections

Clients unable to connect to USG despite having a good config. Double check your firewall rules. I had neglected to create a WAN LOCAL rule allowing UDP packets on my wireguard port. Once that was configured, handshakes completed successfully.

Firewalls can’t ping each other

I had an issue where the firewalls would pass traffic through, but they couldn’t ping each other. The solution was to add the VPN subnet you created to allowed-ips on both sides of the connection.

Site to site Wireguard VPN between OPNSense & Debian Linux server

I have a Debian linode box acting as a wireguard server. I wanted to join my opnsense firewall to it to allow devices behind it to access the box through the wireguard tunnel. It was not as straightforward as I had hoped, but thankfully I got it all working.

OPNSense side

Documentation link

Install wireguard via GUI

Install the os-wireguard package. Manually drop to the CLI and install the wireguard package as well:
sudo pkg install wireguard

Configure Local instance

  • Name and listen port can be random. Tunnel address is the subnet you wish to expose to the other end (the subnet you wish to have access to the tunnel.)
  • Leave everything else blank and hit save
  • Edit your new connection and copy the Public key, this will need to be sent to the Debian server

Configure Endpoint

  • Name: hostname of Debian server
  • Public Key: Public key of server (can be obtained by running wg show on the server)
  • Shared Secret: blank (unless you’ve configured it on the server)
  • Allowed IPs: IPs or subnets on the Debian server you wish to expose to the client side (the OPNSense box)
  • Endpoint address: DNS name of Debian server
  • Endpoint port: Port Debian wireguard instance is listening on

Enable the VPN

General tab / Enable wireguard checkbox and hit apply.

Debian side

Take down the tunnel

sudo wg-quick down wg0

Edit wireguard config to add peer

sudo vim /etc/wireguard/wg0.conf

[Peer]
PublicKey = <PUBLIC_KEY_YOU_COPIED_IN_LOCAL_INSTANCE_STEP>
AllowedIPs = <IPs or Subnets behind the OPNSense side you wish to be exposed to the Debian side> 

Restart wireguard

sudo wg-quick up wg0

Check connections

Example wg show output below with dummy IPs:

sudo wg show
interface: wg0
  public key: f+/J4JO0aL6kwOaudAvZVa1H2mDzR8Nh3Vfeqq+anF8=
  private key: (hidden)
  listening port: 12345

peer: TuUW7diXcWlaV97z3cQ1/92Zal2Pm9Qz/W2OMN+v20g=
  endpoint: 10.0.0.1:54137
  allowed ips: 10.0.0.2/32
  latest handshake: 17 seconds ago
  transfer: 5.14 KiB received, 3.81 KiB sent

peer: CZuC/+wxvzj9+TiGeyZtcT/lMGZnXsfSs/h5Jtw2VSE=
  endpoint: 8.8.8.8:12345
  allowed ips: 192.168.1.1/32
  latest handshake: 7 minutes, 8 seconds ago
  transfer: 5.89 MiB received, 952.20 MiB sent

The endpoint: line gets populated when a successful VPN connection is made. If it’s missing, the tunnel was not established.

Troubleshooting

OPNSense box

Nothing happens after saving information and enabling tunnel

Make sure latest wireguard package is installed

sudo pkg install wireguard

Get more log output by opening a shell on your OPNSense box and running

sudo /usr/local/etc/rc.d/wireguard start

In my case I was getting this interesting message

[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg0
┌──────────────────────────────────────────────────────┐
│                                                                                                                                                   │
│   Running wireguard-go is not required because this                                                     │
│   kernel has first class support for WireGuard. For                                                          │
│   information on installing the kernel module,                                                                 │
│   please visit                                                                                                                             │
│         https://www.wireguard.com/install/                                                                           │
└──────────────────────────────────────────────────────┘

I fixed this problem by manually installing wireguard with the pkg install command above.

Debian box

Wireguard config not saving

make sure to stop the tunnel first, otherwise your changes get overwritten by the daemon.

sudo wg-quick down wg0
<make changes>
sudo wg-quick up wg0

Self host postfix SMTP relay for Zimbra Mail Server

My notes for spinning up a small Debian linode server to act as an SMTP relay for my home network (note you will have to engage with linode support to enable mail ports for new accounts.)

Relay server configuration

Install postfix

sudo apt install postfix

Modify main.cf

sudo vim /etc/postfix/main.cf

Under TLS parameters, add TLS security to enable secure transfer of mail

smtp_tls_security_level = may
I decided not to open up postfix to the internet but instead my relay has a wireguard tunnel and postfix is allowed to relay only from that VPN subnet.

Add your subnets and relay restrictions further down:

mynetworks = 127.0.0.0/8 <YOUR_SERVER_SUBNET>
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated check_relay_domains
relay_domains = <MY_DOMAIN_NAME>
myhostname = <RELAYSERVER_HOSTNAME>
inet_interfaces = 127.0.0.1, <IP_OF_WIREGUARD_VPN_INTERFACE>

Zimbra configuration

In Zimmbra admin panel, edit your mail server

Configure / Servers / your_mail_server

MTA section

Add the DNS name and port of the relay system next to “Relay MTA for external deliverability”

If it won’t let you save, saying ::1 is required, you can add ::1 to MTA Trusted networks, however, on my Zimbra server this broke postfix. The symptoms were e-mails hanging and not sending. To fix, log into the Zimbra mail server and run as the zimbra user:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks ‘127.0.0.1 192.168.0.0/16’ (list of networks you had before but excluding ::1)

Then, issue postfix reload

That was it. A simple postfix SMTP relay which only accepts mail from my internal VPN (it doesn’t listen on the external interface at all.)

Troubleshooting

Realyed mail shows red unlock icon in Gmail (mail getting sent unencrypted)

Per postfix documentation I needed to enable secure transfer of mail by adding

smtp_tls_security_level = may

to main.cf

Mail does not send after adding ::1 to MTA Trusted Networks

Remove it via the CLI and reload postfix:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks '127.0.0.1 192.168.0.0/16' (list of networks you had before but excluding ::1)
postfix reload