Tag Archives: USG

OPENVPN site to site vpn between USG and openwrt

A new firewall means a new site to site VPN configuration. My current iteration of this is a USG Pro 4 serving as an OpenVPN server and a Netgear Nighthawk R8000 serving as a VPN client joining their two networks together.

First, I had to wrap my head around some concepts. To set this up you need three sets of certificates and a DH file:

  • CA: To generate and validate certificates
  • Server: To encrypt/decrypt traffic for the Server
  • Client: To encrypt/decrypt traffic from the Client
  • DH: Not a certificate but still needed by the server for encryption

The server and client will also need openvpn configurations containing matching encryption/hashing methods, CA public key, and protocol/port settings.

Generate certificates

If you already have PKI infrastructure in place you simply need to generate two sets of keys and a DH file for the server/client to use. If you don’t, the easy-rsa project comes to the rescue. This tutorial uses easy-rsa version 3.

I didn’t want to generate the certificates on my firewall so I picked a Debian system to do the certificate generation. First, install easy-rsa:

sudo apt install easy-rsa

In Debian easy-rsa is installed to /usr/share/easy-rsa/

Optional: Set desired variables by moving /usr/share/easy-rsa/vars.example to /usr/share/easy-rsa/vars and un-commenting / editing to suit your needs (in my case I like to extend the life of my certificates beyond two years.)

Next, create your PKI and generate CA certificates:

/usr/share/easy-rsa/easyrsa init-pki
/usr/share/easy-rsa/easyrsa build-ca

Now create your DH file. Grab a cup of coffee for this one, it can take up to ten minutes to complete:

/usr/share/easy-rsa/easyrsa gen-dh

Then create your server & client certificates. For this guide we are calling the server ovpn-server and the client ovpn-client

#For the server
/usr/share/easy-rsa/easyrsa gen-req ovpn-server nopass 
/usr/share/easy-rsa/easyrsa sign-req server ovpn-server

#For the client
/usr/share/easy-rsa/easyrsa gen-req ovpn-client nopass
/usr/share/easy-rsa/easyrsa sign-req client ovpn-client

You will be asked for a common name. Remember what you put here, you will need it later. If you just hit enter and accept the default the common name will match what was passed in the above commands (ovpn-server for the server certificate and ovpn-client for the client certificate.)

Lastly, copy these files to their respective hosts:

USG Server: CA, Server key & cert, and DH file. (substitute with IP of your device)

scp pki/dh.pem pki/ca.crt pki/private/ovpn-server.key  pki/issued/ovpn-server.crt admin@IP_OF_YOUR_USG:/config/auth/

OpenWRT Client: Client key & cert, and CA cert:

scp pki/private/ovpn-client.key pki/issued/ovpn-client.crt pki/ca.crt root@IP_OF_YOUR_OPENWRT:/etc/config/

USG: VPN Server

Documentation for the EdgeRouter is much easier to find than for the USG. Since they use the same operating system I based this off of this guide from Logan Marchione for the EdgeRouter. SSH into your USG and issue the following, substituting the $variables with the values you desire for your network.

Explanation of variables:

VPN_SUBNET: Used for VPN communication. Must be different from both server and client subnets.
SERVER_SUBNET: Subnet on server side you wish to pass to client network
VPN_PORT: Change this to desired listening port for the OpenVPN server
REMOTE_SUBNET: Subnet on client side you wish to pass to server network
REMOTE_NETMASK: Netmask of client subnet
REMOTE_VPN_IP: Static IP you wish to give the client on the VPN subnet.
REMOTE_CERT_NAME: Common name given to client certificate generated previously.

Replace $variables below before pasting into USG terminal:

configure
#OpenVPN config
set interfaces openvpn vtun0
set interfaces openvpn vtun0 description "OpenVPN Site to Site"
set interfaces openvpn vtun0 mode server
set interfaces openvpn vtun0 encryption aes256
set interfaces openvpn vtun0 hash sha256
set interfaces openvpn vtun0 server subnet $VPN_SUBNET
set interfaces openvpn vtun0 server push-route $SERVER_SUBNET
set interfaces openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
set interfaces openvpn vtun0 tls cert-file /config/auth/ovpn-client.crt
set interfaces openvpn vtun0 tls key-file /config/auth/ovpn-client.key
set interfaces openvpn vtun0 tls dh-file /config/auth/dh.pem
set interfaces openvpn vtun0 openvpn-option "--port $VPN_PORT"
set interfaces openvpn vtun0 openvpn-option --tls-server
set interfaces openvpn vtun0 openvpn-option "--comp-lzo yes"
set interfaces openvpn vtun0 openvpn-option --persist-key
set interfaces openvpn vtun0 openvpn-option --persist-tun
set interfaces openvpn vtun0 openvpn-option "--keepalive 10 120"
set interfaces openvpn vtun0 openvpn-option "--user nobody"
set interfaces openvpn vtun0 openvpn-option "--group nogroup"
set interfaces openvpn vtun0 openvpn-option "--route $REMOTE_SUBNET $REMOTE_NETMASK $REMOTE_VPN_IP"
set interfaces openvpn vtun0 server client $REMOTE_CERT_NAME ip $REMOTE_VPN_IP
set interfaces openvpn vtun0 server client $REMOTE_CERT_NAME subnet $REMOTE_SUBNET $REMOTE_NETMASK

#Firewall config
set firewall name WAN_LOCAL rule 50 action accept
set firewall name WAN_LOCAL rule 50 description "OpenVPN Site to Site"
set firewall name WAN_LOCAL rule 50 destination port $VPN_PORT
set firewall name WAN_LOCAL rule 50 log enable
set firewall name WAN_LOCAL rule 50 protocol udp
commit

If the code above commits successfully, the next step is to add the config to config.gateway.json. The USG’s config is managed by its Unifi controller, so for any of the changes made above to stick we must copy them to /usr/lib/unifi/data/sites/default/config.gateway.json on the controller (create the file if it doesn’t already exist.)

A quick shortcut is to run the mca-ctrl -t dump-cfg command, then parse out the parts you want to go into config.gateway.json as outlined in the UniFi documentation. For the lazy, here is the config.gateway.json generated from the above commands (be sure to modify $variables to suit your needs.)

{
  "firewall": {
    "WAN_LOCAL": {
      "rule": {
        "50": {
          "action": "accept",
          "description": "OpenVPN Site to Site",
          "destination": {
            "port": "$VPN_PORT"
          },
          "log": "enable",
          "protocol": "udp"
        }
      }
    }
  },
  "interfaces": {
    "openvpn": {
      "vtun0": {
        "description": "OpenVPN Site to Site",
        "encryption": "aes256",
        "hash": "sha256",
        "mode": "server",
        "openvpn-option": [
          "--port $VPN_PORT",
          "--tls-server",
          "--comp-lzo yes",
          "--persist-key",
          "--persist-tun",
          "--keepalive 10 120",
          "--user nobody",
          "--group nogroup",
          "--route $REMOTE_SUBNET $REMOTE_NETMASK $REMOTE_VPN_IP"
        ],
        "server": {
          "client": {
            "$REMOTE_CERT_NAME": {
              "ip": "$REMOTE_VPN_IP",
              "subnet": [
                "$REMOTE_SUBNET $REMOTE_NETMASK"
              ]
            }
          },
          "push-route": [
            "$SERVER_SUBNET"
          ],
          "subnet": "$VPN_SUBNET"
        },
        "tls": {
          "ca-cert-file": "/config/auth/ca.crt",
          "cert-file": "/config/auth/ovpn-client.crt",
          "dh-file": "/config/auth/dh.pem",
          "key-file": "/config/auth/ovpn-client.key"
        }
      }
    }
  }
}

OpenWRT: VPN client

Configuration is doable from the GUI but I found much easier with the command line. I got a lot of the configuration from this gist from braian87b

Install openvpn and the luci-app-openvpn packages:

opkg update
opkg install openvpn luci-app-openvpn

OpenVPN config files are located in /etc/config. In addition to the certificates we copied there earlier, we will also want to copy the openvpn client configuration to that directory.

Here is the config file matching the configuration generated above. Again, remember to replace $variables with your config matching what was generated above. Save it to /etc/config/site2site.conf

#/etc/config/site2site.conf
client
dev tun
proto udp
remote $DNS_OR_IP_OF_USG_OPENVPN_SERVER $VPN_PORT
cipher AES-256-CBC
auth SHA256
resolv-retry infinite
nobind
comp-lzo yes
persist-key
persist-tun
verb 3
ca /etc/config/ca.crt
cert /etc/config/ovpn-client.crt
key /etc/config/ovpn-client.key
remote-cert-tls server

With the openvpn config file, client certificate & key, and CA certificate we are ready to configure firewall rules and instruct the router to initiate the VPN connection.

# a new OpenVPN instance:
uci set openvpn.site2site=openvpn
uci set openvpn.site2site.enabled='1'
uci set openvpn.site2site.config='/etc/config/site2site.conf'

# a new network interface for tun:
uci set network.site2sitevpn=interface
uci set network.site2sitevpn.proto='none' #dhcp #none
uci set network.site2sitevpn.ifname='tun0'

# a new firewall zone (for VPN):
uci add firewall zone
uci set firewall.@zone[-1].name='vpn'
uci set firewall.@zone[-1].input='ACCEPT'
uci set firewall.@zone[-1].output='ACCEPT'
uci set firewall.@zone[-1].forward='ACCEPT'
uci set firewall.@zone[-1].masq='1'
uci set firewall.@zone[-1].mtu_fix='1'
uci add_list firewall.@zone[-1].network='site2sitevpn'

# enable forwarding from LAN to VPN:
uci add firewall forwarding
uci set firewall.@forwarding[-1].src='lan'
uci set firewall.@forwarding[-1].dest='vpn'

# Finally, you should commit UCI changes:
uci commit

Monitor VPN connection progress by using logread. If all goes well you will see the successful connection established message. If not, you’ll be able to get an idea of what’s wrong.

logread -f

If all goes well you’ll now have a bidirectional VPN between your two sites; however, traffic from the server’s subnet going directly to the client router itself (the OpenWRT device’s IP) will be considered as coming from the WAN interface and will be blocked. If you need to access the OpenWRT device directly from the USG’s subnet, you’ll need to add a firewall rule allowing it to do so:

uci add firewall rule
uci set firewall.@rule[-1]=rule
uci set firewall.@rule[-1].enabled='1'
uci set firewall.@rule[-1].target='ACCEPT'
uci set firewall.@rule[-1].src='wan'
uci set firewall.@rule[-1].name='Allow VPN to access router'
uci set firewall.@rule[-1].src_ip='$SERVER_SUBNET'
uci set firewall.@rule[-1].dest_ip='$INTERNALL_IP_OF_OPENWRT_ROUTER'
uci commit

Troubleshooting

One-sided VPN

I fought for some time with the fact that the VPN was established, but only traffic going from the Client network to the Server network would work. Traffic from the OpenVPN server subnet to the OpenVPN client subnet would simply hang and not work.

I finally found on the ubiquiti forums that this is due to default OpenVPN behavior of restricting traffic from the server subnet to the client subnet (see the OpenVPN how-to for more information.) The solution is to add lines in the server config informing it of the client network and to allow traffic to it. Below is an example USG config allowing informing it of remote subnet 192.168.230/24 and assigning the Client an IP of 10.0.76.253:

set interfaces openvpn vtun5 server client client1 ip 10.0.76.253
set interfaces openvpn vtun5 server client client1 subnet 192.168.230.0/24

VPN status stays “stopped” in OpenWRT

The best way to troubleshoot is to look at the logs in realtime. SSH to the OpenWRT router and run the command “logread -f” then try to initiate the connection again. The errors there will point you to the problem.

Automate USG config deploy with Ubiquiti API in Bash

I have a new Ubiquiti Unifi Security Gateway Pro 4 which is pretty neat; however, the Unifi web interface is pretty limited. Most advanced firewall functions must be configured outside of the GUI. One must create a .json file with the configuration they need, copy that file to the Unifi controller, and then force a provision of the gateway to get it to pick up the new config.

I wanted a way to automate this process but very frustratingly Ubiquiti hasn’t documented their Unifi Controller API. I had to resort to reverse engineering their API by using my browser’s developer console to figure out which API calls were needed to do what I wanted. I then took the API functions from https://dl.ui.com/unifi/5.10.25/unifi_sh_api (the current unifi controller software download link which has unifi_sh_api) and embedded them into a bash script. Thanks to this forum post for the information on how to do this.

This bash script copies the specified config file to the Unifi controller via SCP, then uses curl to issue the API call to tell the controller to force a provision to the device having the supplied mac address.

#!/bin/bash
# Written by Nick Jeppson 08/01/2019
# Inspired by posts made from ubiquiti forums: https://community.ui.com/questions/API/82a3a9c7-60da-4ec2-a4d1-cac68e86b53c
# API interface functions taken from unifi_sh_api shipped with controller version 5.10.25, https://dl.ui.com/unifi/5.10.25/unifi_sh_api
#
# This bash script copies the specified config file to the Unifi controller via SCP
# It then uses curl to issue an API call to tell the controller to force a provision to the device with the supplied mac address. 

#### BEGIN VARIABLES ####
#Fill out to match your environment

gateway_mac="12:34:56:78:90:ab" #MAC address of the gateway you wish to manage
config_file="your_config_file.json"   #Path to config file
unifi_server="unifi_server_name"         #Name/IP of unifi controller server
unifi_gateway_path="/usr/lib/unifi/data/sites/default/config.gateway.json"    #Path to config.gateway.json on the controller
ssh_user="root"                 #User to SSH to controller as
username="unifi_admin_username"             #Unifi username
password="unifi_admin_password" #Unifi password
baseurl="https://unifi_server_name:8443" #Unifi URL
site="default"                  #Unifi site the gateway resides in

#### END VARIABLES ####

#Copy updated config to controller
scp $config_file $ssh_user@$unifi_server:$unifi_gateway_path

#API interface functions
cookie=$(mktemp)
curl_cmd="curl --tlsv1 --silent --cookie ${cookie} --cookie-jar ${cookie} --insecure "
unifi_login() {
    # authenticate against unifi controller
    ${curl_cmd} --data "{\"username\":\"$username\", \"password\":\"$password\"}" $baseurl/api/login
}

unifi_logout() {
    # logout
    ${curl_cmd} $baseurl/logout
}

unifi_api() {
    if [ $# -lt 1 ] ; then
        echo "Usage: $0 <uri> [json]"
        echo "    uri example /stat/sta "
        return
    fi
    uri=$1
    shift
    [ "${uri:0:1}" != "/" ] && uri="/$uri"
    json="$@"
    [ "$json" = "" ] && json="{}"
    ${curl_cmd} --data "$json" $baseurl/api/s/$site$uri
}

#Trigger a provision
unifi_login 
unifi_api /cmd/devmgr {\"mac\": \"$gateway_mac\", \"cmd\": \"force-provision\"}
unifi_logout

No more manually clicking provision after manually editing the config file on the controller!