VyOS Policy Based Routing with OpenVPN

UPDATE:
On newer versions og VyOS, it is not possible to have OpenVPN tunnels without certificate based authentication. This kills the possibility of using PIA without patching the image. I've created a new guide for the 1.4-rolling-202101300218 releade using Mullvad and WireGuard instead.

VyOS Policy Based Routing with WireGuard

Here is a quick guide to setting up policy based routes for VyOS or other Vyattavariants such as EdgeOS. I wrote a similar post on doing this with OpenVPN usingPIA, however this setup is much simpler as we are able to make use of interfaceroutes. Check out the older article in the link below if y…

Hedberg.ioMathias

Sometimes you want some hosts on your network to use a vpn connection. This is where policy based routing comes in to play. Here is how i got it to work on VyOS 1.3 with PIA VPN.

This may work on the LTS releases of VyOS. I however need features only available in the 1.3.x releases, so results may vary. This config was successful on 1.3-rolling-202010260327, but not on the newer 1.3-rolling-202011070217 as this version introduced bugs in OpenVPN.

Goals/functionality

• Have four different simultaneous OpenVPN connections to PIA spread out geographically.
• Use firewall groups to force specific hosts on a specific connection
• Pool the OpenVPN connections together in a load-balanced fashion so that it is possible for specific hosts to use all connections at once.

Setting it up

This configuration guide assumes you have experience in configuration of Vyatta based systems (EdgeOS, VyOS). We make use of openvpn, routing tables, firewall groups and policies to get this to work.

OpenVPN Connection

This is a fairly standard OpenVPN conection for PIA (unofficial). Note that the openvpn option --proto udp4 is required, even with protocol udp.

# show interfaces openvpn
 openvpn vtun101 {
     authentication {
         password xxxxxxx
         username xxxxxxx
     }
     ddescription Norway
     encryption {
         cipher aes128
     }
     firewall {
         in {
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     hash sha1
     mode client
     openvpn-option --fast-io
     openvpn-option --route-nopull
     openvpn-option "--ping-restart 60"
     openvpn-option --persist-tun
     openvpn-option --persist-key
     openvpn-option "--ping 10"
     openvpn-option "--proto udp4"
     persistent-tunnel
     protocol udp
     remote-host no.privacy.network
     remote-port 1198
     tls {
         ca-cert-file /config/auth/ca.rsa.2048.crt
         crl-file /config/auth/crl.rsa.2048.pem
     }
     use-lzo-compression
 }

I repeated this for a total of 4 tunnels (vtun101-104), where every tunnel uses a different PIA country.

Firewall groups

Next we create firewall groups for each vpn tunnel, designated by the country the tunnel terminates to. I chose to use VPN-PIA-XX, where XX is replaced by the two letter representation of the country. This is done for all 4 tunnels. Any hosts added to these groups will use that specific VPN connection.

# set firewall group address-group VPN-PIA-NO

Then we create a firewall group for the load balanced VPN group, for hosts that can use all connections at the same time.

set firewall group address-group VPN-PIA-LB

Routing tables

Next we set up our routing tables for these connections. To to this we need information on the vpn tunnels, so that we can find the gateway address on each tunnel.

$ ip route | grep vtun
10.7.112.0/24 dev vtun101 proto kernel scope link src 10.7.112.x 
10.11.112.0/24 dev vtun102 proto kernel scope link src 10.11.112.x 
10.25.112.0/24 dev vtun103 proto kernel scope link src 10.25.112.x 
10.27.112.0/24 dev vtun104 proto kernel scope link src 10.27.112.x 

We assume that the gateway is located at the first address of the subnet, so in the example of vtun101 the gateway is at 10.7.112.1.

We apply this knowledge to create four routing tables (one for each VPN) 101-104, and also for the load balancing table at table 100.

# show protocols static 
 table 100 {
     route 0.0.0.0/0 {
         next-hop 10.7.112.1 {
             next-hop-interface vtun101
         }
         next-hop 10.11.112.1 {
             next-hop-interface vtun102
         }
         next-hop 10.25.112.1 {
             next-hop-interface vtun103
         }
         next-hop 10.27.112.1 {
             next-hop-interface vtun104
         }
     }
 }
 table 101 {
     route 0.0.0.0/0 {
         next-hop 10.7.112.1 {
             next-hop-interface vtun101
         }
     }
 }
 table 102 {
     route 0.0.0.0/0 {
         next-hop 10.11.112.1 {
             next-hop-interface vtun102
         }
     }
 }
 table 103 {
     route 0.0.0.0/0 {
         next-hop 10.25.112.1 {
             next-hop-interface vtun103
         }
     }
 }
 table 104 {
     route 0.0.0.0/0 {
         next-hop 10.27.112.1 {
             next-hop-interface vtun104
         }
     }
 }

Policy

Create a policy that modifies the routing table used by hosts found in our previously defined firewall groups.

# show policy 
 route VPN-PIA {
     rule 1 {
         description "Local destinations"
         destination {
             group {
                 network-group RFC1918
             }
         }
         set {
             table main
         }
     }
     rule 100 {
         description LoadBalance
         destination {
             address 0.0.0.0/0
         }
         set {
             table 100
         }
         source {
             group {
                 address-group VPN-PIA-LB
             }
         }
     }
     rule 101 {
         description NO
         destination {
             address 0.0.0.0/0
         }
         set {
             table 101
         }
         source {
             group {
                 address-group VPN-PIA-NO
             }
         }
     }
     rule 102 {
         description SE
         destination {
             address 0.0.0.0/0
         }
         set {
             table 102
         }
         source {
             group {
                 address-group VPN-PIA-SE
             }
         }
     }
     rule 103 {
         description DK
         destination {
             address 0.0.0.0/0
         }
         set {
             table 103
         }
         source {
             group {
                 address-group VPN-PIA-DK
             }
         }
     }
     rule 104 {
         description FI
         destination {
             address 0.0.0.0/0
         }
         set {
             table 104
         }
         source {
             group {
                 address-group VPN-PIA-FI
             }
         }
     }
 }

Apply the policy to packets coming in from eth1 (and any other interfaces where your clients may be located)

# set interfaces ethernet eth1 policy route VPN-PIA

Auto update gateway address

Since the PIA gateway addresses may change, i created a script to update the value once a minute if changes are detected. This script is a bit hacky, and I welcome you to send me any changes you made to it.

#!/bin/vbash
source /opt/vyatta/etc/functions/script-template
configure
is_changes=false
echo ""
for table in 101 102 103 104
do
    echo ""
    echo "------------------$table-------------------"
    openvpn_interface="vtun$table"
    openvpn_route=$(ip route | grep $openvpn_interface | awk {'print $1'} | cut -f1 -d"/" | sed 's/\([0-9]*\.[0-9]*\.[0-9]*\.\)[0-9]*/\11/')
    current_route=$(show protocols static table $table route 0.0.0.0/0 next-hop | grep next-hop | head -n 1 | awk {'print $2'} | awk {'print $1'})
    echo current=$current_route openvpn=$openvpn_route
    if [ ! -z "$openvpn_route" ] && [ "$current_route" != "$openvpn_route" ]; then
        echo "Table $table default route mismatch"
        if [ ! -z "$current_route" ]; then
            delete protocols static table $table route 0.0.0.0/0 next-hop $current_route
            delete protocols static table 100 route 0.0.0.0/0 next-hop $current_route
        fi
        set protocols static table $table route 0.0.0.0/0 next-hop $openvpn_route next-hop-interface $openvpn_interface
        set protocols static table 100 route 0.0.0.0/0 next-hop $openvpn_route next-hop-interface $openvpn_interface
        is_changes=true
    else
        echo "Table $table default route is OK"
    fi
    echo "----------------------------------------"
    echo ""
done
if [ "$is_changes" = true ] ; then
    commit
else
    echo "No changes"
fi
exit

chmod +x /config/pbr.sh

set system task-scheduler task openvpn-pbr executable path /config/pbr.sh
set system task-scheduler task openvpn-pbr interval 5m

Killswitch

If you dont want these clients using your WAN connection directly if the internet is down, add a WAN_OUT rule for these clients:

# show firewall name WAN_OUT
 default-action accept
 rule 10 {
     action accept
     state {
         established enable
         related enable
     }
 }
 rule 20 {
     action reject
     source {
         group {
             address-group VPN-PIA-NO
         }
     }
 }
 # repeat for all firewall groups
 
 # show interfaces ethernet eth0 firewall 
 in {
     name WAN_IN
 }
 local {
     name WAN_LOCAL
 }
 out {
     name WAN_OUT
 }

Done!

Im now seeing downloads at my full 300mbit connection speed to my ISP over the load balanced connection (parallel downloads only).