EdgeRouter : Dual WAN with Hair Pin

Starting with firmware version 1.4 EdgeRouter can be configured in a dual WAN setup using a very simple wizard.

Then to configure some port forwarding rules to access the internal web server or VPNs there is a nice GUI setup window.

Note that I experienced frequent switches between the main WAN and the backup WAN. The issue was caused by the slow DNS (UPC DNS has issues) which was causing pings to the default ping target of the watchdog ping.ubnt.com to fail.
I decided to ping a more reliable target and also use an IP to avoid the DNS issues.

configured
set load-balance group G interface eth0 route-test type ping target 8.8.8.8
set load-balance group G interface eth1 route-test type ping target 8.8.8.8
commit
save
exit

The test if the changes are done correctly

root@ubnt:~# show load-balance watchdog
Group G
eth0
status: Running
pings: 27
fails: 0
run fails: 0/3
route drops: 0
ping gateway: 8.8.8.8 – REACHABLE

eth1
status: Running
failover-only mode
pings: 0
fails: 0
run fails: 0/3
route drops: 4
ping gateway: 8.8.8.8 – REACHABLE
last route drop : Mon Oct 19 20:15:26 2015
last route recover: Mon Oct 19 20:49:21 2015

Note that to be able to access from inside the network the forwarded resources we need the so called “Hair Pin” box checked.

The above configuration done from GUI is enough for standard port forwarding and hair pin in the single WAN setup. In case of dual WAN setup the load-balance feature assume the traffic goes from LAN to WAN or vise versa, but in the case of port-forward hairpin the traffic goes from LAN to router and back out to LAN. So we can add a rule to our modify chain to NOT load-balance traffic that’s destination address is the WAN address.

The following CLI configuration will do the trick:

configure
edit firewall modify balance
rename rule 1 to rule 20
commit
set rule 10 modify table main
set rule 10 destination group address-group ADDRv4_eth0
commit
save
exit

Note that the port-forward feature currently only supports 1 wan interface (eth0 in my case), so during fail-over your port forward won’t work. This is a big issue at this point but can be configured outside the EdgeRouter. In my case the fail-over WAN (eth1) consist of a mobile 4G connection through a D-Link DWR-921 LTE Router. By defining another set of port forwarding rules on the D-Link router and several routing rules it should be possible to port-forward also through eth1.

Another important change is to add some DNS changes. Both WANs should be able to see the same DNS so disable the ISP specific DNSs.

configure
set interfaces ethernet eth0 dhcp-options name-server no-update
set interfaces ethernet eth1 dhcp-options name-server no-update
commit
save

configure
set system name-server 8.8.8.8
commit
save
exit

In order to avoid the switch from main WAN to the secondary WAN also make sure the sticky connection is forced.

configure
set load-balance group G sticky source-addr enable
set load-balance group G sticky dest-addr enable
set load-balance group G sticky dest-port enable
commit
save
exit

All seemed well but then I hit a wall. My mobile connection WAN was randomly promoted to main WAN without any reason.I struggled for several days until I decided to create a new thread on EdgeRouter help forum (see thread3 bellow).

By looking at the output of “show load-balance status” and “show load-balance watchdog” as instructed by UBNT-stig (the very helpful forum moderator) we noticed that WAN OUT was empty for both WAN interfaces.

Note that during a network speed test run both WANs seem to work at full speed (orange is the mobile WAN, red the UPC WAN).

To quote UBNT-stig:

Hmm “WAN Out” is empty. I wonder if you ran into a bug I fixed last week. Can you try doing “sudo pkill ubnt-util” from the CLI and then see if “WAN Out” gets filled in?

This finally fixed all the issues. It looks like somehow the load-balancer was acting as a classic load-balancer not as a fail-over only setup as I wanted. By killing the ubnt-util process that in fact implements the load-balancing capabilities of the EdgeRouter and forcing the reload of the configuration things got fixed.

Note that during a network speed test run only the UPC WAN works at full speed (Vodafone 4G is the mobile WAN, red the UPC WAN) and the mobile WAN sits idle.

Thanks to the following Ubiquity Forum threads that clarified this for me.
thread 1
thread2
thread 3