EdgeRouter Lite Performance Tests

Starting with two laptops connected to one ERL each. ERLs both running 1.5.0.

Laptop A <-> ERL <-> ERL <-> Laptop B
192.168.20.100 <-> 192.168.20.1 192.168.1.2 <-> 192.168.1.1 192.168.10.1 <-> 192.168.10.100

I’m using OSPF to expose these two networks to each other for this test. See EdgeRouter Lite Simple OSPF Guide

iperf results from one laptop to the other:

$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 42797
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.10 GBytes 940 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 42820
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.0 sec 11.0 GBytes 941 Mbits/sec

 Near wire. Great!

 Next we’ll remove the OSPF routing and setup an IPSec tunnel between the two an retest.

I used the GUI’s IPSec site-to-site feature to set this up. Here’s what it generated:

vpn {
    ipsec {
        auto-firewall-nat-exclude enable
        esp-group FOO0 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes128
                hash sha1
            }
        }
        ike-group FOO0 {
            lifetime 28800
            proposal 1 {
                dh-group 14
                encryption aes128
                hash sha1
            }
        }
        ipsec-interfaces {
            interface eth0
        }
        nat-networks {
            allowed-network 0.0.0.0/0 {
            }
        }
        nat-traversal enable
        site-to-site {
            peer 192.168.1.2 {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret ****************
                }
                connection-type initiate
                ike-group FOO0
                local-ip 192.168.1.1
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO0
                    local {
                        subnet 192.168.10.0/24
                    }
                    remote {
                        subnet 192.168.20.0/24
                    }
                }
            }
        }
    }
}

And the results with IPSec hardware offload disabled.

'set system offload ipsec disable'

[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44600
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.1 sec 718 MBytes 60.2 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44601
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 72.5 MBytes 60.5 Mbits/sec

And with IPSec hardware offload enabled.

'set system offload ipsec enable'

[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44586
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 180 MBytes 151 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44590
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 179 MBytes 150 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44596
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.0 sec 1.75 GBytes 151 Mbits/sec

Not bad!

However, even with IPSec enabled, load average increases a good bit when pushing max data through the IPSec VPN

jdrews@ERL10:~$ uptime
 10:23:23 up 23 min, 1 user, load average: 0.58, 0.66, 0.47

Compared to without pushing data

jdrews@ERL10:~$ uptime
 20:47:29 up 10:47, 1 user, load average: 0.00, 0.01, 0.05

While this didn’t seem to affect the usage of the router, it is a little worrying. I don’t have any firewalls or other services enabled on the router. What happens when we take this into the real world? It’s worth noting that this is a dual core router, so really that’s only 1/4 of total processing power.