EdgeRouter Lite Performance Tests

Starting with two laptops connected to one ERL each. ERLs both running 1.5.0.

Laptop A <-> ERL <-> ERL <-> Laptop B
192.168.20.100 <-> 192.168.20.1 192.168.1.2 <-> 192.168.1.1 192.168.10.1 <-> 192.168.10.10020141010_22362

 

I’m using OSPF to expose these two networks to each other for this test. See EdgeRouter Lite Simple OSPF Guide

iperf results from one laptop to the other:

$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 42797
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.10 GBytes 940 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 42820
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.0 sec 11.0 GBytes 941 Mbits/sec

 Near wire. Great!

 Next we’ll remove the OSPF routing and setup an IPSec tunnel between the two an retest.

I used the GUI’s IPSec site-to-site feature to set this up. Here’s what it generated:

vpn {
    ipsec {
        auto-firewall-nat-exclude enable
        esp-group FOO0 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes128
                hash sha1
            }
        }
        ike-group FOO0 {
            lifetime 28800
            proposal 1 {
                dh-group 14
                encryption aes128
                hash sha1
            }
        }
        ipsec-interfaces {
            interface eth0
        }
        nat-networks {
            allowed-network 0.0.0.0/0 {
            }
        }
        nat-traversal enable
        site-to-site {
            peer 192.168.1.2 {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret ****************
                }
                connection-type initiate
                ike-group FOO0
                local-ip 192.168.1.1
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO0
                    local {
                        subnet 192.168.10.0/24
                    }
                    remote {
                        subnet 192.168.20.0/24
                    }
                }
            }
        }
    }
}

And the results with IPSec hardware offload disabled.

'set system offload ipsec disable'

[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44600
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.1 sec 718 MBytes 60.2 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44601
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 72.5 MBytes 60.5 Mbits/sec

And with IPSec hardware offload enabled.

'set system offload ipsec enable'

[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44586
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 180 MBytes 151 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44590
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 179 MBytes 150 Mbits/sec
[ 4] local 192.168.10.100 port 5001 connected with 192.168.20.100 port 44596
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-100.0 sec 1.75 GBytes 151 Mbits/sec

Not bad!

However, even with IPSec enabled, load average increases a good bit when pushing max data through the IPSec VPN

jdrews@ERL10:~$ uptime
 10:23:23 up 23 min, 1 user, load average: 0.58, 0.66, 0.47

Compared to without pushing data

jdrews@ERL10:~$ uptime
 20:47:29 up 10:47, 1 user, load average: 0.00, 0.01, 0.05

While this didn’t seem to affect the usage of the router, it is a little worrying. I don’t have any firewalls or other services enabled on the router. What happens when we take this into the real world? It’s worth noting that this is a dual core router, so really that’s only 1/4 of total processing power.

EdgeRouter Lite Simple OSPF Guide

Here’s a really simple OSPF setup on two Ubiquiti Edge Router Lites.

Two ERLs hooked together via eth0.

ERL20 <-> ERL10
192.168.20.1 192.168.1.2 <-> 192.168.1.1 192.168.10.1

20141010_22362

on ERL20 get into configure and run the following:

# give this router an id. 
set protocols ospf parameters router-id 192.168.20.1

# 0.0.0.0 is backbone. Add in networks to advertise. 
set protocols ospf area 0.0.0.0 network 192.168.1.0/24
set protocols ospf area 0.0.0.0 network 192.168.20.0/24

# enable ospf on eth0 interface (going to ERL10) 
set interfaces ethernet eth0 ip ospf

on ERL10 get into configure and run the following.

# give this router an id. 
set protocols ospf parameters router-id 192.168.10.1

# 0.0.0.0 is backbone. Add in networks to advertise. 
set protocols ospf area 0.0.0.0 network 192.168.1.0/24
set protocols ospf area 0.0.0.0 network 192.168.10.0/24

# enable ospf on eth0 interface (going to ERL20) 
set interfaces ethernet eth0 ip ospf

run ‘commit’ on both them, (‘save’ if you want to) and ‘exit’ to prompt.

Now watch ospf work it’s magic.

jdrews@ERL10:~$ show ip ospf neighbor

 Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.20.1 1 2-Way/DROther 34.213s 192.168.1.2 eth0:192.168.1.1 0 0 0

jdrews@ERL20:~$ show ip ospf neighbor

 Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.10.1 1 2-Way/DROther 36.422s 192.168.1.1 eth0:192.168.1.2 0 0 0

jdrews@ERL10:~$ show ip ospf neighbor

 Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.20.1 1 ExStart/DR 30.837s 192.168.1.2 eth0:192.168.1.1 0 0 0

jdrews@ERL20:~$ show ip ospf neighbor

 Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.10.1 1 Full/Backup 39.764s 192.168.1.1 eth0:192.168.1.2 2 0 0

jdrews@ERL10:~$ show ip ospf neighbor

 Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.20.1 1 Full/DR 39.674s 192.168.1.2 eth0:192.168.1.1 1 0 0

Now you should have routes

jdrews@ERL10:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
 I - ISIS, B - BGP, > - selected route, * - FIB route

C>* 127.0.0.0/8 is directly connected, lo
O 192.168.1.0/24 [110/10] is directly connected, eth0, 00:11:40
C>* 192.168.1.0/24 is directly connected, eth0
O 192.168.10.0/24 [110/10] is directly connected, eth1, 00:11:39
C>* 192.168.10.0/24 is directly connected, eth1
O>* 192.168.20.0/24 [110/20] via 192.168.1.2, eth0, 00:10:58

jdrews@ERL20:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
 I - ISIS, B - BGP, > - selected route, * - FIB route

C>* 127.0.0.0/8 is directly connected, lo
O 192.168.1.0/24 [110/10] is directly connected, eth0, 00:11:34
C>* 192.168.1.0/24 is directly connected, eth0
O>* 192.168.10.0/24 [110/20] via 192.168.1.1, eth0, 00:10:44
O 192.168.20.0/24 [110/10] is directly connected, eth1, 00:11:33
C>* 192.168.20.0/24 is directly connected, eth1