EdgeRouter - Hardware Offloading Explained


Overview


This article explains the function, benefits, and implementation of hardware offloading. As of EdgeOS firmware version v1.9.1, all EdgeRouter models support some type of hardware offloading.

NOTES & REQUIREMENTS:
Applicable to the latest EdgeOS firmware on all EdgeRouter models. Please see the Related Articles below for more information.

Table of Contents


  1. What is Hardware Offloading?
  2. IPsec Offloading
  3. How to Enable/Disable Offloading
  4. Related Articles

What is Hardware Offloading?


Back to Top

Offloading is used to execute functions of the router using the hardware directly, rather than a process of software functions to greatly increase performance. Without offloading the router will use the CPU for all decisions. The benefit of offloading in EdgeOS is increased performance and throughput by not depending on the CPU for forwarding decisions.

There are many processes/features that can take advantage of the offloading engine. One of the most basic examples is IPv4 traffic forwarding. Without offloading enabled IPv4 traffic will be routed via the CPU and will be limited to around 300Mbps on the EdgeRouter Lite (ERLite-3). With offloading enabled the throughput will be about 950Mbps.

Some processes currently can not take advantage of hardware offloading. This can lead to confusion where it is assumed that offloading is disabled. A more accurate statement is that some features are not eligible to be offloaded and will always depend on the CPU. The tables below summarizes the features that can be offloaded on each platform.

MediaTek Devices (ER-X / ER-X-SFP / EP-R6)

 Feature Eligible for Offloading Enabled by Command
Bridged Interfaces (br) hwnat
Deep Packet Inspection (DPI) hwnat
NAT hwnat
VLANs hwnat
GRE hwnat
PPPoE hwnat
IPsec ipsec
QoS -
NetFlow -
Bonding (802.3ad) -

 

Cavium Devices (ER-4 / ER-6P / ERLite-3 / ERPoE-5 / ER-8 / ERPro-8 / EP-R8 / ER-8-XG )

 Feature Eligible for Offloading Enabled by Command
Bridged Interfaces (br) -
Deep Packet Inspection (DPI) ipv4 forwarding
NAT ipv4 forwarding
VLANs ipv4 / ipv6 vlan
GRE ipv4 gre
PPPoE ipv4 / ipv6 pppoe
IPsec ipsec
QoS -
NetFlow -
Bonding (802.3ad) -

IPsec Offloading


Back to Top

IPsec offload provides significant IPsec performance improvements, increasing throughput for site-to-site and client-to-site tunnels by offloading the ESP (Encapsulated Security Payload) traffic. Not all available ESP hashing/encryption algorithms are compatible with offloading. IKE traffic is not offloaded but this is only used to establish the tunnel and will not affect the performance. The tables below summarizes the algorithms that can be offloaded on each platform.

MediaTek Devices (ER-X / ER-X-SFP / EP-R6)

ESP Encryption/Hashing Algorithm Eligible for Offloading
3DES / AES-128 / AES-256 / MD5 / SHA-1 / SHA-256
SHA-384 / SHA-512 / AES-128-GCM / AES-256-GCM

IKE does not support GCM ciphers on these devices.

 

Cavium Devices (ER-4 / ER-6P / ERLite-3 / ERPoE-5 / ER-8 / ERPro-8 / EP-R8 / ER-8-XG )

ESP Encryption/Hashing Algorithm Eligible for Offloading
3DES / AES-128 / AES-256 / MD5 / SHA-1
SHA-256 / SHA-384 / SHA-512 / AES-128-GCM / AES-256-GCM

How to Enable/Disable Offloading


Back to Top

That commands that enable/disable offloading are shown below. You should only need to enable offloading for these features if you are using them in your environment. However, enabling offloading for all features will not cause a negative impact if those features are not being used.

ATTENTION: Offloading needs to be explicitly enabled/disabled with the set system offload ... statements. It is not possible to disable offloading by deleting the entire offloading sub-tree in the system configuration, for example, delete system offload.

MediaTek Devices (ER-X / ER-X-SFP / EP-R6) 

CLI: Access the command line interface (CLI). You can do this using the CLI button in the GUI or by using a program such as PuTTY.

Enable hwnat and ipsec offloading.

configure

set system offload hwnat enable
set system offload ipsec enable

commit ; save

Disable hwnat and ipsec offloading.

configure

set system offload hwnat disable
set system offload ipsec disable

commit ; save
NOTE: IPsec offloading requires a device reboot to become active/inactive.

Cavium Devices (ER-4 / ER-6P / ERLite-3 / ERPoE-5 / ER-8 / ERPro-8 / EP-R8 / ER-8-XG )

CLI: Access the command line interface (CLI). You can do this using the CLI button in the GUI or by using a program such as PuTTY.

Enable IPv4/IPv6 and ipsec offloading.

configure

set system offload ipv4 forwarding enable
set system offload ipv4 gre enable
set system offload ipv4 pppoe enable
set system offload ipv4 vlan enable

set system offload ipv6 forwarding enable
set system offload ipv6 pppoe enable
set system offload ipv6 vlan enable

set system offload ipsec enable

commit ; save

Disable IPv4/IPv6 and ipsec offloading.

configure

set system offload ipv4 forwarding disable
set system offload ipv4 gre disable
set system offload ipv4 pppoe disable
set system offload ipv4 vlan disable

set system offload ipv6 forwarding disable
set system offload ipv6 pppoe disable
set system offload ipv6 vlan disable

set system offload ipsec disable

commit ; save
NOTE: It is currently not possible to enable IPv6 offloading for PPPoE and VLANs simultaneously. IPsec offloading requires a device reboot to become active/inactive.

Verify the offloading state by running the following command in operational mode.

show ubnt offload  

IP offload module : loaded
IPv4
forwarding: enabled
vlan : enabled
pppoe : enabled
gre : enabled
IPv6
forwarding: disabled
vlan : disabled
pppoe : disabled

IPSec offload module: loaded

Traffic Analysis :
export : enabled
dpi : enabled

When high throughput is flowing on the router and the traffic is not offloaded, you will see an increase in CPU usage. When traffic is offloaded the throughput will be higher and CPU usage will remain low.

Using a tool like iPerf / iPerf3 is a common way to generate and test throughput. It is important to not use the EdgeRouter as the server or client for iPerf when running the test, as the routers are designed to route/forward traffic and not to generate it.


Related Articles


Back to Top