Subject: Re: Problem about gre tunnel route offload in mlxsw

From: Ido Schimmel <[email-masked]>
Date: Thu, 26 Dec 2019 09:49:14 +0200

On Thu, Dec 26, 2019 at 01:07:33PM +0800, wenxu wrote:
> Hi mlxsw team,
> 
> 
> I did a route test with gre tunnel and vrf.
> 
> This test under current net-next tree with following script:
> 
> 
> ifconfig enp3s0np31 up
> ip a a dev enp3s0np31 172.168.152.247/24
> 
> ip l add dev vrf11 type vrf table 11
> ifconfig vrf11 up
> ip l set dev enp3s0np11 master vrf11
> ifconfig enp3s0np11 10.0.7.1/24 up
> ip tunnel add name gre11 mode gre local 172.168.152.247 remote 172.168.152.73 key 11 tos inherit  ttl inherit
> ip l set dev gre11 master vrf11
> ifconfig gre11 10.0.2.1/24 up
> 
> ip l add dev vrf21 type vrf table 21
> ifconfig vrf21 up
> ip l set dev enp3s0np21 master vrf21
> ifconfig enp3s0np21 10.0.7.1/24 up
> ip tunnel add name gre21 mode gre local 172.168.152.247 remote 172.168.152.73 key 21 tos inherit  ttl inherit
> ip l set dev gre21 master vrf21
> ifconfig gre21 10.0.2.1/24 up
> 
> 
> If there is only one tunnel. The route rule can be offloaded. But two tunnel only with different key can't be offloaded.
> 
> If I add a new address 172.168.152.248 for tunnel source and change the gre21 to
> 
> "ip tunnel add name gre21 mode gre local 172.168.152.248 remote 172.168.152.73 key 21 tos inherit  ttl inherit"
> 
> It's work.
> 
> So it means dispatch based on tunnel key is not supported ?

Yes. See:
"No two tunnels that share underlay VRF shall share a local address
(i.e. dispatch based on tunnel key is not supported)"

https://github.com/Mellanox/mlxsw/wiki/L3-Tunneling#features-and-limitations

> It is a hardware limits or just software unsupported?

Software. In hardware you can perform decapsulation in the router or
using ACLs. mlxsw uses the former so the key is {tb_id, prefix}. With
ACLs it is possible to match on more attributes.

> 
> 
> And if a replace the gre device to vxlan device,  the route can't be offloaded again only with one vxlan tunnel.
> 
> "ip l add dev vxlan11 type vxlan local 172.168.152.247 remote 172.168.152.73 id 11 noudpcsum tos inherit ttl inherit dstport 4789"
> 
> So currently the vxlan device can't work with routing?

Yes. We don't perform VXLAN encapsulation in the router, only in bridge.
See:
https://github.com/Mellanox/mlxsw/wiki/Virtual-eXtensible-Local-Area-Network-%28VXLAN%29

> BR
> wenxu