Subject: Issue with GRE tunnel, created with "local any" on kernel 3.14

From: Sergey Popov <[email-masked]>
Date: Fri, 18 Jul 2014 17:01:23 +0400

Hi. I have updated kernel on one of my servers to 3.14 and hit strange
issue:

Simple setup:

serv1(172.30.0.100) <-> serv2(172.30.0.251)

Serv1 is on 3.11.7
Serv2 is on 3.13.6

Gre tunnel(serv1):
ip tunnel add tun_test mode gre remote 172.30.0.251 local any ttl 225
ifconfig tun_test 192.168.0.1/30

Gre tunnel(serv2):
ip tunnel add tun_test mode gre remote 172.30.0.100 local any ttl 225
ifconfig tun_test 192.168.0.2/30

Everything work as expected.

Now, with updated kernel on serv2 i have this:

serv2 ~ # ping 192.168.0.1 -c 10
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.339 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=0.303 ms

--- 192.168.0.1 ping statistics ---
10 packets transmitted, 2 received, 80% packet loss, time 9002ms
rtt min/avg/max/mdev = 0.303/0.321/0.339/0.018 ms

Huh? tcpdump on serv1 show me really strange thing:

bgp ~ # tcpdump -i lan0 -nn ip proto gre
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lan0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:49:33.338964 IP 172.30.0.251 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 1, length 64
16:49:33.339010 IP 172.30.0.100 > 172.30.0.251: GREv0, length 88: IP
192.168.0.1 > 192.168.0.2: ICMP echo reply, id 5259, seq 1, length 64
16:49:34.337945 IP 172.30.0.251 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 2, length 64
16:49:34.337988 IP 172.30.0.100 > 172.30.0.251: GREv0, length 88: IP
192.168.0.1 > 192.168.0.2: ICMP echo reply, id 5259, seq 2, length 64
16:49:35.336948 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 3, length 64
16:49:36.340999 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 4, length 64
16:49:37.340971 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 5, length 64
16:49:38.340985 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 6, length 64
16:49:39.340971 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 7, length 64
16:49:40.340918 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 8, length 64
16:49:41.340947 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 9, length 64
16:49:42.340970 IP 0.0.0.0 > 172.30.0.100: GREv0, length 88: IP
192.168.0.2 > 192.168.0.1: ICMP echo request, id 5259, seq 10, length 64
^C
12 packets captured
14 packets received by filter
0 packets dropped by kernel

Huh? 0.0.0.0? o_O

I have tried to up and done tunnels few times. Sometimes 0.0.0.0 is
replaced by correct address in the middle of icmp "session" for one or
two packets, but usually only first one or two packets have correct src
address.

I have tried some(not all of them) kernels from 3.14.2 up to 3.15.5 -
all have the same issue.

If i set "local 172.30.0.251" on serv2 the issue is gone completely.

Looking on changelog i discovered some fixes to gre, but i could not
understand if they can be culprit or not.

MTU on both real network ifaces are 1500, MTU on gre tunnels are 1476,
so i suppose that's right(and not an issue, cause mentioned icmp packets
are too small to hit MTU problem).

I am running Gentoo Linux, but issue is detected in vanilla kernel too,
so it's not distro problem, i suppose.

Can anybody help me to track this issue down?

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop-effects project lead
Gentoo Proxy maintainers project lead