Subject: TCP MD5 connections get stuck with IPV6 mapped listener

From: Stephen Hemminger <[email-masked]>
Date: Wed, 15 Jul 2009 12:11:19 -0700

While investigating some TCP MD5 issues, with BGP, broke things
up into a simple test program found that
if server is listening on IPV6 for TCP with MD5 key
and client connects with IPV4 MD5 key, the connection gets stuck after
the initial SYN-SENT, SYN-ACK.

The computed value on the SYN-ACK for MD5 is incorrect, so looks like
some issue in tcp_v6_syn_recv_sock path

Note: key with IPV4mapped
IPV6 address (ie. expects incoming IPV4 client).

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
18:40:04.030651 IP localhost.2030 > localhost.2020: S 1832536952:1832536952(0) win 32792 <nop,nop,md5:can't check - c32795a5d779fd628bbb64da9e859338,mss 16396,nop,nop,sackOK,nop,wscale 6>
18:40:04.030701 IP localhost.2020 > localhost.2030: S 1841519788:1841519788(0) ack 1832536953 win 32792 <nop,nop,md5:can't check - bc0f025d8fa81bf45afe656b21fef9ca,mss 16396,nop,nop,sackOK,nop,wscale 6>
18:40:07.026693 IP localhost.2030 > localhost.2020: S 1832536952:1832536952(0) win 32792 <nop,nop,md5:can't check - c32795a5d779fd628bbb64da9e859338,mss 16396,nop,nop,sackOK,nop,wscale 6>
18:40:07.026739 IP localhost.2020 > localhost.2030: S 1841519788:1841519788(0) ack 1832536953 win 32792 <nop,nop,md5:can't check - bc0f025d8fa81bf45afe656b21fef9ca,mss 16396,nop,nop,sackOK,nop,wscale 6>
18:40:08.226682 IP localhost.2020 > localhost.2030: S 1841519788:1841519788(0) ack 1832536953 win 32792 <nop,nop,md5:can't check - bc0f025d8fa81bf45afe656b21fef9ca,mss 16396,nop,nop,sackOK,nop,wscale 6>

To reproduce:
 $ server -6 2020 127.0.0.1 2030 testkey &
 $ client -4 localhost 2020 127.0.0.1 2030 testkey

...