Subject: Re: [PATCH v4 net-next 12/12] mlx5: support BIG TCP packets

From: Kees Cook <[email-masked]>
Date: Fri, 6 May 2022 23:57:15 -0700

On Fri, May 06, 2022 at 06:54:05PM -0700, Jakub Kicinski wrote:
> On Fri, 6 May 2022 17:32:43 -0700 Eric Dumazet wrote:
> > On Fri, May 6, 2022 at 3:34 PM Jakub Kicinski <[email-masked]> wrote:
> > >
> > > On Fri,  6 May 2022 08:30:48 -0700 Eric Dumazet wrote:  
> > > > From: Coco Li <[email-masked]>
> > > >
> > > > mlx5 supports LSOv2.
> > > >
> > > > IPv6 gro/tcp stacks insert a temporary Hop-by-Hop header
> > > > with JUMBO TLV for big packets.
> > > >
> > > > We need to ignore/skip this HBH header when populating TX descriptor.
> > > >
> > > > Note that ipv6_has_hopopt_jumbo() only recognizes very specific packet
> > > > layout, thus mlx5e_sq_xmit_wqe() is taking care of this layout only.
> > > >
> > > > v2: clear hopbyhop in mlx5e_tx_get_gso_ihs()
> > > > v4: fix compile error for CONFIG_MLX5_CORE_IPOIB=y  
> > >
> > > In file included from ../include/linux/string.h:253,
> > >                  from ../arch/x86/include/asm/page_32.h:22,
> > >                  from ../arch/x86/include/asm/page.h:14,
> > >                  from ../arch/x86/include/asm/processor.h:19,
> > >                  from ../arch/x86/include/asm/timex.h:5,
> > >                  from ../include/linux/timex.h:65,
> > >                  from ../include/linux/time32.h:13,
> > >                  from ../include/linux/time.h:60,
> > >                  from ../include/linux/skbuff.h:15,
> > >                  from ../include/linux/tcp.h:17,
> > >                  from ../drivers/net/ethernet/mellanox/mlx5/core/en_tx.c:33:
> > > In function ‘fortify_memcpy_chk’,
> > >     inlined from ‘mlx5e_insert_vlan’ at ../drivers/net/ethernet/mellanox/mlx5/core/en_tx.c:104:2,
> > >     inlined from ‘mlx5e_sq_xmit_wqe’ at ../drivers/net/ethernet/mellanox/mlx5/core/en_tx.c:404:5:
> > > ../include/linux/fortify-string.h:328:25: warning: call to ‘__write_overflow_field’ declared with attribute warning: detected write beyond size of field (1st parameter); maybe use struct_group()? [-Wattribute-warning]
> > >   328 |                         __write_overflow_field(p_size_field, size);
> > >       |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > In function ‘fortify_memcpy_chk’,
> > >     inlined from ‘mlx5e_sq_xmit_wqe’ at ../drivers/net/ethernet/mellanox/mlx5/core/en_tx.c:408:5:
> > > ../include/linux/fortify-string.h:328:25: warning: call to ‘__write_overflow_field’ declared with attribute warning: detected write beyond size of field (1st parameter); maybe use struct_group()? [-Wattribute-warning]
> > >   328 |                         __write_overflow_field(p_size_field, size);
> > >       |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > In function ‘fortify_memcpy_chk’,
> > >     inlined from ‘mlx5i_sq_xmit’ at ../drivers/net/ethernet/mellanox/mlx5/core/en_tx.c:962:4:
> > > ../include/linux/fortify-string.h:328:25: warning: call to ‘__write_overflow_field’ declared with attribute warning: detected write beyond size of field (1st parameter); maybe use struct_group()? [-Wattribute-warning]
> > >   328 |                         __write_overflow_field(p_size_field, size);
> > >       |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
> > 
> > I guess these warnings show up before this BIG TCP patch ?
> > 
> > I do not see any struct_group() being used in mlx5
> > 
> > May I ask which compiler is used here, and what CONFIG_ option needs to be set ?
> > 
> > Thanks.
> 
> Without our patches drivers/net/ethernet/mellanox/mlx5/core/ builds
> cleanly. Gotta be the new W=1 filed overflow warnings, let's bother
> Kees.

Hello!

These aren't from W=1. The read overflows are hidden behind W=1. I
imagine this is due to gcc getting smarter and being able to introspect
the possible values of ihs during inlining.

> I believe this is the code in question:
> 
> @@ -379,15 +393,36 @@ mlx5e_sq_xmit_wqe(struct mlx5e_txqsq *sq, struct sk_buff *skb,
> 
> +		u8 *start = eseg->inline_hdr.start;
> +
> +		if (unlikely(attr->hopbyhop)) {
> +			/* remove the HBH header.
> +			 * Layout: [Ethernet header][IPv6 header][HBH][TCP header]
> +			 */
> +			if (skb_vlan_tag_present(skb)) {
> +				mlx5e_insert_vlan(start, skb, ETH_HLEN + sizeof(*h6));
> 
> Unhappiness #1 ^^^
> 
> Where mlx5e_insert_vlan() is:
> 
> static inline void mlx5e_insert_vlan(void *start, struct sk_buff *skb, u16 ihs)
> {
> 	struct vlan_ethhdr *vhdr = (struct vlan_ethhdr *)start;
> 	int cpy1_sz = 2 * ETH_ALEN;
> 	int cpy2_sz = ihs - cpy1_sz;

Why are these "int"? Seems like they should be u16?

> 
> 	memcpy(&vhdr->addrs, skb->data, cpy1_sz);
               ^^^^^ this line was actually fixed earlier.

> 	vhdr->h_vlan_proto = skb->vlan_proto;
> 	vhdr->h_vlan_TCI = cpu_to_be16(skb_vlan_tag_get(skb));
> 	memcpy(&vhdr->h_vlan_encapsulated_proto, skb->data + cpy1_sz, cpy2_sz);
               ^^^^^
This one, though, is the new problem. The lack of annotation in the
struct made me miss it -- this code is asking the compiler to
potentially copy beyond the end of the struct declaration. If this is
intentional, I could suggest a solution, but ...

> }
> 
> indeed ihs == ETH_HLEN + sizeof(*h6) will make cpy2_sz come out as something
> much bigger than the vhdr->h_vlan_encapsulated_proto field.

It sounds like it's not. In which case, I would ask: "what validates the
size of ihs?" because neither I nor the compiler can see it. :P If
nothing validates it, then this looks like a potential heap overflow,
though I haven't studied how these is laid out in memory. Maybe it's
harmless, but I never assume that. :)

-- 
Kees Cook