Subject: Re: [PATCH vhost v11 05/10] virtio_ring: introduce virtqueue_dma_dev()

From: Jakub Kicinski <[email-masked]>
Date: Tue, 1 Aug 2023 08:45:10 -0700

On Tue, 1 Aug 2023 10:57:30 +0800 Xuan Zhuo wrote:
> > You have this working and benchmarked or this is just and idea?  
> 
> This is not just an idea. I said that has been used on large scale.
> 
> This is the library for the APP to use the AF_XDP. We has open it.
> https://gitee.com/anolis/libxudp
> 
> This is the Alibaba version of the nginx. That has been opened, that supported
> to work with the libray to use AF_XDP.
> http://tengine.taobao.org/
> 
> I supported this on our kernel release Anolis/Alinux.

Interesting!

> The work was done about 2 years ago. You know, I pushed the first version to
> enable AF_XDP on virtio-net about two years ago. I never thought the job would
> be so difficult.

Me neither, but it is what it is.

> The nic (virtio-net) of AliYun can reach 24,000,000PPS.
> So I think there is no different with the real HW on the performance.
> 
> With the AF_XDP, the UDP pps is seven times that of the kernel udp stack.

UDP pps or QUIC pps? UDP with or without GSO?

Do you have measurements of how much it saves in real world workloads?
I'm asking mostly out of curiosity, not to question the use case.

> > What about io_uring zero copy w/ pre-registered buffers.
> > You'll get csum offload, GSO, all the normal perf features.  
> 
> We tried io-uring, but it was not suitable for our scenario.
> 
> Yes, now the AF_XDP does not support the csum offload and GSO.
> This is indeed a small problem.

Can you say more about io-uring suitability? It can do zero copy
and recently-ish Pavel optimized it quite a bit.