Subject: Re: [RFC] kvm tools: Implement multiple VQ for virtio-net

From: Asias He <[email-masked]>
Date: Mon, 14 Nov 2011 10:04:25 +0800

Hi, Shsha

On 11/13/2011 11:00 PM, Sasha Levin wrote:
> On Sun, 2011-11-13 at 12:24 +0200, Michael S. Tsirkin wrote:
>> On Sat, Nov 12, 2011 at 12:12:01AM +0200, Sasha Levin wrote:
>>> This is a patch based on Krishna Kumar's patch series which implements
>>> multiple VQ support for virtio-net.
>>>
>>> The patch was tested with ver3 of the patch.
>>>
>>> Cc: Krishna Kumar<[email-masked]>
>>> Cc: Michael S. Tsirkin<[email-masked]>
>>> Cc: Rusty Russell<[email-masked]>
>>> Cc: [email-masked]
>>> Cc: [email-masked]
>>> Signed-off-by: Sasha Levin<[email-masked]>
>>
>> Any performance numbers?
>
> I tried finding a box with more than two cores so I could test it on
> something like that as well.
>
>> From what I see this patch causes a performance regression on my 2 core
> box.
>
> I'll send an updated KVM tools patch in a bit as well.
>
> Before:
>
> # netperf -H 192.168.33.4,ipv4 -t TCP_RR
> MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
> to 192.168.33.4 (192.168.33.4) port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
>
> 16384  87380  1        1       10.00    11160.63
> 16384  87380
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_RR
> MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
> to 192.168.33.4 (192.168.33.4) port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
>
> 122880 122880 1        1       10.00    12072.64
> 229376 229376
>
> # netperf -H 192.168.33.4,ipv4 -t TCP_STREAM
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>
>   87380  16384  16384    10.00    4654.50
>
> netperf -H 192.168.33.4,ipv4 -t TCP_STREAM -- -m 128
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>
>   87380  16384    128    10.00     635.45
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_STREAM
> MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Socket  Message  Elapsed      Messages
> Size    Size     Time         Okay Errors   Throughput
> bytes   bytes    secs            #      #   10^6bits/sec
>
> 122880   65507   10.00      113894      0    5968.54
> 229376           10.00       89373           4683.54
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_STREAM -- -m 128
> MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Socket  Message  Elapsed      Messages
> Size    Size     Time         Okay Errors   Throughput
> bytes   bytes    secs            #      #   10^6bits/sec
>
> 122880     128   10.00      550634      0      56.38
> 229376           10.00      398786             40.84
>
>
> After:
>
> # netperf -H 192.168.33.4,ipv4 -t TCP_RR
> MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
> to 192.168.33.4 (192.168.33.4) port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
>
> 16384  87380  1        1       10.00    8952.47
> 16384  87380
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_RR
> MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
> to 192.168.33.4 (192.168.33.4) port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
>
> 122880 122880 1        1       10.00    9534.52
> 229376 229376
>
> # netperf -H 192.168.33.4,ipv4 -t TCP_STREAM
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>
>   87380  16384  16384    10.13    2278.23
>
> # netperf -H 192.168.33.4,ipv4 -t TCP_STREAM -- -m 128
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>
>   87380  16384    128    10.00     623.27
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_STREAM
> MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Socket  Message  Elapsed      Messages
> Size    Size     Time         Okay Errors   Throughput
> bytes   bytes    secs            #      #   10^6bits/sec
>
> 122880   65507   10.00      136930      0    7175.72
> 229376           10.00       16726            876.51
>
> # netperf -H 192.168.33.4,ipv4 -t UDP_STREAM -- -m 128
> MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 192.168.33.4 (192.168.33.4) port 0 AF_INET
> Socket  Message  Elapsed      Messages
> Size    Size     Time         Okay Errors   Throughput
> bytes   bytes    secs            #      #   10^6bits/sec
>
> 122880     128   10.00      982492      0     100.61
> 229376           10.00      249597             25.56
>

Why both the bandwidth and latency performance are dropping so 
dramatically with multiple VQ?

-- 
Asias He