Subject: Re: [kvm-devel] QEMU PIC indirection patch for in-kernel APIC work

From: Avi Kivity <[email-masked]>
Date: Mon, 16 Apr 2007 08:13:28 +0300

Rusty Russell wrote:
> On Thu, 2007-04-12 at 06:32 +0300, Avi Kivity wrote:
>   
>> I hadn't considered an always-blocking (or unbuffered) networking API. 
>> It's very counter to current APIs, but does make sense with things like
>> syslets.  Without syslets, I don't think it's very useful as you need
>> some artificial threads to keep things humming along.
>>
>> (How would userspace specify it? O_DIRECT when opening the tap?)
>>     
>
> TBH, I hadn't thought that far.  Tap already has those IFF_NO_PI etc
> flags, but it might make sense to just be the default.  From userspace's
> POV it's not a semantic change.
>
> OK, just tested: I can get 230,000 packets (28 byte UDP) through the tun
> device in a second (130,000 actually out the 100-base-T NIC, 100,000
> dropped).  If the tun driver's write() blocks until the skb is
> destroyed, it's 4,000 packets.
>
> So your intuition was right: skb_free latency on xmit (at least for this
> e1000) is far too large for anything but an async solution.
>
> Will ponder further.
>   

I think aio_write (but done copyless-lessly) is the way to go.  Not only
is the infrastructure there, but the API already allows for multiple
packet submission and for batching completions.  Fitting into that
framework ought to be easier than starting yet another one.

It still misses scatter/gather and integration with fd-based
notification, but there are patches around for that.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.