From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 00:27:43 +0300 Message-ID: <20090811212743.GA26309@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii To: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, akpm@linux-founda Return-path: Content-Disposition: inline Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org This implements vhost: a kernel-level backend for virtio, The main motivation for this work is to reduce virtualization overhead for virtio by removing system calls on data path, without guest changes. For virtio-net, this removes up to 4 system calls per packet: vm exit for kick, reentry for kick, iothread wakeup for packet, interrupt injection for packet. Some more detailed description attached to the patch itself. The patches are against 2.6.31-rc4. I'd like them to go into linux-next and down the road 2.6.32 if possible. Please comment. Changes from v1: - Move use_mm/unuse_mm from fs/aio.c to mm instead of copying. - Reorder code to avoid need for forward declarations - Kill a couple of debugging printks Michael S. Tsirkin (2): mm: export use_mm/unuse_mm to modules vhost_net: a kernel-level virtio server MAINTAINERS | 10 + arch/x86/kvm/Kconfig | 1 + drivers/Makefile | 1 + drivers/vhost/Kconfig | 11 + drivers/vhost/Makefile | 2 + drivers/vhost/net.c | 411 +++++++++++++++++++++++++++ drivers/vhost/vhost.c | 663 +++++++++++++++++++++++++++++++++++++++++++ drivers/vhost/vhost.h | 108 +++++++ fs/aio.c | 47 +--- include/linux/Kbuild | 1 + include/linux/miscdevice.h | 1 + include/linux/mmu_context.h | 9 + include/linux/vhost.h | 100 +++++++ mm/Makefile | 2 +- mm/mmu_context.c | 58 ++++ 15 files changed, 1378 insertions(+), 47 deletions(-) create mode 100644 drivers/vhost/Kconfig create mode 100644 drivers/vhost/Makefile create mode 100644 drivers/vhost/net.c create mode 100644 drivers/vhost/vhost.c create mode 100644 drivers/vhost/vhost.h create mode 100644 include/linux/mmu_context.h create mode 100644 include/linux/vhost.h create mode 100644 mm/mmu_context.c -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory Haskins Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Tue, 11 Aug 2009 19:49:37 -0400 Message-ID: <4A820391.1090404@gmail.com> References: <20090811212743.GA26309@redhat.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigA223C963729B04B1689E60E6" Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, akpm@linux-foundation.org, hpa@zytor.com To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20090811212743.GA26309@redhat.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA223C963729B04B1689E60E6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Michael S. Tsirkin wrote: > This implements vhost: a kernel-level backend for virtio, > The main motivation for this work is to reduce virtualization > overhead for virtio by removing system calls on data path, > without guest changes. For virtio-net, this removes up to > 4 system calls per packet: vm exit for kick, reentry for kick, > iothread wakeup for packet, interrupt injection for packet. >=20 > Some more detailed description attached to the patch itself. >=20 > The patches are against 2.6.31-rc4. I'd like them to go into linux-nex= t > and down the road 2.6.32 if possible. Please comment. I will add this series to my benchmark run in the next day or so. Any specific instructions on how to set it up and run? Regards, -Greg --------------enigA223C963729B04B1689E60E6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqCA5EACgkQP5K2CMvXmqHIvQCfYcoxCyKZvGg3C6EAOrpAEwIH JmkAnRmvb/eJZIoYNF9JHmDPGvTAdvT/ =dirP -----END PGP SIGNATURE----- --------------enigA223C963729B04B1689E60E6-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 10:16:36 +0300 Message-ID: <20090812071636.GA26847@redhat.com> References: <20090811212743.GA26309@redhat.com> <4A820391.1090404@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, akpm@linux-foundation.org, hpa@zytor.com To: Gregory Haskins Return-path: Content-Disposition: inline In-Reply-To: <4A820391.1090404@gmail.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > This implements vhost: a kernel-level backend for virtio, > > The main motivation for this work is to reduce virtualization > > overhead for virtio by removing system calls on data path, > > without guest changes. For virtio-net, this removes up to > > 4 system calls per packet: vm exit for kick, reentry for kick, > > iothread wakeup for packet, interrupt injection for packet. > > > > Some more detailed description attached to the patch itself. > > > > The patches are against 2.6.31-rc4. I'd like them to go into linux-next > > and down the road 2.6.32 if possible. Please comment. > > I will add this series to my benchmark run in the next day or so. Any > specific instructions on how to set it up and run? > > Regards, > -Greg > 1. use a dedicated network interface with SRIOV, program mac to match that of guest (for testing, you can set promisc mode, but that is bad for performance) 2. disable tso,gso,lro with ethtool 3. add vhost=ethX -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory Haskins Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 07:56:05 -0400 Message-ID: <4A82ADD5.6040909@gmail.com> References: <20090811212743.GA26309@redhat.com> <4A820391.1090404@gmail.com> <20090812071636.GA26847@redhat.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD7605C67DFA4B328715B65EB" Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, "akpm@linux-foundation.org >> Andrew Morton" , hpa@zytor.com, Patrick Mullaney To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20090812071636.GA26847@redhat.com> Sender: kvm-owner@vger.kernel.org List-Id: netdev.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD7605C67DFA4B328715B65EB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Michael S. Tsirkin wrote: > On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> This implements vhost: a kernel-level backend for virtio, >>> The main motivation for this work is to reduce virtualization >>> overhead for virtio by removing system calls on data path, >>> without guest changes. For virtio-net, this removes up to >>> 4 system calls per packet: vm exit for kick, reentry for kick, >>> iothread wakeup for packet, interrupt injection for packet. >>> >>> Some more detailed description attached to the patch itself. >>> >>> The patches are against 2.6.31-rc4. I'd like them to go into linux-n= ext >>> and down the road 2.6.32 if possible. Please comment. >> I will add this series to my benchmark run in the next day or so. Any= >> specific instructions on how to set it up and run? >> >> Regards, >> -Greg >> >=20 > 1. use a dedicated network interface with SRIOV, program mac to match > that of guest (for testing, you can set promisc mode, but that is > bad for performance) Are you saying SRIOV is a requirement, and I can either program the SRIOV adapter with a mac or use promis? Or are you saying I can use SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > 2. disable tso,gso,lro with ethtool Out of curiosity, wouldnt you only need to disable LRO on the adapter, since the other two (IIUC) are transmit path and are therefore influenced by the skb's you generate in vhost? > 3. add vhost=3DethX You mean via "ip link" I assume? Regards, -Greg --------------enigD7605C67DFA4B328715B65EB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqCrdUACgkQP5K2CMvXmqEOwACeNAOQtMMRFiCXlgHvg9A3/BC2 g4AAnRa44uxf7P8j1pmsxBIk2t1ehw2Q =Khle -----END PGP SIGNATURE----- --------------enigD7605C67DFA4B328715B65EB-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 15:05:41 +0300 Message-ID: <20090812120541.GA29158@redhat.com> References: <20090811212743.GA26309@redhat.com> <4A820391.1090404@gmail.com> <20090812071636.GA26847@redhat.com> <4A82ADD5.6040909@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, "akpm@linux-foundation.org >> Andrew Morton" , hpa@zytor.com, Patrick Mullaney To: Gregory Haskins Return-path: Content-Disposition: inline In-Reply-To: <4A82ADD5.6040909@gmail.com> Sender: kvm-owner@vger.kernel.org List-Id: netdev.vger.kernel.org On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: > >> Michael S. Tsirkin wrote: > >>> This implements vhost: a kernel-level backend for virtio, > >>> The main motivation for this work is to reduce virtualization > >>> overhead for virtio by removing system calls on data path, > >>> without guest changes. For virtio-net, this removes up to > >>> 4 system calls per packet: vm exit for kick, reentry for kick, > >>> iothread wakeup for packet, interrupt injection for packet. > >>> > >>> Some more detailed description attached to the patch itself. > >>> > >>> The patches are against 2.6.31-rc4. I'd like them to go into linux-next > >>> and down the road 2.6.32 if possible. Please comment. > >> I will add this series to my benchmark run in the next day or so. Any > >> specific instructions on how to set it up and run? > >> > >> Regards, > >> -Greg > >> > > > > 1. use a dedicated network interface with SRIOV, program mac to match > > that of guest (for testing, you can set promisc mode, but that is > > bad for performance) > > Are you saying SRIOV is a requirement, and I can either program the > SRIOV adapter with a mac or use promis? Or are you saying I can use > SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). SRIOV is not a requirement. And you can also use a dedicated nic+programmed mac if you are so inclined. > > 2. disable tso,gso,lro with ethtool > > Out of curiosity, wouldnt you only need to disable LRO on the adapter, > since the other two (IIUC) are transmit path and are therefore > influenced by the skb's you generate in vhost? Hmm, makes sense. I'll check this and let you know. > > > 3. add vhost=ethX > > You mean via "ip link" I assume? No, that's a new flag for virtio in qemu: -net nic,model=virtio,vhost=veth0 > Regards, > -Greg > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory Haskins Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 08:41:31 -0400 Message-ID: <4A82B87B.4010208@gmail.com> References: <20090811212743.GA26309@redhat.com> <4A820391.1090404@gmail.com> <20090812071636.GA26847@redhat.com> <4A82ADD5.6040909@gmail.com> <20090812120541.GA29158@redhat.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig174E0DB952567894B5C84450" Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, "akpm@linux-foundation.org >> Andrew Morton" , hpa@zytor.com, Patrick Mullaney To: "Michael S. Tsirkin" Return-path: Received: from mail-gx0-f213.google.com ([209.85.217.213]:37513 "EHLO mail-gx0-f213.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750926AbZHLMlf (ORCPT ); Wed, 12 Aug 2009 08:41:35 -0400 In-Reply-To: <20090812120541.GA29158@redhat.com> Sender: netdev-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig174E0DB952567894B5C84450 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> >>> 1. use a dedicated network interface with SRIOV, program mac to match= >>> that of guest (for testing, you can set promisc mode, but that is >>> bad for performance) >> >> Are you saying SRIOV is a requirement, and I can either program the >> SRIOV adapter with a mac or use promis? Or are you saying I can use >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty).= >=20 > SRIOV is not a requirement. And you can also use a dedicated > nic+programmed mac if you are so inclined. Makes sense. Got it. I was going to add guest-to-guest to the test matrix, but I assume that is not supported with vhost unless you have something like a VEPA enabled bridge? >>> 3. add vhost=3DethX >> You mean via "ip link" I assume? >=20 > No, that's a new flag for virtio in qemu: >=20 > -net nic,model=3Dvirtio,vhost=3Dveth0 Ah, ok. Even better. Thanks! -Greg --------------enig174E0DB952567894B5C84450 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqCuHsACgkQP5K2CMvXmqFbSQCfXtpBd2Q0mrhRjwRuIAyyTt5Y Ng4An3c7WsysfPVp8bJdLHpPutqgws9L =2P8B -----END PGP SIGNATURE----- --------------enig174E0DB952567894B5C84450-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arnd Bergmann Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 14:52:01 +0200 Message-ID: <200908121452.01802.arnd@arndb.de> References: <20090811212743.GA26309@redhat.com> <20090812120541.GA29158@redhat.com> <4A82B87B.4010208@gmail.com> Mime-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Cc: "Michael S. Tsirkin" , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, "Andrew Morton" , hpa@zytor.com, Patrick Mullaney To: Gregory Haskins Return-path: In-Reply-To: <4A82B87B.4010208@gmail.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wednesday 12 August 2009, Gregory Haskins wrote: > >> Are you saying SRIOV is a requirement, and I can either program the > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > SRIOV is not a requirement. And you can also use a dedicated > > nic+programmed mac if you are so inclined. > > Makes sense. Got it. > > I was going to add guest-to-guest to the test matrix, but I assume that > is not supported with vhost unless you have something like a VEPA > enabled bridge? > If I understand it correctly, you can at least connect a veth pair to a bridge, right? Something like veth0 - veth1 - vhost - guest 1 eth0 - br0-| veth2 - veth3 - vhost - guest 2 It's a bit more complicated than it need to be, but should work fine. Arnd <>< -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 16:04:34 +0300 Message-ID: <20090812130433.GB29200@redhat.com> References: <20090811212743.GA26309@redhat.com> <4A820391.1090404@gmail.com> <20090812071636.GA26847@redhat.com> <4A82ADD5.6040909@gmail.com> <20090812120541.GA29158@redhat.com> <4A82B87B.4010208@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, "akpm@linux-foundation.org >> Andrew Morton" , hpa@zytor.com, Patrick Mullaney To: Gregory Haskins Return-path: Received: from mx2.redhat.com ([66.187.237.31]:55151 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752213AbZHLNHL (ORCPT ); Wed, 12 Aug 2009 09:07:11 -0400 Content-Disposition: inline In-Reply-To: <4A82B87B.4010208@gmail.com> Sender: netdev-owner@vger.kernel.org List-ID: On Wed, Aug 12, 2009 at 08:41:31AM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: > >> Michael S. Tsirkin wrote: > > > > >>> > >>> 1. use a dedicated network interface with SRIOV, program mac to match > >>> that of guest (for testing, you can set promisc mode, but that is > >>> bad for performance) > >> > >> Are you saying SRIOV is a requirement, and I can either program the > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > SRIOV is not a requirement. And you can also use a dedicated > > nic+programmed mac if you are so inclined. > > Makes sense. Got it. > > I was going to add guest-to-guest to the test matrix, but I assume that > is not supported with vhost unless you have something like a VEPA > enabled bridge? > > Presumably you mean on the same host? There were also some patches to enable local guest to guest for macvlan, that would be a nice software-only solution. For back to back, I just tried over veth, seems to work fine. > >>> 3. add vhost=ethX > >> You mean via "ip link" I assume? > > > > No, that's a new flag for virtio in qemu: > > > > -net nic,model=virtio,vhost=veth0 > > Ah, ok. Even better. > > Thanks! > -Greg > From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 16:06:12 +0300 Message-ID: <20090812130612.GC29200@redhat.com> References: <20090811212743.GA26309@redhat.com> <20090812120541.GA29158@redhat.com> <4A82B87B.4010208@gmail.com> <200908121452.01802.arnd@arndb.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Gregory Haskins , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: Arnd Bergmann Return-path: Content-Disposition: inline In-Reply-To: <200908121452.01802.arnd@arndb.de> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wed, Aug 12, 2009 at 02:52:01PM +0200, Arnd Bergmann wrote: > On Wednesday 12 August 2009, Gregory Haskins wrote: > > >> Are you saying SRIOV is a requirement, and I can either program the > > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > > > SRIOV is not a requirement. And you can also use a dedicated > > > nic+programmed mac if you are so inclined. > > > > Makes sense. Got it. > > > > I was going to add guest-to-guest to the test matrix, but I assume that > > is not supported with vhost unless you have something like a VEPA > > enabled bridge? > > > > If I understand it correctly, you can at least connect a veth pair > to a bridge, right? Something like > > veth0 - veth1 - vhost - guest 1 > eth0 - br0-| > veth2 - veth3 - vhost - guest 2 > > It's a bit more complicated than it need to be, but should work fine. > > Arnd <>< Heh, you don't need a bridge in this picture: guest 1 - vhost - veth0 - veth1 - vhost guest 2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arnd Bergmann Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 15:40:44 +0200 Message-ID: <200908121540.44928.arnd@arndb.de> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> Mime-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Cc: Gregory Haskins , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20090812130612.GC29200@redhat.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > > If I understand it correctly, you can at least connect a veth pair > > to a bridge, right? Something like > > > > veth0 - veth1 - vhost - guest 1 > > eth0 - br0-| > > veth2 - veth3 - vhost - guest 2 > > > Heh, you don't need a bridge in this picture: > > guest 1 - vhost - veth0 - veth1 - vhost guest 2 Sure, but the setup I described is the one that I would expect to see in practice because it gives you external connectivity. Measuring two guests communicating over a veth pair is interesting for finding the bottlenecks, but of little practical relevance. Arnd <>< -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 16:42:47 +0300 Message-ID: <20090812134247.GA29340@redhat.com> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> <200908121540.44928.arnd@arndb.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Gregory Haskins , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: Arnd Bergmann Return-path: Content-Disposition: inline In-Reply-To: <200908121540.44928.arnd@arndb.de> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wed, Aug 12, 2009 at 03:40:44PM +0200, Arnd Bergmann wrote: > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > > > If I understand it correctly, you can at least connect a veth pair > > > to a bridge, right? Something like > > > > > > veth0 - veth1 - vhost - guest 1 > > > eth0 - br0-| > > > veth2 - veth3 - vhost - guest 2 > > > > > Heh, you don't need a bridge in this picture: > > > > guest 1 - vhost - veth0 - veth1 - vhost guest 2 > > Sure, but the setup I described is the one that I would expect > to see in practice because it gives you external connectivity. > > Measuring two guests communicating over a veth pair is > interesting for finding the bottlenecks, but of little > practical relevance. > > Arnd <>< Oh, hopefully macvlan will soon allow that. -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory Haskins Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 09:51:45 -0400 Message-ID: <4A82C8F1.4030703@gmail.com> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> <200908121540.44928.arnd@arndb.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig87A30C9A577D0E621E7B2C9A" Cc: "Michael S. Tsirkin" , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: Arnd Bergmann Return-path: In-Reply-To: <200908121540.44928.arnd@arndb.de> Sender: linux-kernel-owner@vger.kernel.org List-Id: netdev.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig87A30C9A577D0E621E7B2C9A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Arnd Bergmann wrote: > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>> If I understand it correctly, you can at least connect a veth pair >>> to a bridge, right? Something like >>> >>> veth0 - veth1 - vhost - guest 1=20 >>> eth0 - br0-| >>> veth2 - veth3 - vhost - guest 2 >>> =20 >> Heh, you don't need a bridge in this picture: >> >> guest 1 - vhost - veth0 - veth1 - vhost guest 2 >=20 > Sure, but the setup I described is the one that I would expect > to see in practice because it gives you external connectivity. >=20 > Measuring two guests communicating over a veth pair is > interesting for finding the bottlenecks, but of little > practical relevance. >=20 > Arnd <>< Yeah, this would be the config I would be interested in. Regards, -Greg --------------enig87A30C9A577D0E621E7B2C9A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqCyPEACgkQP5K2CMvXmqHKgwCbBrxhIdqOX31o4APQvc7hWcWt y0oAn1INe0wEK/9n2tSfeBeMCClGjSXU =S5BJ -----END PGP SIGNATURE----- --------------enig87A30C9A577D0E621E7B2C9A-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 17:02:24 +0300 Message-ID: <20090812140224.GA29345@redhat.com> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> <200908121540.44928.arnd@arndb.de> <4A82C8F1.4030703@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Arnd Bergmann , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: Gregory Haskins Return-path: Content-Disposition: inline In-Reply-To: <4A82C8F1.4030703@gmail.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: > Arnd Bergmann wrote: > > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > >>> If I understand it correctly, you can at least connect a veth pair > >>> to a bridge, right? Something like > >>> > >>> veth0 - veth1 - vhost - guest 1 > >>> eth0 - br0-| > >>> veth2 - veth3 - vhost - guest 2 > >>> > >> Heh, you don't need a bridge in this picture: > >> > >> guest 1 - vhost - veth0 - veth1 - vhost guest 2 > > > > Sure, but the setup I described is the one that I would expect > > to see in practice because it gives you external connectivity. > > > > Measuring two guests communicating over a veth pair is > > interesting for finding the bottlenecks, but of little > > practical relevance. > > > > Arnd <>< > > Yeah, this would be the config I would be interested in. Hmm, this wouldn't be the config to use for the benchmark though: there are just too many variables. If you want both guest to guest and guest to host, create 2 nics in the guest. Here's one way to do this: -net nic,model=virtio,vlan=0 -net user,vlan=0 -net nic,vlan=1,model=virtio,vhost=veth0 -redir tcp:8022::22 -net nic,model=virtio,vlan=0 -net user,vlan=0 -net nic,vlan=1,model=virtio,vhost=veth1 -redir tcp:8023::22 In guests, for simplicity, configure eth1 and eth0 to use separate subnets. Long term, I hope macvlan will be extended to support guest to guest. > Regards, > -Greg > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory Haskins Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 12:13:43 -0400 Message-ID: <4A82EA37.3010902@gmail.com> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> <200908121540.44928.arnd@arndb.de> <4A82C8F1.4030703@gmail.com> <20090812140224.GA29345@redhat.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig4E37A2FAF65F7E8AF474EC70" Cc: Arnd Bergmann , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20090812140224.GA29345@redhat.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig4E37A2FAF65F7E8AF474EC70 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: >> Arnd Bergmann wrote: >>> On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>>>> If I understand it correctly, you can at least connect a veth pair >>>>> to a bridge, right? Something like >>>>> >>>>> veth0 - veth1 - vhost - guest 1=20 >>>>> eth0 - br0-| >>>>> veth2 - veth3 - vhost - guest 2 >>>>> =20 >>>> Heh, you don't need a bridge in this picture: >>>> >>>> guest 1 - vhost - veth0 - veth1 - vhost guest 2 >>> Sure, but the setup I described is the one that I would expect >>> to see in practice because it gives you external connectivity. >>> >>> Measuring two guests communicating over a veth pair is >>> interesting for finding the bottlenecks, but of little >>> practical relevance. >>> >>> Arnd <>< >> Yeah, this would be the config I would be interested in. >=20 > Hmm, this wouldn't be the config to use for the benchmark though: there= > are just too many variables. If you want both guest to guest and guest= > to host, create 2 nics in the guest. >=20 > Here's one way to do this: >=20 > -net nic,model=3Dvirtio,vlan=3D0 -net user,vlan=3D0 > -net nic,vlan=3D1,model=3Dvirtio,vhost=3Dveth0 > -redir tcp:8022::22 >=20 > -net nic,model=3Dvirtio,vlan=3D0 -net user,vlan=3D0 > -net nic,vlan=3D1,model=3Dvirtio,vhost=3Dveth1 > -redir tcp:8023::22 >=20 > In guests, for simplicity, configure eth1 and eth0 > to use separate subnets. I can try to do a few variations, but what I am interested is in performance in a real-world L2 configuration. This would generally mean all hosts (virtual or physical) in the same L2 domain. If I get a chance, though, I will try to also wire them up in isolation as another data point. Regards, -Greg --------------enig4E37A2FAF65F7E8AF474EC70 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqC6jcACgkQP5K2CMvXmqG0FQCfVhCkLU4jF4NKuVMP5GrYh+cH NF4AoIqK9SqEu0RYq/LqHoplHQBbErgz =ydLy -----END PGP SIGNATURE----- --------------enig4E37A2FAF65F7E8AF474EC70-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCHv2 0/2] vhost: a kernel-level virtio server Date: Wed, 12 Aug 2009 19:37:37 +0300 Message-ID: <20090812163737.GA29903@redhat.com> References: <20090811212743.GA26309@redhat.com> <200908121452.01802.arnd@arndb.de> <20090812130612.GC29200@redhat.com> <200908121540.44928.arnd@arndb.de> <4A82C8F1.4030703@gmail.com> <20090812140224.GA29345@redhat.com> <4A82EA37.3010902@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Arnd Bergmann , netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@elte.hu, linux-mm@kvack.org, Andrew Morton , hpa@zytor.com, Patrick Mullaney To: Gregory Haskins Return-path: Content-Disposition: inline In-Reply-To: <4A82EA37.3010902@gmail.com> Sender: owner-linux-mm@kvack.org List-Id: netdev.vger.kernel.org On Wed, Aug 12, 2009 at 12:13:43PM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: > >> Arnd Bergmann wrote: > >>> On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > >>>>> If I understand it correctly, you can at least connect a veth pair > >>>>> to a bridge, right? Something like > >>>>> > >>>>> veth0 - veth1 - vhost - guest 1 > >>>>> eth0 - br0-| > >>>>> veth2 - veth3 - vhost - guest 2 > >>>>> > >>>> Heh, you don't need a bridge in this picture: > >>>> > >>>> guest 1 - vhost - veth0 - veth1 - vhost guest 2 > >>> Sure, but the setup I described is the one that I would expect > >>> to see in practice because it gives you external connectivity. > >>> > >>> Measuring two guests communicating over a veth pair is > >>> interesting for finding the bottlenecks, but of little > >>> practical relevance. > >>> > >>> Arnd <>< > >> Yeah, this would be the config I would be interested in. > > > > Hmm, this wouldn't be the config to use for the benchmark though: there > > are just too many variables. If you want both guest to guest and guest > > to host, create 2 nics in the guest. > > > > Here's one way to do this: > > > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > > -net nic,vlan=1,model=virtio,vhost=veth0 > > -redir tcp:8022::22 > > > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > > -net nic,vlan=1,model=virtio,vhost=veth1 > > -redir tcp:8023::22 > > > > In guests, for simplicity, configure eth1 and eth0 > > to use separate subnets. > > I can try to do a few variations, but what I am interested is in > performance in a real-world L2 configuration. This would generally mean > all hosts (virtual or physical) in the same L2 domain. > > If I get a chance, though, I will try to also wire them up in isolation > as another data point. > > Regards, > -Greg > > Or patch macvlan to support guest to guest: http://markmail.org/message/sjy74g57qsvdo2wh That patch needs to be updated to support guest to guest multiast, but it seems functional enough for your purposes. -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org