public inbox for netdev@vger.kernel.org 
 help / color / mirror / Atom feed

From: Matthew Monaco <Matthew.Monaco@Colorado•EDU>
To: netdev@vger•kernel.org
Subject: sr-iov and bridges (mlx4)
Date: Sat, 19 Sep 2015 17:58:23 -0600	[thread overview]
Message-ID: <55FDF69F.6000105@colorado.edu> (raw)

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

Hello. I have a Mellanox ConnectX-3 Pro EN (MCX314A-BCCT). I'm only using a
single port so it must provide IP for my host as well as connectivity for VMs.
SR-IOV VFs are working great, my KVM VMs have Ethernet and RDMA.

However, I also want to support virtio VMs. Assuming eth0 is the first port on
my mlx nic, I've tried placing VMs on a bridge with the primary physical
interface, and giving an IP for management to a VF:

br0
|--- eth0
|--- VM
|--- VM
vf0 (IP)
vf1 -> VM
vf2 -> VM
vf3 -> VM

I've tried placing VMs on a bridge with one of the VFs and using the primary
iface for IP.

eth0 (IP)
br0
|--- vf0
|--- VM
|--- VM
vf1 -> VM
vf2 -> VM
vf3 -> VM

And I've also tried using a veth pair to really spread things out:

br0 (IP)
|--- eth0
|--- veth-a
br1       |
|--- veth-b
|--- VM
|--- VM
vf1 -> VM
vf2 -> VM
vf3 -> VM

In all cases, VMs with SR-IOV work fine, IP on the host works fine, outbound
DHCP from the virtio VMs work fine, but inbound frames are not making it back to
the VM.

Is there a know limitation of mixing SR-IOV and bridges in general? Does the
SR-IOV switch specific to the mlx4 hardware not work well with linux bridges? ...?

Thanks!
Matt


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

next             reply	other threads:[~2015-09-19 23:58 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-19 23:58 Matthew Monaco [this message]
2015-09-20 11:01 ` sr-iov and bridges (mlx4) Or Gerlitz
2015-09-20 16:21   ` Matthew Monaco
2015-09-20 18:18     ` Nieścierowicz Adam
2015-09-20 18:45       ` Matthew Monaco

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=55FDF69F.6000105@colorado.edu \
    --to=matthew.monaco@colorado$(echo .)edu \
    --cc=netdev@vger$(echo .)kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox