From: "Michael S. Tsirkin" <mst@redhat.com>
To: Sridhar Samudrala <sridhar.samudrala@intel.com>
Cc: stephen@networkplumber.org, davem@davemloft.net,
netdev@vger.kernel.org,
virtualization@lists.linux-foundation.org,
virtio-dev@lists.oasis-open.org, jesse.brandeburg@intel.com,
alexander.h.duyck@intel.com, kubakici@wp.pl, jasowang@redhat.com,
loseweigh@gmail.com, jiri@resnulli.us
Subject: Re: [PATCH v7 net-next 2/4] net: Introduce generic failover module
Date: Fri, 20 Apr 2018 06:34:42 +0300 [thread overview]
Message-ID: <20180420054853-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <1524188524-28411-3-git-send-email-sridhar.samudrala@intel.com>
On Thu, Apr 19, 2018 at 06:42:02PM -0700, Sridhar Samudrala wrote:
> +static struct net_device *failover_get_bymac(u8 *mac, struct failover_ops **ops)
> +{
> + struct net_device *failover_dev;
> + struct failover *failover;
> +
> + spin_lock(&failover_lock);
> + list_for_each_entry(failover, &failover_list, list) {
> + failover_dev = rtnl_dereference(failover->failover_dev);
> + if (ether_addr_equal(failover_dev->perm_addr, mac)) {
> + *ops = rtnl_dereference(failover->ops);
> + spin_unlock(&failover_lock);
> + return failover_dev;
> + }
> + }
> + spin_unlock(&failover_lock);
> + return NULL;
> +}
So it bothers me that this ties to just any device at all.
I thought hard about it, and I see a nice solution.
Here goes:
QEMU has a pci-bridge-seat device. It is used currently to
group e.g. input devices for multiseat support.
Now if you squint at it hard enough, you can see we are also
in a grouping problem. So how about the following:
1. allocate a virtio pci bridge device failover grouping id (reserve through virtio TC).
2. only group two devices in a failover configuration if both
are under the same bridge with this id (in addition to a mac check).
In particular a bridged configuration makes it easier to
make sure the standby is enumerated before the primary.
In fact we could fail init of failover if we see
appear standby *after* primary.
And this allows many devices with the same ethernet address
without any issues - just under separate bridges.
Further if we ever want to enumerate in parallel this can
be supported by adding a driver for the bridge.
In fact, I see how down the road such a device could
be the beginning of the more ambitious plan to
expose a powerful switchdev interface for
more advanced
So far so good, but I see a couple of issues:
- it is PCI specific
Not a big deal: we limit ourselves to PCI anyway ATM.
- does not work across PCI domains - which are helpful for NUMA
(e.g. we want to be able to move to primary
which is on a different numa
node without losing connectivity).
Idea: add a "group ID" register to each of these pci bridge
devices (e.g. in device specific config space).
Match two bridges if they have the same group ID.
- all these extra bridges slow enumeration down somewhat
Idea: as a fallback if no bridge is found,
just assume all devices match, which will result
in falling back on the "match by mac" logic like in
this patchset. Will be fine for simple setups.
Thoughts?
--
MST
WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com>
To: Sridhar Samudrala <sridhar.samudrala@intel.com>
Cc: stephen@networkplumber.org, davem@davemloft.net,
netdev@vger.kernel.org,
virtualization@lists.linux-foundation.org,
virtio-dev@lists.oasis-open.org, jesse.brandeburg@intel.com,
alexander.h.duyck@intel.com, kubakici@wp.pl, jasowang@redhat.com,
loseweigh@gmail.com, jiri@resnulli.us
Subject: [virtio-dev] Re: [PATCH v7 net-next 2/4] net: Introduce generic failover module
Date: Fri, 20 Apr 2018 06:34:42 +0300 [thread overview]
Message-ID: <20180420054853-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <1524188524-28411-3-git-send-email-sridhar.samudrala@intel.com>
On Thu, Apr 19, 2018 at 06:42:02PM -0700, Sridhar Samudrala wrote:
> +static struct net_device *failover_get_bymac(u8 *mac, struct failover_ops **ops)
> +{
> + struct net_device *failover_dev;
> + struct failover *failover;
> +
> + spin_lock(&failover_lock);
> + list_for_each_entry(failover, &failover_list, list) {
> + failover_dev = rtnl_dereference(failover->failover_dev);
> + if (ether_addr_equal(failover_dev->perm_addr, mac)) {
> + *ops = rtnl_dereference(failover->ops);
> + spin_unlock(&failover_lock);
> + return failover_dev;
> + }
> + }
> + spin_unlock(&failover_lock);
> + return NULL;
> +}
So it bothers me that this ties to just any device at all.
I thought hard about it, and I see a nice solution.
Here goes:
QEMU has a pci-bridge-seat device. It is used currently to
group e.g. input devices for multiseat support.
Now if you squint at it hard enough, you can see we are also
in a grouping problem. So how about the following:
1. allocate a virtio pci bridge device failover grouping id (reserve through virtio TC).
2. only group two devices in a failover configuration if both
are under the same bridge with this id (in addition to a mac check).
In particular a bridged configuration makes it easier to
make sure the standby is enumerated before the primary.
In fact we could fail init of failover if we see
appear standby *after* primary.
And this allows many devices with the same ethernet address
without any issues - just under separate bridges.
Further if we ever want to enumerate in parallel this can
be supported by adding a driver for the bridge.
In fact, I see how down the road such a device could
be the beginning of the more ambitious plan to
expose a powerful switchdev interface for
more advanced
So far so good, but I see a couple of issues:
- it is PCI specific
Not a big deal: we limit ourselves to PCI anyway ATM.
- does not work across PCI domains - which are helpful for NUMA
(e.g. we want to be able to move to primary
which is on a different numa
node without losing connectivity).
Idea: add a "group ID" register to each of these pci bridge
devices (e.g. in device specific config space).
Match two bridges if they have the same group ID.
- all these extra bridges slow enumeration down somewhat
Idea: as a fallback if no bridge is found,
just assume all devices match, which will result
in falling back on the "match by mac" logic like in
this patchset. Will be fine for simple setups.
Thoughts?
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
next prev parent reply other threads:[~2018-04-20 3:34 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-20 1:42 [PATCH net-next v7 0/4] Enable virtio_net to act as a standby for a passthru device Sridhar Samudrala
2018-04-20 1:42 ` [virtio-dev] " Sridhar Samudrala
2018-04-20 1:42 ` [PATCH v7 net-next 1/4] virtio_net: Introduce VIRTIO_NET_F_STANDBY feature bit Sridhar Samudrala
2018-04-20 1:42 ` [virtio-dev] " Sridhar Samudrala
2018-04-20 1:42 ` Sridhar Samudrala
2018-04-20 1:42 ` [PATCH v7 net-next 2/4] net: Introduce generic failover module Sridhar Samudrala
2018-04-20 1:42 ` Sridhar Samudrala
2018-04-20 1:42 ` [virtio-dev] " Sridhar Samudrala
2018-04-20 2:44 ` Michael S. Tsirkin
2018-04-20 2:44 ` Michael S. Tsirkin
2018-04-20 2:44 ` [virtio-dev] " Michael S. Tsirkin
2018-04-20 15:21 ` Samudrala, Sridhar
2018-04-20 15:21 ` [virtio-dev] " Samudrala, Sridhar
2018-04-20 15:34 ` Michael S. Tsirkin
2018-04-20 15:34 ` Michael S. Tsirkin
2018-04-20 15:34 ` [virtio-dev] " Michael S. Tsirkin
2018-04-20 15:56 ` Alexander Duyck
2018-04-20 15:56 ` Alexander Duyck
2018-04-20 16:03 ` Michael S. Tsirkin
2018-04-20 16:03 ` Michael S. Tsirkin
2018-04-20 16:03 ` Michael S. Tsirkin
2018-04-20 15:56 ` Alexander Duyck
2018-04-20 15:21 ` Samudrala, Sridhar
2018-04-20 3:34 ` Michael S. Tsirkin [this message]
2018-04-20 3:34 ` [virtio-dev] " Michael S. Tsirkin
2018-04-20 3:34 ` Michael S. Tsirkin
2018-04-22 17:06 ` Michael S. Tsirkin
2018-04-22 17:06 ` Michael S. Tsirkin
2018-04-22 17:06 ` [virtio-dev] " Michael S. Tsirkin
2018-04-23 17:21 ` Samudrala, Sridhar
2018-04-23 17:21 ` [virtio-dev] " Samudrala, Sridhar
2018-04-22 18:29 ` kbuild test robot
2018-04-22 18:29 ` kbuild test robot
2018-04-20 1:42 ` [PATCH v7 net-next 3/4] virtio_net: Extend virtio to use VF datapath when available Sridhar Samudrala
2018-04-20 1:42 ` [virtio-dev] " Sridhar Samudrala
2018-04-20 2:46 ` Michael S. Tsirkin
2018-04-20 2:46 ` [virtio-dev] " Michael S. Tsirkin
2018-04-20 2:46 ` Michael S. Tsirkin
2018-04-22 15:41 ` kbuild test robot
2018-04-22 15:41 ` kbuild test robot
2018-04-22 15:41 ` kbuild test robot
2018-04-22 15:41 ` kbuild test robot
2018-04-20 1:42 ` Sridhar Samudrala
2018-04-20 1:42 ` [PATCH v7 net-next 4/4] netvsc: refactor notifier/event handling code to use the failover framework Sridhar Samudrala
2018-04-20 1:42 ` Sridhar Samudrala
2018-04-20 1:42 ` [virtio-dev] " Sridhar Samudrala
2018-04-20 15:28 ` Stephen Hemminger
2018-04-20 15:43 ` Michael S. Tsirkin
2018-04-20 15:43 ` Michael S. Tsirkin
2018-04-20 15:43 ` [virtio-dev] " Michael S. Tsirkin
2018-04-20 15:47 ` David Miller
2018-04-20 15:47 ` David Miller
2018-04-20 15:46 ` David Miller
2018-04-20 15:46 ` David Miller
2018-04-20 15:46 ` Samudrala, Sridhar
2018-04-20 15:46 ` Samudrala, Sridhar
2018-04-20 15:46 ` [virtio-dev] " Samudrala, Sridhar
2018-04-20 16:00 ` Jiri Pirko
2018-04-20 16:00 ` Jiri Pirko
2018-04-23 17:04 ` Stephen Hemminger
2018-04-23 17:24 ` Michael S. Tsirkin
2018-04-23 17:24 ` Michael S. Tsirkin
2018-04-23 17:24 ` [virtio-dev] " Michael S. Tsirkin
2018-04-23 17:44 ` Stephen Hemminger
2018-04-23 17:56 ` Michael S. Tsirkin
2018-04-23 17:56 ` Michael S. Tsirkin
2018-04-23 17:56 ` [virtio-dev] " Michael S. Tsirkin
2018-04-23 19:44 ` Siwei Liu
2018-04-23 19:44 ` Siwei Liu
2018-04-23 19:44 ` [virtio-dev] " Siwei Liu
2018-04-23 20:06 ` Michael S. Tsirkin
2018-04-23 20:06 ` Michael S. Tsirkin
2018-04-23 20:06 ` [virtio-dev] " Michael S. Tsirkin
2018-04-24 1:28 ` Stephen Hemminger
2018-04-24 1:28 ` Stephen Hemminger
2018-04-25 21:38 ` Siwei Liu
2018-04-25 21:38 ` [virtio-dev] " Siwei Liu
2018-04-25 22:22 ` Michael S. Tsirkin
2018-04-25 22:22 ` Michael S. Tsirkin
2018-04-25 22:22 ` [virtio-dev] " Michael S. Tsirkin
2018-04-25 22:57 ` Siwei Liu
2018-04-25 22:57 ` [virtio-dev] " Siwei Liu
2018-04-26 0:18 ` Stephen Hemminger
2018-04-26 0:18 ` Stephen Hemminger
2018-04-26 2:43 ` Michael S. Tsirkin
2018-04-26 2:43 ` Michael S. Tsirkin
2018-04-26 2:43 ` [virtio-dev] " Michael S. Tsirkin
2018-04-26 2:28 ` Michael S. Tsirkin
2018-04-26 2:28 ` Michael S. Tsirkin
2018-04-26 2:28 ` [virtio-dev] " Michael S. Tsirkin
2018-04-26 22:14 ` Siwei Liu
2018-04-26 22:14 ` [virtio-dev] " Siwei Liu
2018-04-26 23:42 ` Michael S. Tsirkin
2018-04-26 23:42 ` [virtio-dev] " Michael S. Tsirkin
2018-04-28 0:43 ` Siwei Liu
2018-04-28 0:43 ` Siwei Liu
2018-04-28 0:43 ` [virtio-dev] " Siwei Liu
2018-04-26 23:42 ` Michael S. Tsirkin
2018-04-26 22:14 ` Siwei Liu
2018-04-25 22:57 ` Siwei Liu
2018-04-24 1:25 ` Stephen Hemminger
2018-04-24 1:25 ` Stephen Hemminger
2018-04-24 1:42 ` Michael S. Tsirkin
2018-04-24 1:42 ` Michael S. Tsirkin
2018-04-24 1:42 ` [virtio-dev] " Michael S. Tsirkin
2018-04-24 5:07 ` Stephen Hemminger
2018-04-24 5:07 ` Stephen Hemminger
2018-04-23 17:44 ` Stephen Hemminger
2018-04-23 17:25 ` Jiri Pirko
2018-04-23 17:25 ` Jiri Pirko
2018-04-23 17:04 ` Stephen Hemminger
2018-04-20 15:28 ` Stephen Hemminger
2018-04-22 15:41 ` kbuild test robot
2018-04-22 15:41 ` kbuild test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180420054853-mutt-send-email-mst@kernel.org \
--to=mst@redhat.com \
--cc=alexander.h.duyck@intel.com \
--cc=davem@davemloft.net \
--cc=jasowang@redhat.com \
--cc=jesse.brandeburg@intel.com \
--cc=jiri@resnulli.us \
--cc=kubakici@wp.pl \
--cc=loseweigh@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=sridhar.samudrala@intel.com \
--cc=stephen@networkplumber.org \
--cc=virtio-dev@lists.oasis-open.org \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.