Re: [PATCH v5 02/11] pci: add option for net failover

[Alex Williamson <alex.williamson@redhat.com>]

On Wed, 23 Oct 2019 21:30:35 +0200
Jens Freimann <jfreimann@redhat.com> wrote:

> On Wed, Oct 23, 2019 at 12:06:48PM -0600, Alex Williamson wrote:
> >On Wed, 23 Oct 2019 10:27:02 +0200
> >Jens Freimann <jfreimann@redhat.com> wrote:
> >  
> >> This patch adds a net_failover_pair_id property to PCIDev which is
> >> used to link the primary device in a failover pair (the PCI dev) to
> >> a standby (a virtio-net-pci) device.
> >>
> >> It only supports ethernet devices. Also currently it only supports
> >> PCIe devices. QEMU will exit with an error message otherwise.
> >>
> >> Signed-off-by: Jens Freimann <jfreimann@redhat.com>
> >> ---
> >>  hw/pci/pci.c         | 17 +++++++++++++++++
> >>  include/hw/pci/pci.h |  3 +++
> >>  2 files changed, 20 insertions(+)
> >>
> >> diff --git a/hw/pci/pci.c b/hw/pci/pci.c
> >> index aa05c2b9b2..fa9b5219f8 100644
> >> --- a/hw/pci/pci.c
> >> +++ b/hw/pci/pci.c
> >> @@ -75,6 +75,8 @@ static Property pci_props[] = {
> >>                      QEMU_PCIE_LNKSTA_DLLLA_BITNR, true),
> >>      DEFINE_PROP_BIT("x-pcie-extcap-init", PCIDevice, cap_present,
> >>                      QEMU_PCIE_EXTCAP_INIT_BITNR, true),
> >> +    DEFINE_PROP_STRING("net_failover_pair_id", PCIDevice,
> >> +            net_failover_pair_id),  
> >
> >Nit, white space appears broken here.  
> 
> I'll fix it.
> 
> >>      DEFINE_PROP_END_OF_LIST()
> >>  };
> >>
> >> @@ -2077,6 +2079,7 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
> >>      ObjectClass *klass = OBJECT_CLASS(pc);
> >>      Error *local_err = NULL;
> >>      bool is_default_rom;
> >> +    uint16_t class_id;
> >>
> >>      /* initialize cap_present for pci_is_express() and pci_config_size(),
> >>       * Note that hybrid PCIs are not set automatically and need to manage
> >> @@ -2101,6 +2104,20 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
> >>          }
> >>      }
> >>
> >> +    if (pci_dev->net_failover_pair_id) {
> >> +        if (!pci_is_express(pci_dev)) {
> >> +            error_setg(errp, "failover device is not a PCIExpress device");
> >> +            error_propagate(errp, local_err);
> >> +            return;
> >> +        }  
> >
> >Did we decide we don't need to test that the device is also in a
> >hotpluggable slot?    
> 
> Hmm, my reply to you was never sent. I thought the check for
> qdev_device_add() was sufficient but you said that works only
> after qdev_hotplug is set (after machine creation). I modified
> the check to this:
> 
>     hide = should_hide_device(opts);                                                                      
>                                                                                                           
>     if ((hide || qdev_hotplug) && bus && !qbus_is_hotpluggable(bus)) {                                    
>         error_setg(errp, QERR_BUS_NO_HOTPLUG, bus->name);                                                 
>         return NULL;                                                                                      
>     }                                                                                                     
>                                                                                                           
>     if (hide) {                                                                                           
>         return NULL;                                                                                      
>     }
> 
> This will make qemu fail when we have the device in the initial
> configuration or when we hotplug it to a bus that doesn't support it.
> I tested both with a device on pcie.0. Am I missing something? 

Nope, sorry, I was expecting the check here and didn't see that you
perform it elsewhere.  Seems good enough for me.
 
> >Are there also multi-function considerations that
> >should be prevented or documented?  For example, if a user tries to
> >configure both the primary and failover NICs in the same slot, I assume
> >bad things will happen.  
> 
>   I would have expected that this is already checked in pci code, but
> it is not. I tried it and when I put both devices into the same slot
> they are both unplugged from the guest during boot but nothing else
> happens. I don't know what triggers that unplug of the devices.
> 
> I'm not aware of any other problems regarding multi-function, which
> doesn't mean there aren't any. 

Hmm, was the hidden device at function #0?  The guest won't find any
functions if function #0 isn't present, but I don't know what would
trigger the hotplug.  The angle I'm thinking is that we only have slot
level granularity for hotplug, so any sort of automatic hotplug of a
slot should probably think about bystander devices within the slot.
Thanks,

Alex

> >> +        class_id = pci_get_word(pci_dev->config + PCI_CLASS_DEVICE);
> >> +        if (class_id != PCI_CLASS_NETWORK_ETHERNET) {
> >> +            error_setg(errp, "failover device is not an Ethernet device");
> >> +            error_propagate(errp, local_err);
> >> +            return;
> >> +        }
> >> +    }  
> >
> >Looks like cleanup is missing both both error cases, the option rom
> >error path below this does a pci_qdev_unrealize() before returning so
> >I'd assume we want the same here.  Thanks,  
> 
> Thanks, I'll fix this too.
> 
> regards,
> Jens