Re: [PATCH v5 5/9] s390x/pci: enable for load/store intepretation

[Pierre Morel <pmorel@linux.ibm.com>]

On 4/20/22 17:12, Matthew Rosato wrote:
> On 4/19/22 3:47 PM, Pierre Morel wrote:
>>
>>
>> On 4/4/22 20:17, Matthew Rosato wrote:
>>> If the appropriate CPU facilty is available as well as the necessary
>>> ZPCI_OP ioctl, then the underlying KVM host will enable load/store
>>> intepretation for any guest device without a SHM bit in the guest
>>> function handle.  For a device that will be using interpretation
>>> support, ensure the guest function handle matches the host function
>>> handle; this value is re-checked every time the guest issues a SET 
>>> PCI FN
>>> to enable the guest device as it is the only opportunity to reflect
>>> function handle changes.
>>>
>>> By default, unless interpret=off is specified, interpretation support 
>>> will
>>> always be assumed and exploited if the necessary ioctl and features are
>>> available on the host kernel.  When these are unavailable, we will 
>>> silently
>>> revert to the interception model; this allows existing guest 
>>> configurations
>>> to work unmodified on hosts with and without zPCI interpretation 
>>> support,
>>> allowing QEMU to choose the best support model available.
>>>
>>> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
>>> ---
>>>   hw/s390x/meson.build            |  1 +
>>>   hw/s390x/s390-pci-bus.c         | 66 ++++++++++++++++++++++++++++++++-
>>>   hw/s390x/s390-pci-inst.c        | 12 ++++++
>>>   hw/s390x/s390-pci-kvm.c         | 21 +++++++++++
>>>   include/hw/s390x/s390-pci-bus.h |  1 +
>>>   include/hw/s390x/s390-pci-kvm.h | 24 ++++++++++++
>>>   target/s390x/kvm/kvm.c          |  7 ++++
>>>   target/s390x/kvm/kvm_s390x.h    |  1 +
>>>   8 files changed, 132 insertions(+), 1 deletion(-)
>>>   create mode 100644 hw/s390x/s390-pci-kvm.c
>>>   create mode 100644 include/hw/s390x/s390-pci-kvm.h
>>>
>>
>> ...snip...
>>
>>>           if (s390_pci_msix_init(pbdev)) {
>>> @@ -1360,6 +1423,7 @@ static Property s390_pci_device_properties[] = {
>>>       DEFINE_PROP_UINT16("uid", S390PCIBusDevice, uid, UID_UNDEFINED),
>>>       DEFINE_PROP_S390_PCI_FID("fid", S390PCIBusDevice, fid),
>>>       DEFINE_PROP_STRING("target", S390PCIBusDevice, target),
>>> +    DEFINE_PROP_BOOL("interpret", S390PCIBusDevice, interp, true),
>>>       DEFINE_PROP_END_OF_LIST(),
>>>   };
>>> diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c
>>> index 6d400d4147..c898c8abe9 100644
>>> --- a/hw/s390x/s390-pci-inst.c
>>> +++ b/hw/s390x/s390-pci-inst.c
>>> @@ -18,6 +18,8 @@
>>>   #include "sysemu/hw_accel.h"
>>>   #include "hw/s390x/s390-pci-inst.h"
>>>   #include "hw/s390x/s390-pci-bus.h"
>>> +#include "hw/s390x/s390-pci-kvm.h"
>>> +#include "hw/s390x/s390-pci-vfio.h"
>>>   #include "hw/s390x/tod.h"
>>>   #ifndef DEBUG_S390PCI_INST
>>> @@ -246,6 +248,16 @@ int clp_service_call(S390CPU *cpu, uint8_t r2, 
>>> uintptr_t ra)
>>>                   goto out;
>>>               }
>>> +            /*
>>> +             * Take this opportunity to make sure we still have an 
>>> accurate
>>> +             * host fh.  It's possible part of the handle changed 
>>> while the
>>> +             * device was disabled to the guest (e.g. vfio hot reset 
>>> for
>>> +             * ISM during plug)
>>> +             */
>>> +            if (pbdev->interp) {
>>> +                /* Take this opportunity to make sure we are sync'd 
>>> with host */
>>> +                s390_pci_get_host_fh(pbdev, &pbdev->fh);

Here we should check the return value and, AFAIU, assume that the device 
disappear if it did return false.

>>> +            }
>>>               pbdev->fh |= FH_MASK_ENABLE;
>>
>> Are we sure here that the PCI device is always enabled?
>> Shouldn't we check?
> 
> I guess you mean the host device?  Interesting thought.
> 
> So, to be clear, the idea on setting FH_MASK_ENABLE here is that we are 
> handling a guest CLP SET PCI FN enable so the guest fh should always 
> have FH_MASK_ENABLE set if we return CLP_RC_OK to the guest.
> 
> But for interpretation, if we find the host function is disabled, I 
> suppose we could return an error on the guest CLP (not sure which error 
> yet); otherwise, if we return the force-enabled handle and CLP_RC_OK as 
> we do here then the guest will just get errors attempting to use it.

hum, in this case can't we have a loop on
clp enable->error->clp disable->clp enable->error...

I think we should return an error if what the guest asked for could not 
be done.


> 
> 
> 

-- 
Pierre Morel
IBM Lab Boeblingen