Re: [Qemu-devel] [PATCH v3 6/7] s390x/kvm: handle AP instruction interception

[Halil Pasic <pasic@linux.vnet.ibm.com>]

On 04/04/2018 10:12 PM, Tony Krowiak wrote:
> On 04/04/2018 09:43 AM, Pierre Morel wrote:
>> On 04/04/2018 15:33, Tony Krowiak wrote:
>>> On 04/04/2018 07:09 AM, Pierre Morel wrote:
>>>> On 02/04/2018 18:36, Tony Krowiak wrote:
>>>>> On 03/26/2018 05:03 AM, Pierre Morel wrote:
>>>>>> On 26/03/2018 10:32, David Hildenbrand wrote:
>>>>>>> On 16.03.2018 00:24, Tony Krowiak wrote:
>>>>>>>> If the CPU model indicates that AP facility is installed on
>>>>>>>> the guest (i.e., -cpu xxxx,ap=on), then the expectation is that
>>>>>>>> the AP bus running in the guest will initialize; however, if the
>>>>>>>> AP instructions are not being interpreted by the firmware, then
>>>>>>>> they will be intercepted and routed back to QEMU for handling.
>>>>>>>> If a handler is not defined to process the intercepted instruciton,
>>>>>>>> t
>>>>>> ...snip...
>>>>>>>
>>>>>>>> +int ap_device_handle_nqap(S390CPU *cpu)
>>>>>>>> +{
>>>>>>>> +    CPUS390XState *env = &cpu->env;
>>>>>>>> +
>>>>>>>> +    if (s390_has_feat(S390_FEAT_AP)) {
>>>>>>>> +        env->regs[1] = 0x10000;
>>>>>>>> +
>>>>>>>> +        return 0;
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    return -EOPNOTSUPP;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +int ap_device_handle_dqap(S390CPU *cpu)
>>>>>>>> +{
>>>>>>>> +    CPUS390XState *env = &cpu->env;
>>>>>>>> +
>>>>>>>> +    if (s390_has_feat(S390_FEAT_AP)) {
>>>>>>>> +        env->regs[1] = 0x10000;
>>>>>>>> +
>>>>>>>> +        return 0;
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    return -EOPNOTSUPP;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +int ap_device_handle_pqap(S390CPU *cpu)
>>>>>>>> +{
>>>>>>>> +    CPUS390XState *env = &cpu->env;
>>>>>>>> +    int fc = 4 & (env->regs[0] >> 24);
>>>>>>>> +
>>>>>>>> +    /*
>>>>>>>> +     * The Query Configuration Information (QCI) function (fc == 4) does not
>>>>>>>> +     * set a response code in reg 1, so check for that along with the
>>>>>>>> +     * AP feature.
>>>>>>>> +     */
>>>>>>>> +    if ((fc != 4) && s390_has_feat(S390_FEAT_AP)) {
>>>>>>>> +        env->regs[1] = 0x10000;
>>>>>>>> +
>>>>>>>> +        return 0;
>>>>>>>> +    }
>>>>>>> This would imply an operation exception in case fc==4, which sounds very
>>>>>>> wrong.
>>>>>>
>>>>>> It depends but I think that the S390_FEAT_AP_QUERY_CONFIG_INFO must be tested
>>>>>> to know what to answer.
>>>>>> If the feature is there, QCI must be answered correctly.
>>>>> This is an interesting proposition which raises several issues that will need to
>>>>> be addressed. The only time the PQAP(QCI) instruction is intercepted is when:
>>>>> * A vfio-ap device is NOT defined for the guest because the vfio_ap device driver
>>>>>   will set ECA.28 and the PQAP(QCI) instruction will be interpreted
>>>>> * STFLE.12 is set for the guest
>>>>>
>>>>> You say that the QCI must be answered correctly, but what is the correct response?
>>>>> If we return the matrix - i.e., APM, ADM and AQM - configured via the mediated
>>>>> matrix device's sysfs attributes files, then if any APQNs are defined in the matrix,
>>>>> we will have to handle *ALL* AP instructions by executing them on behalf of the
>>>>> guest. I suppose we could return an empty matrix in which case the AP bus will come
>>>>> up without any devices on the guest, but what is the expectation of an admin who
>>>>> deliberately configures the mediated matrix device? Should we forego handling interception
>>>>> of AP instructions and consider a guest configuration that turns on S390_FEAT_AP but
>>>>> does not define a vfio-ap device to be erroneous and terminate starting of the guest?
>>>>> Anybody have any thoughts?
>>>>>>
>>>>>>
>>>>>> there are also some error situations to handle in all three functions.
>>>>> To what error situations are you referring?
>>>>
>>>> program exceptions, like access, privilege or specification exceptions.
>>> Are you suggesting that these handlers need to verify that the instruction
>>> specification is correct? For example, the NQAP instruction - NQAP r1,r2 -
>>> expects an even-odd pair of general registers in the r1 and r2 and must
>>> designate an even register; otherwise a specification exception is
>>> recognized. Are you saying that the handler for NQAP must verify this
>>> requirement and inject a specification exception if it is not met? If not,
>>> then can you provide a specific example of what you are talking about?
>>
>> Yes I mean this.
>>
>> But one other thing.
>> Returning code 0x01 seems wrong for me, this is right if no AP card
>> are in the system but I do not thing that it is what we mean.
> I disagree .... for the guest, this is exactly what we mean. In my opinion, assigning
> AP devices to the mediated matrix device is like assigning AP devices to an LPAR.
> If no vfio-ap device is defined for the guest, then it is effectively the same as
> starting a linux host in an LPAR without any AP devices assigned to it.
>>
>> I think we should return code 0x03 stating that the AP card is not in configured state
> 
>>
>>
>> The difference is that we can get out of a not-configured state with a SCLP command but
>> we can not recover from an un-existing APQN.
> Remember, this state will occur only when the AP feature of the CPU model is turned on for
> the guest and no vfio-ap device is defined, so there is no configuration to recover for
> the guest because no AP devices will be configured for the guest.

Spoke with Pierre and we agreed that the response code 01 is what
we need to do.

Regards,
Halil