From: Tony Krowiak <akrowiak@linux.vnet.ibm.com>
To: Pierre Morel <pmorel@linux.vnet.ibm.com>,
Halil Pasic <pasic@linux.vnet.ibm.com>,
linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
kvm@vger.kernel.org
Cc: freude@de.ibm.com, schwidefsky@de.ibm.com,
heiko.carstens@de.ibm.com, borntraeger@de.ibm.com,
cohuck@redhat.com, kwankhede@nvidia.com,
bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com,
alex.williamson@redhat.com, alifm@linux.vnet.ibm.com,
mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com,
thuth@redhat.com, berrange@redhat.com, fiuczy@linux.vnet.ibm.com,
buendgen@de.ibm.com
Subject: Re: [PATCH v3 04/14] KVM: s390: device attribute to set AP interpretive execution
Date: Fri, 16 Mar 2018 12:09:39 -0400 [thread overview]
Message-ID: <c2af69b7-2d62-fbc6-e281-905490b3157e@linux.vnet.ibm.com> (raw)
In-Reply-To: <68e9e3ea-f99a-da88-5e56-21e38b438b4f@linux.vnet.ibm.com>
On 03/16/2018 03:51 AM, Pierre Morel wrote:
> On 16/03/2018 00:39, Tony Krowiak wrote:
>> On 03/15/2018 01:56 PM, Pierre Morel wrote:
>>> On 15/03/2018 18:21, Tony Krowiak wrote:
>>>> On 03/15/2018 11:45 AM, Pierre Morel wrote:
>>>>> On 15/03/2018 16:26, Tony Krowiak wrote:
>>>>>> On 03/15/2018 09:00 AM, Pierre Morel wrote:
>>>>>>> On 14/03/2018 22:57, Halil Pasic wrote:
>>>>>>>>
>>>>>>>> On 03/14/2018 07:25 PM, Tony Krowiak wrote:
>>>>>>>>> The VFIO AP device model exploits interpretive execution of AP
>>>>>>>>> instructions (APIE) to provide guests passthrough access to AP
>>>>>>>>> devices. This patch introduces a new device attribute in the
>>>>>>>>> KVM_S390_VM_CRYPTO device attribute group to set APIE from
>>>>>>>>> the VFIO AP device defined on the guest.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Tony Krowiak <akrowiak@linux.vnet.ibm.com>
>>>>>>>>> ---
>>>>>>>> [..]
>>>>>>>>
>>>>>>>>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>>>>>>>>> index a60c45b..bc46b67 100644
>>>>>>>>> --- a/arch/s390/kvm/kvm-s390.c
>>>>>>>>> +++ b/arch/s390/kvm/kvm-s390.c
>>>>>>>>> @@ -815,6 +815,19 @@ static int kvm_s390_vm_set_crypto(struct
>>>>>>>>> kvm *kvm, struct kvm_device_attr *attr)
>>>>>>>>> sizeof(kvm->arch.crypto.crycb->dea_wrapping_key_mask));
>>>>>>>>> VM_EVENT(kvm, 3, "%s", "DISABLE: DEA keywrapping
>>>>>>>>> support");
>>>>>>>>> break;
>>>>>>>>> + case KVM_S390_VM_CRYPTO_INTERPRET_AP:
>>>>>>>>> + if (attr->addr) {
>>>>>>>>> + if (!test_kvm_cpu_feat(kvm,
>>>>>>>>> KVM_S390_VM_CPU_FEAT_AP))
>>>>>>>> Unlock mutex before returning?
>>>>>>>>
>>>>>>>> Maybe flip conditions (don't allow manipulating apie if feature
>>>>>>>> not there).
>>>>>>>> Clearing the anyways clear apie if feature not there ain't too
>>>>>>>> bad, but
>>>>>>>> rejecting the operation appears nicer to me.
>>>>>>>>
>>>>>>>>> + return -EOPNOTSUPP;
>>>>>>>>> + kvm->arch.crypto.apie = 1;
>>>>>>>>> + VM_EVENT(kvm, 3, "%s",
>>>>>>>>> + "ENABLE: AP interpretive execution");
>>>>>>>>> + } else {
>>>>>>>>> + kvm->arch.crypto.apie = 0;
>>>>>>>>> + VM_EVENT(kvm, 3, "%s",
>>>>>>>>> + "DISABLE: AP interpretive execution");
>>>>>>>>> + }
>>>>>>>>> + break;
>>>>>>>>> default:
>>>>>>>>> mutex_unlock(&kvm->lock);
>>>>>>>>> return -ENXIO;
>>>>>>>> I wonder how the loop after this switch works for
>>>>>>>> KVM_S390_VM_CRYPTO_INTERPRET_AP:
>>>>>>>>
>>>>>>>> kvm_for_each_vcpu(i, vcpu, kvm) {
>>>>>>>> kvm_s390_vcpu_crypto_setup(vcpu);
>>>>>>>> exit_sie(vcpu);
>>>>>>>> }
>>>>>>>>
>>>>>>>> From not doing something like for KVM_S390_VM_CRYPTO_INTERPRET_AP
>>>>>>>>
>>>>>>>> if (kvm->created_vcpus) {
>>>>>>>> mutex_unlock(&kvm->lock);
>>>>>>>> return -EBUSY;
>>>>>>>> and from the aforementioned loop I guess ECA.28 can be changed
>>>>>>>> for a running guest.
>>>>>>>>
>>>>>>>> If there are running vcpus when KVM_S390_VM_CRYPTO_INTERPRET_AP is
>>>>>>>> changed (set) these will be taken out of SIE by exit_sie().
>>>>>>>> Then for the
>>>>>>>> corresponding threads the control probably goes to QEMU (the
>>>>>>>> emulator in
>>>>>>>> the userspace). And it puts that vcpu back into the SIE, and
>>>>>>>> then that
>>>>>>>> cpu starts acting according to the new ECA.28 value. While
>>>>>>>> other vcpus
>>>>>>>> may still work with the old value of ECA.28.
>>>>>>>>
>>>>>>>> I'm not saying what I describe above is necessarily something
>>>>>>>> broken.
>>>>>>>> But I would like to have it explained, why is it OK -- provided
>>>>>>>> I did not
>>>>>>>> make any errors in my reasoning (assumptions included).
>>>>>>>>
>>>>>>>> Can you help me understand this code?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Halil
>>>>>>>>
>>>>>>>> [..]
>>>>>>>>
>>>>>>>
>>>>>>> I have the same concerns as Halil.
>>>>>>>
>>>>>>> We do not need to change the virtulization type
>>>>>>> (hardware/software) on the fly for the current use case.
>>>>>>>
>>>>>>> Couldn't we delay this until we have one and in between only
>>>>>>> make the vCPU hotplug clean?
>>>>>>>
>>>>>>> We only need to let the door open for the day we have such a use
>>>>>>> case.
>>>>>> Are you suggesting this code be removed? If so, then where and
>>>>>> under what conditions would
>>>>>> you suggest setting ECA.28 given you objected to setting it based
>>>>>> on whether the
>>>>>> AP feature is installed?
>>>>>
>>>>> I would only call kvm_s390_vcpu_crypto_setup() from inside
>>>>> kvm_arch_vcpu_init()
>>>>> as it is already.
>>>> It is not called from kvm_arch_vcpu_init(), it is called from
>>>> kvm_arch_vcpu_setup().
>>>
>>> hum, sorry for this.
>>> However, the idea pertains, not to call this function from inside an
>>> ioctl changing crypto parameters, but only during vcpu creation.
>> Unfortunately, the ioctl does not get called until after the vcpus
>> are created (see my comments below)
>
> That is why I think you should not change the ECA field from the
> crypto ioctl but only during the vcpu initialization phase.
By what means do you suggest we do that?
>
>
>>>
>>>
>>>
>>>> Also,
>>>> this loop was already here, I did not put it in. Assuming whomever
>>>> put it there did so
>>>> for a reason, it is not my place to remove it. According to a trace
>>>> I ran, the calls to this
>>>> function occur after the vcpus are created. Consequently, the
>>>> kvm_s390_vcpu_crypto_setup()
>>>> function would not be called without the loop and neither the key
>>>> wrapping support nor the
>>>> ECA_APIE would be configured in the vcpu's SIE descriptor.
>>>>
>>>> If you have a better idea for where/how to set this flag, I'm all
>>>> ears. It would be nice if it could be set before the vcpus are
>>>> created, but I haven't
>>>> found a good candidate. I suspect that the loop was put in to make
>>>> sure that all vcpus
>>>> get updated regardless of whether they are running or not, but I
>>>> don't know what happens
>>>> after a vcpu is kicked out of SIE. I suspect, as Halil surmised,
>>>> that QEMU
>>>> restores the vcpus to SIE. This would seemingly cause the
>>>> kvm_arch_vcpu_setup() to get
>>>> called at which time the ECA_APIE value as well as the key wrapping
>>>> values will get set.
>>>> If somebody has knowledge of the flow here, please feel free to
>>>> pitch in.
>>>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Pierre
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
next prev parent reply other threads:[~2018-03-16 16:19 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-14 18:25 [PATCH v3 00/14] s390: vfio-ap: guest dedicated crypto adapters Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 01/14] KVM: s390: refactor crypto initialization Tony Krowiak
2018-03-15 12:26 ` Pierre Morel
2018-03-15 14:48 ` Tony Krowiak
2018-03-15 14:55 ` Pierre Morel
2018-03-26 8:44 ` Cornelia Huck
2018-03-29 18:57 ` Tony Krowiak
2018-04-03 11:26 ` Cornelia Huck
2018-04-05 10:42 ` Christian Borntraeger
2018-04-05 10:45 ` Christian Borntraeger
2018-04-05 13:16 ` Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 02/14] s390: zcrypt: externalize AP instructions available function Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 03/14] KVM: s390: CPU model support for AP virtualization Tony Krowiak
2018-03-27 10:59 ` Cornelia Huck
2018-03-27 11:22 ` Pierre Morel
2018-03-27 11:30 ` Cornelia Huck
2018-03-14 18:25 ` [PATCH v3 04/14] KVM: s390: device attribute to set AP interpretive execution Tony Krowiak
2018-03-14 21:57 ` Halil Pasic
2018-03-15 13:00 ` Pierre Morel
2018-03-15 15:26 ` Tony Krowiak
2018-03-15 15:45 ` Pierre Morel
2018-03-15 17:21 ` Tony Krowiak
2018-03-15 17:56 ` Pierre Morel
2018-03-15 23:39 ` Tony Krowiak
2018-03-16 7:51 ` Pierre Morel
2018-03-16 16:09 ` Tony Krowiak [this message]
2018-03-20 17:58 ` Tony Krowiak
2018-03-20 22:48 ` Halil Pasic
2018-04-02 18:55 ` Tony Krowiak
2018-03-15 15:23 ` Tony Krowiak
2018-03-15 16:00 ` Pierre Morel
2018-03-15 23:37 ` Tony Krowiak
2018-03-15 16:25 ` Halil Pasic
2018-03-14 18:25 ` [PATCH v3 05/14] s390: vfio-ap: base implementation of VFIO AP device driver Tony Krowiak
2018-03-15 13:25 ` Pierre Morel
2018-03-15 17:25 ` Tony Krowiak
2018-03-27 11:17 ` Cornelia Huck
2018-03-27 14:45 ` Pierre Morel
2018-04-03 9:56 ` Cornelia Huck
2018-04-03 10:57 ` Cornelia Huck
2018-04-03 13:02 ` Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 06/14] s390: vfio-ap: register matrix device with VFIO mdev framework Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 07/14] KVM: s390: interfaces to configure/deconfigure guest's AP matrix Tony Krowiak
2018-04-03 11:07 ` Cornelia Huck
2018-04-03 13:17 ` Tony Krowiak
2018-04-03 13:38 ` Cornelia Huck
2018-03-14 18:25 ` [PATCH v3 08/14] s390: vfio-ap: sysfs interfaces to configure adapters Tony Krowiak
2018-04-03 11:10 ` Cornelia Huck
2018-04-03 13:33 ` Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 09/14] s390: vfio-ap: sysfs interfaces to configure domains Tony Krowiak
2018-04-03 11:17 ` Cornelia Huck
[not found] ` <1860430c-df59-6d58-77f9-b36c51595b4b@linux.vnet.ibm.com>
2018-04-03 15:19 ` Cornelia Huck
2018-04-03 15:42 ` Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 10/14] s390: vfio-ap: sysfs interfaces to configure control domains Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 11/14] s390: vfio-ap: sysfs interface to view matrix mdev matrix Tony Krowiak
2018-03-15 9:42 ` Pierre Morel
2018-03-15 14:52 ` Tony Krowiak
2018-03-15 15:35 ` Pierre Morel
2018-03-27 11:19 ` Cornelia Huck
2018-03-14 18:25 ` [PATCH v3 12/14] KVM: s390: configure the guest's AP devices Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 13/14] s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl Tony Krowiak
2018-03-14 18:25 ` [PATCH v3 14/14] s390: doc: detailed specifications for AP virtualization Tony Krowiak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c2af69b7-2d62-fbc6-e281-905490b3157e@linux.vnet.ibm.com \
--to=akrowiak@linux.vnet.ibm.com \
--cc=alex.williamson@redhat.com \
--cc=alifm@linux.vnet.ibm.com \
--cc=berrange@redhat.com \
--cc=bjsdjshi@linux.vnet.ibm.com \
--cc=borntraeger@de.ibm.com \
--cc=buendgen@de.ibm.com \
--cc=cohuck@redhat.com \
--cc=fiuczy@linux.vnet.ibm.com \
--cc=freude@de.ibm.com \
--cc=heiko.carstens@de.ibm.com \
--cc=jjherne@linux.vnet.ibm.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mjrosato@linux.vnet.ibm.com \
--cc=pasic@linux.vnet.ibm.com \
--cc=pbonzini@redhat.com \
--cc=pmorel@linux.vnet.ibm.com \
--cc=schwidefsky@de.ibm.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).