From: Tony Krowiak <akrowiak@linux.ibm.com>
To: Halil Pasic <pasic@linux.ibm.com>
Cc: Cornelia Huck <cohuck@redhat.com>,
linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
kvm@vger.kernel.org, stable@vger.kernel.org,
borntraeger@de.ibm.com, kwankhede@nvidia.com,
pbonzini@redhat.com, alex.williamson@redhat.com,
pasic@linux.vnet.ibm.com
Subject: Re: [PATCH 1/1] s390/vfio-ap: fix circular lockdep when setting/clearing crypto masks
Date: Thu, 11 Feb 2021 09:21:26 -0500 [thread overview]
Message-ID: <8c461602-8c2c-4dd9-1d2b-5e424fc701f8@linux.ibm.com> (raw)
In-Reply-To: <20210210234606.1d0dbdec.pasic@linux.ibm.com>
On 2/10/21 5:46 PM, Halil Pasic wrote:
> On Wed, 10 Feb 2021 17:05:48 -0500
> Tony Krowiak <akrowiak@linux.ibm.com> wrote:
>
>> On 2/10/21 10:32 AM, Halil Pasic wrote:
>>> On Wed, 10 Feb 2021 16:24:29 +0100
>>> Halil Pasic <pasic@linux.ibm.com> wrote:
>>>
>>>>> Maybe you could
>>>>> - grab a reference to kvm while holding the lock
>>>>> - call the mask handling functions with that kvm reference
>>>>> - lock again, drop the reference, and do the rest of the processing?
>>>> I agree, matrix_mdev->kvm can go NULL any time and we are risking
>>>> a null pointer dereference here.
>>>>
>>>> Another idea would be to do
>>>>
>>>>
>>>> static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)
>>>> {
>>>> struct kvm *kvm;
>>>>
>>>> mutex_lock(&matrix_dev->lock);
>>>> if (matrix_mdev->kvm) {
>>>> kvm = matrix_mdev->kvm;
>>>> matrix_mdev->kvm = NULL;
>>>> mutex_unlock(&matrix_dev->lock);
>>>> kvm_arch_crypto_clear_masks(kvm);
>>>> mutex_lock(&matrix_dev->lock);
>>>> matrix_mdev->kvm->arch.crypto.pqap_hook = NULL;
>>> s/matrix_mdev->kvm/kvm
>>>> vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
>>>> kvm_put_kvm(kvm);
>>>> }
>>>> mutex_unlock(&matrix_dev->lock);
>>>> }
>>>>
>>>> That way only one unset would actually do the unset and cleanup
>>>> and every other invocation would bail out with only checking
>>>> matrix_mdev->kvm.
>>> But the problem with that is that we enable the the assign/unassign
>>> prematurely, which could interfere wit reset_queues(). Forget about
>>> it.
>> Not sure what you mean by this.
>>
>>
> I mean because above I first do
> (1) matrix_mdev->kvm = NULL;
> and then do
> (2) vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
> another thread could do
> static ssize_t unassign_adapter_store(struct device *dev,
> struct device_attribute *attr,
> const char *buf, size_t count)
> {
> int ret;
> unsigned long apid;
> struct mdev_device *mdev = mdev_from_dev(dev);
> struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
>
> /* If the guest is running, disallow un-assignment of adapter */
> if (matrix_mdev->kvm)
> return -EBUSY;
> ...
> }
> between (1) and (2), and we would not bail out with -EBUSY because !!kvm
> because of (1). That means we would change matrix_mdev->matrix and we
> would not reset the queues that correspond to the apid that was just
> removed, because by the time we do the reset_queues, the queues are
> not in the matrix_mdev->matrix any more.
>
> Does that make sense?
Yes, it makes sense. I guess I didn't look closely at your
suggestion when I said it was exactly what I implemented
after agreeing with Connie. I had a slight difference in
my implementation:
static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)
{
struct kvm *kvm;
mutex_lock(&matrix_dev->lock);
if (matrix_mdev->kvm) {
kvm = matrix_mdev->kvm;
mutex_unlock(&matrix_dev->lock);
kvm_arch_crypto_clear_masks(kvm);
mutex_lock(&matrix_dev->lock);
kvm->arch.crypto.pqap_hook = NULL;
vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
matrix_mdev->kvm = NULL;
kvm_put_kvm(kvm);
}
mutex_unlock(&matrix_dev->lock);
}
In your scenario, the unassignment would fail with -EBUSY because
the matrix_mdev->kvm pointer would not have yet been
cleared. The other problem with your implementation is that
IRQ resources would not get cleared after the reset because
the matrix_mdev->kvm pointer would be NULL at that time.
next prev parent reply other threads:[~2021-02-11 14:30 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-09 19:48 [PATCH 0/1] fix circular lockdep when staring SE guest Tony Krowiak
2021-02-09 19:48 ` [PATCH 1/1] s390/vfio-ap: fix circular lockdep when setting/clearing crypto masks Tony Krowiak
2021-02-10 10:53 ` Cornelia Huck
2021-02-10 15:24 ` Halil Pasic
2021-02-10 15:32 ` Halil Pasic
2021-02-10 22:05 ` Tony Krowiak
2021-02-10 22:46 ` Halil Pasic
2021-02-11 14:21 ` Tony Krowiak [this message]
2021-02-11 16:47 ` Halil Pasic
2021-02-11 19:18 ` Tony Krowiak
2021-02-10 22:03 ` Tony Krowiak
2021-02-10 20:34 ` Tony Krowiak
2021-02-11 12:23 ` Cornelia Huck
2021-02-11 14:38 ` Tony Krowiak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8c461602-8c2c-4dd9-1d2b-5e424fc701f8@linux.ibm.com \
--to=akrowiak@linux.ibm.com \
--cc=alex.williamson@redhat.com \
--cc=borntraeger@de.ibm.com \
--cc=cohuck@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=pasic@linux.ibm.com \
--cc=pasic@linux.vnet.ibm.com \
--cc=pbonzini@redhat.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).