From: "Cédric Le Goater" <clg@kaod.org> To: kvm-ppc@vger.kernel.org Cc: kvm@vger.kernel.org, "Paul Mackerras" <paulus@samba.org>, "Cédric Le Goater" <clg@kaod.org>, linuxppc-dev@lists.ozlabs.org, "David Gibson" <david@gibson.dropbear.id.au> Subject: [PATCH v4 17/17] KVM: PPC: Book3S HV: XIVE: clear the vCPU interrupt presenters Date: Wed, 20 Mar 2019 09:37:51 +0100 [thread overview] Message-ID: <20190320083751.27001-18-clg@kaod.org> (raw) In-Reply-To: <20190320083751.27001-1-clg@kaod.org> When the VM boots, the CAS negotiation process determines which interrupt mode to use and invokes a machine reset. At that time, the previous KVM interrupt device is 'destroyed' before the chosen one is created. Upon destruction, the vCPU interrupt presenters using the KVM device should be cleared first, the machine will reconnect them later to the new device after it is created. Signed-off-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> --- Changes since v2 : - removed comments on possible race in kvmppc_native_connect_vcpu() for the XIVE KVM device. This is still an issue in the XICS-over-XIVE device. arch/powerpc/kvm/book3s_xics.c | 19 +++++++++++++ arch/powerpc/kvm/book3s_xive.c | 39 +++++++++++++++++++++++++-- arch/powerpc/kvm/book3s_xive_native.c | 12 +++++++++ 3 files changed, 68 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index f27ee57ab46e..81cdabf4295f 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -1342,6 +1342,25 @@ static void kvmppc_xics_free(struct kvm_device *dev) struct kvmppc_xics *xics = dev->private; int i; struct kvm *kvm = xics->kvm; + struct kvm_vcpu *vcpu; + + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + /* + * call kick_all_cpus_sync() to ensure that all CPUs + * have executed any pending interrupts + */ + if (is_kvmppc_hv_enabled(kvm)) + kick_all_cpus_sync(); + + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xics_free_icp(vcpu); + } debugfs_remove(xics->dentry); diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index 480a3fc6b9fd..cf6a4c6c5a28 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -1100,11 +1100,19 @@ void kvmppc_xive_disable_vcpu_interrupts(struct kvm_vcpu *vcpu) void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) { struct kvmppc_xive_vcpu *xc = vcpu->arch.xive_vcpu; - struct kvmppc_xive *xive = xc->xive; + struct kvmppc_xive *xive; int i; + if (!kvmppc_xics_enabled(vcpu)) + return; + + if (!xc) + return; + pr_devel("cleanup_vcpu(cpu=%d)\n", xc->server_num); + xive = xc->xive; + /* Ensure no interrupt is still routed to that VP */ xc->valid = false; kvmppc_xive_disable_vcpu_interrupts(vcpu); @@ -1141,6 +1149,10 @@ void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) } /* Free the VP */ kfree(xc); + + /* Cleanup the vcpu */ + vcpu->arch.irq_type = KVMPPC_IRQ_DEFAULT; + vcpu->arch.xive_vcpu = NULL; } int kvmppc_xive_connect_vcpu(struct kvm_device *dev, @@ -1158,7 +1170,7 @@ int kvmppc_xive_connect_vcpu(struct kvm_device *dev, } if (xive->kvm != vcpu->kvm) return -EPERM; - if (vcpu->arch.irq_type) + if (vcpu->arch.irq_type != KVMPPC_IRQ_DEFAULT) return -EBUSY; if (kvmppc_xive_find_server(vcpu->kvm, cpu)) { pr_devel("Duplicate !\n"); @@ -1828,8 +1840,31 @@ static void kvmppc_xive_free(struct kvm_device *dev) { struct kvmppc_xive *xive = dev->private; struct kvm *kvm = xive->kvm; + struct kvm_vcpu *vcpu; int i; + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + /* + * call kick_all_cpus_sync() to ensure that all CPUs + * have executed any pending interrupts + */ + if (is_kvmppc_hv_enabled(kvm)) + kick_all_cpus_sync(); + + /* + * TODO: There is still a race window with the early + * checks in kvmppc_native_connect_vcpu() + */ + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xive_cleanup_vcpu(vcpu); + } + debugfs_remove(xive->dentry); if (kvm) diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c index 6a502eee6744..96e6b5c50eb3 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -961,8 +961,20 @@ static void kvmppc_xive_native_free(struct kvm_device *dev) { struct kvmppc_xive *xive = dev->private; struct kvm *kvm = xive->kvm; + struct kvm_vcpu *vcpu; int i; + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xive_native_cleanup_vcpu(vcpu); + } + debugfs_remove(xive->dentry); pr_devel("Destroying xive native device\n"); -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: "Cédric Le Goater" <clg@kaod.org> To: kvm-ppc@vger.kernel.org Cc: kvm@vger.kernel.org, "Paul Mackerras" <paulus@samba.org>, "Cédric Le Goater" <clg@kaod.org>, linuxppc-dev@lists.ozlabs.org, "David Gibson" <david@gibson.dropbear.id.au> Subject: [PATCH v4 17/17] KVM: PPC: Book3S HV: XIVE: clear the vCPU interrupt presenters Date: Wed, 20 Mar 2019 08:37:51 +0000 [thread overview] Message-ID: <20190320083751.27001-18-clg@kaod.org> (raw) In-Reply-To: <20190320083751.27001-1-clg@kaod.org> When the VM boots, the CAS negotiation process determines which interrupt mode to use and invokes a machine reset. At that time, the previous KVM interrupt device is 'destroyed' before the chosen one is created. Upon destruction, the vCPU interrupt presenters using the KVM device should be cleared first, the machine will reconnect them later to the new device after it is created. Signed-off-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> --- Changes since v2 : - removed comments on possible race in kvmppc_native_connect_vcpu() for the XIVE KVM device. This is still an issue in the XICS-over-XIVE device. arch/powerpc/kvm/book3s_xics.c | 19 +++++++++++++ arch/powerpc/kvm/book3s_xive.c | 39 +++++++++++++++++++++++++-- arch/powerpc/kvm/book3s_xive_native.c | 12 +++++++++ 3 files changed, 68 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index f27ee57ab46e..81cdabf4295f 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -1342,6 +1342,25 @@ static void kvmppc_xics_free(struct kvm_device *dev) struct kvmppc_xics *xics = dev->private; int i; struct kvm *kvm = xics->kvm; + struct kvm_vcpu *vcpu; + + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + /* + * call kick_all_cpus_sync() to ensure that all CPUs + * have executed any pending interrupts + */ + if (is_kvmppc_hv_enabled(kvm)) + kick_all_cpus_sync(); + + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xics_free_icp(vcpu); + } debugfs_remove(xics->dentry); diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index 480a3fc6b9fd..cf6a4c6c5a28 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -1100,11 +1100,19 @@ void kvmppc_xive_disable_vcpu_interrupts(struct kvm_vcpu *vcpu) void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) { struct kvmppc_xive_vcpu *xc = vcpu->arch.xive_vcpu; - struct kvmppc_xive *xive = xc->xive; + struct kvmppc_xive *xive; int i; + if (!kvmppc_xics_enabled(vcpu)) + return; + + if (!xc) + return; + pr_devel("cleanup_vcpu(cpu=%d)\n", xc->server_num); + xive = xc->xive; + /* Ensure no interrupt is still routed to that VP */ xc->valid = false; kvmppc_xive_disable_vcpu_interrupts(vcpu); @@ -1141,6 +1149,10 @@ void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) } /* Free the VP */ kfree(xc); + + /* Cleanup the vcpu */ + vcpu->arch.irq_type = KVMPPC_IRQ_DEFAULT; + vcpu->arch.xive_vcpu = NULL; } int kvmppc_xive_connect_vcpu(struct kvm_device *dev, @@ -1158,7 +1170,7 @@ int kvmppc_xive_connect_vcpu(struct kvm_device *dev, } if (xive->kvm != vcpu->kvm) return -EPERM; - if (vcpu->arch.irq_type) + if (vcpu->arch.irq_type != KVMPPC_IRQ_DEFAULT) return -EBUSY; if (kvmppc_xive_find_server(vcpu->kvm, cpu)) { pr_devel("Duplicate !\n"); @@ -1828,8 +1840,31 @@ static void kvmppc_xive_free(struct kvm_device *dev) { struct kvmppc_xive *xive = dev->private; struct kvm *kvm = xive->kvm; + struct kvm_vcpu *vcpu; int i; + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + /* + * call kick_all_cpus_sync() to ensure that all CPUs + * have executed any pending interrupts + */ + if (is_kvmppc_hv_enabled(kvm)) + kick_all_cpus_sync(); + + /* + * TODO: There is still a race window with the early + * checks in kvmppc_native_connect_vcpu() + */ + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xive_cleanup_vcpu(vcpu); + } + debugfs_remove(xive->dentry); if (kvm) diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c index 6a502eee6744..96e6b5c50eb3 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -961,8 +961,20 @@ static void kvmppc_xive_native_free(struct kvm_device *dev) { struct kvmppc_xive *xive = dev->private; struct kvm *kvm = xive->kvm; + struct kvm_vcpu *vcpu; int i; + /* + * When destroying the VM, the vCPUs are destroyed first and + * the vCPU list should be empty. If this is not the case, + * then we are simply destroying the device and we should + * clean up the vCPU interrupt presenters first. + */ + if (atomic_read(&kvm->online_vcpus) != 0) { + kvm_for_each_vcpu(i, vcpu, kvm) + kvmppc_xive_native_cleanup_vcpu(vcpu); + } + debugfs_remove(xive->dentry); pr_devel("Destroying xive native device\n"); -- 2.20.1
next prev parent reply other threads:[~2019-03-20 8:37 UTC|newest] Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-20 8:37 [PATCH v4 00/17] KVM: PPC: Book3S HV: add XIVE native exploitation mode Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 01/17] powerpc/xive: add OPAL extensions for the XIVE native exploitation support Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 02/17] KVM: PPC: Book3S HV: add a new KVM device for the XIVE native exploitation mode Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 03/17] KVM: PPC: Book3S HV: XIVE: introduce a new capability KVM_CAP_PPC_IRQ_XIVE Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 04/17] KVM: PPC: Book3S HV: XIVE: add a control to initialize a source Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 05/17] KVM: PPC: Book3S HV: XIVE: add a control to configure " Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 06/17] KVM: PPC: Book3S HV: XIVE: add controls for the EQ configuration Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 23:09 ` David Gibson 2019-03-20 23:09 ` David Gibson 2019-03-21 8:48 ` Cédric Le Goater 2019-03-21 8:48 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 07/17] KVM: PPC: Book3S HV: XIVE: add a global reset control Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 08/17] KVM: PPC: Book3S HV: XIVE: add a control to sync the sources Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 09/17] KVM: PPC: Book3S HV: XIVE: add a control to dirty the XIVE EQ pages Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 10/17] KVM: PPC: Book3S HV: XIVE: add get/set accessors for the VP XIVE state Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-04-09 6:19 ` Paul Mackerras 2019-04-09 6:19 ` Paul Mackerras 2019-04-09 6:19 ` Paul Mackerras 2019-04-09 9:18 ` Cédric Le Goater 2019-04-09 9:18 ` Cédric Le Goater 2019-04-09 9:18 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 11/17] KVM: introduce a 'mmap' method for KVM devices Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 12/17] KVM: PPC: Book3S HV: XIVE: add a TIMA mapping Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 13/17] KVM: PPC: Book3S HV: XIVE: add a mapping for the source ESB pages Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 14/17] KVM: PPC: Book3S HV: XIVE: add passthrough support Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 15/17] KVM: PPC: Book3S HV: XIVE: activate XIVE exploitation mode Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-03-20 8:37 ` [PATCH v4 16/17] KVM: introduce a KVM_DESTROY_DEVICE ioctl Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater 2019-04-09 14:12 ` Cédric Le Goater 2019-04-09 14:12 ` Cédric Le Goater 2019-04-09 14:12 ` Cédric Le Goater 2019-03-20 8:37 ` Cédric Le Goater [this message] 2019-03-20 8:37 ` [PATCH v4 17/17] KVM: PPC: Book3S HV: XIVE: clear the vCPU interrupt presenters Cédric Le Goater 2019-04-09 14:13 ` [RFC PATCH v4.1 16/17] KVM: PPC: Book3S HV: XIVE: introduce a xive_devices array under the VM Cédric Le Goater 2019-04-09 14:13 ` Cédric Le Goater 2019-04-09 14:13 ` [RFC PATCH v4 17/17] KVM: PPC: Book3S HV: XIVE: introduce a 'release' device operation Cédric Le Goater 2019-04-09 14:13 ` Cédric Le Goater 2019-04-15 3:32 ` David Gibson 2019-04-15 3:32 ` David Gibson 2019-04-15 9:25 ` Paul Mackerras 2019-04-15 9:25 ` Paul Mackerras 2019-04-15 13:56 ` Cédric Le Goater 2019-04-15 13:56 ` Cédric Le Goater 2019-04-15 13:48 ` Cédric Le Goater 2019-04-15 13:48 ` Cédric Le Goater 2019-04-17 2:05 ` David Gibson 2019-04-17 2:05 ` David Gibson 2019-04-15 3:26 ` [RFC PATCH v4.1 16/17] KVM: PPC: Book3S HV: XIVE: introduce a xive_devices array under the VM David Gibson 2019-04-15 3:26 ` David Gibson
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190320083751.27001-18-clg@kaod.org \ --to=clg@kaod.org \ --cc=david@gibson.dropbear.id.au \ --cc=kvm-ppc@vger.kernel.org \ --cc=kvm@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=paulus@samba.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.