From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Gibson Subject: Re: [PATCH v2 16/16] KVM: PPC: Book3S HV: XIVE: clear the vCPU interrupt presenters Date: Thu, 14 Mar 2019 13:26:09 +1100 Message-ID: <20190314022609.GJ8211@umbus.fritz.box> References: <20190222112840.25000-1-clg@kaod.org> <20190222112840.25000-17-clg@kaod.org> <20190225041858.GT7668@umbus.fritz.box> <3604baf2-0a39-6990-968f-1222d32f9278@kaod.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="I/5syFLg1Ed7r+1G" Cc: kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, Paul Mackerras , linuxppc-dev@lists.ozlabs.org To: =?iso-8859-1?Q?C=E9dric?= Le Goater Return-path: Content-Disposition: inline In-Reply-To: <3604baf2-0a39-6990-968f-1222d32f9278@kaod.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" List-Id: kvm.vger.kernel.org --I/5syFLg1Ed7r+1G Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 13, 2019 at 09:17:17AM +0100, C=E9dric Le Goater wrote: > On 2/25/19 5:18 AM, David Gibson wrote: > > On Fri, Feb 22, 2019 at 12:28:40PM +0100, C=E9dric Le Goater wrote: > >> When the VM boots, the CAS negotiation process determines which > >> interrupt mode to use and invokes a machine reset. At that time, the > >> previous KVM interrupt device is 'destroyed' before the chosen one is > >> created. Upon destruction, the vCPU interrupt presenters using the KVM > >> device should be cleared first, the machine will reconnect them later > >> to the new device after it is created. > >> > >> When using the KVM device, there is still a race window with the early > >> checks in kvmppc_native_connect_vcpu(). Yet to be fixed. > >> > >> Signed-off-by: C=E9dric Le Goater > >> --- > >> arch/powerpc/kvm/book3s_xics.c | 19 +++++++++++++ > >> arch/powerpc/kvm/book3s_xive.c | 39 +++++++++++++++++++++++++-- > >> arch/powerpc/kvm/book3s_xive_native.c | 16 +++++++++++ > >> 3 files changed, 72 insertions(+), 2 deletions(-) > >> > >> diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_= xics.c > >> index f27ee57ab46e..81cdabf4295f 100644 > >> --- a/arch/powerpc/kvm/book3s_xics.c > >> +++ b/arch/powerpc/kvm/book3s_xics.c > >> @@ -1342,6 +1342,25 @@ static void kvmppc_xics_free(struct kvm_device = *dev) > >> struct kvmppc_xics *xics =3D dev->private; > >> int i; > >> struct kvm *kvm =3D xics->kvm; > >> + struct kvm_vcpu *vcpu; > >> + > >> + /* > >> + * When destroying the VM, the vCPUs are destroyed first and > >> + * the vCPU list should be empty. If this is not the case, > >> + * then we are simply destroying the device and we should > >> + * clean up the vCPU interrupt presenters first. > >> + */ > >> + if (atomic_read(&kvm->online_vcpus) !=3D 0) { > >> + /* > >> + * call kick_all_cpus_sync() to ensure that all CPUs > >> + * have executed any pending interrupts > >> + */ > >> + if (is_kvmppc_hv_enabled(kvm)) > >> + kick_all_cpus_sync(); > >> + > >> + kvm_for_each_vcpu(i, vcpu, kvm) > >> + kvmppc_xics_free_icp(vcpu); > >> + } > >> =20 > >> debugfs_remove(xics->dentry); > >> =20 > >> diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_= xive.c > >> index 7a14512b8944..0a1c11d6881c 100644 > >> --- a/arch/powerpc/kvm/book3s_xive.c > >> +++ b/arch/powerpc/kvm/book3s_xive.c > >> @@ -1105,11 +1105,19 @@ void kvmppc_xive_disable_vcpu_interrupts(struc= t kvm_vcpu *vcpu) > >> void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) > >> { > >> struct kvmppc_xive_vcpu *xc =3D vcpu->arch.xive_vcpu; > >> - struct kvmppc_xive *xive =3D xc->xive; > >> + struct kvmppc_xive *xive; > >> int i; > >> =20 > >> + if (!kvmppc_xics_enabled(vcpu)) > >=20 > > This should be kvmppc_xive_enabled(), no? >=20 > This is the KVM XICS-on-XIVE device and its IRQ type is KVMPPC_IRQ_XICS. > So this is correct :/=20 Ah, right, sorry. > May be we should introduce a KVMPPC_IRQ_XICS_ON_XIVE macro to > clarify. Yeah, maybe. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --I/5syFLg1Ed7r+1G Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEdfRlhq5hpmzETofcbDjKyiDZs5IFAlyJu74ACgkQbDjKyiDZ s5LKFA/+LzfjqE5drr5Ofe5gBMqFYKtgh2EfJOxxudjr+clIpCccJZp3esC7sXLD JUHWUlPd+k+LRc99ekNtv0XiGqix2zovuKzeaMIVVouNEF/WZO/i1waZUG8jfMdN 7KW6JOQjxphIwXEmcbg0A7WXj8du7Uh6bpABjw2F+a6l8I1cxXrNyH1a97Ju2F99 2dUEdzBehptrgnDHRd/srNI9Q4BFLxSKw3xVpGCyHWP1i+KVhZvTDKQJZzdWnpc9 85X22q7WPSLG4Us8oJQ42vmVTygyDA2kIRZMQoLzvlFAfYEjUgrXiO/mroCezKmL 5vIW2SLIA5wPJYkWuCOaM2VT1/2CMeEdkKiN/CkYOrllsGUUTQzfgFLWxVl4yt1R B17sq1fVHqGQGYeNNJPfhnGFNTd0P4FqvlJSHx0eJO8LZxEM74NR16zGqlhqGZMC 96/jtP9Fj3YEYlwflmwh3QniXwqRaH78sJRjgJ88DdPQHqgFfRj8QSGrcAXDLAUH 26A5E58lIQSSHH21KPMqlKVhWWaXCRps5AfQC1jwb/TKXaB6sGsR1NBZr+jQIsWl /p7TR1VHTQc1cTQs1pfp0eR8cLKqZyzCr0S/1dTnQSs2u34jQLiYfDKxfNdYSg82 s6H2B5kJx6v1tRwWNJdVK81zxqpLnU/bEgZ2+XdtZA5fORy45Os= =rBt5 -----END PGP SIGNATURE----- --I/5syFLg1Ed7r+1G-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Gibson Date: Thu, 14 Mar 2019 02:26:09 +0000 Subject: Re: [PATCH v2 16/16] KVM: PPC: Book3S HV: XIVE: clear the vCPU interrupt presenters Message-Id: <20190314022609.GJ8211@umbus.fritz.box> MIME-Version: 1 Content-Type: multipart/mixed; boundary="I/5syFLg1Ed7r+1G" List-Id: References: <20190222112840.25000-1-clg@kaod.org> <20190222112840.25000-17-clg@kaod.org> <20190225041858.GT7668@umbus.fritz.box> <3604baf2-0a39-6990-968f-1222d32f9278@kaod.org> In-Reply-To: <3604baf2-0a39-6990-968f-1222d32f9278@kaod.org> To: =?iso-8859-1?Q?C=E9dric?= Le Goater Cc: kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, Paul Mackerras , linuxppc-dev@lists.ozlabs.org --I/5syFLg1Ed7r+1G Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 13, 2019 at 09:17:17AM +0100, C=E9dric Le Goater wrote: > On 2/25/19 5:18 AM, David Gibson wrote: > > On Fri, Feb 22, 2019 at 12:28:40PM +0100, C=E9dric Le Goater wrote: > >> When the VM boots, the CAS negotiation process determines which > >> interrupt mode to use and invokes a machine reset. At that time, the > >> previous KVM interrupt device is 'destroyed' before the chosen one is > >> created. Upon destruction, the vCPU interrupt presenters using the KVM > >> device should be cleared first, the machine will reconnect them later > >> to the new device after it is created. > >> > >> When using the KVM device, there is still a race window with the early > >> checks in kvmppc_native_connect_vcpu(). Yet to be fixed. > >> > >> Signed-off-by: C=E9dric Le Goater > >> --- > >> arch/powerpc/kvm/book3s_xics.c | 19 +++++++++++++ > >> arch/powerpc/kvm/book3s_xive.c | 39 +++++++++++++++++++++++++-- > >> arch/powerpc/kvm/book3s_xive_native.c | 16 +++++++++++ > >> 3 files changed, 72 insertions(+), 2 deletions(-) > >> > >> diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_= xics.c > >> index f27ee57ab46e..81cdabf4295f 100644 > >> --- a/arch/powerpc/kvm/book3s_xics.c > >> +++ b/arch/powerpc/kvm/book3s_xics.c > >> @@ -1342,6 +1342,25 @@ static void kvmppc_xics_free(struct kvm_device = *dev) > >> struct kvmppc_xics *xics =3D dev->private; > >> int i; > >> struct kvm *kvm =3D xics->kvm; > >> + struct kvm_vcpu *vcpu; > >> + > >> + /* > >> + * When destroying the VM, the vCPUs are destroyed first and > >> + * the vCPU list should be empty. If this is not the case, > >> + * then we are simply destroying the device and we should > >> + * clean up the vCPU interrupt presenters first. > >> + */ > >> + if (atomic_read(&kvm->online_vcpus) !=3D 0) { > >> + /* > >> + * call kick_all_cpus_sync() to ensure that all CPUs > >> + * have executed any pending interrupts > >> + */ > >> + if (is_kvmppc_hv_enabled(kvm)) > >> + kick_all_cpus_sync(); > >> + > >> + kvm_for_each_vcpu(i, vcpu, kvm) > >> + kvmppc_xics_free_icp(vcpu); > >> + } > >> =20 > >> debugfs_remove(xics->dentry); > >> =20 > >> diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_= xive.c > >> index 7a14512b8944..0a1c11d6881c 100644 > >> --- a/arch/powerpc/kvm/book3s_xive.c > >> +++ b/arch/powerpc/kvm/book3s_xive.c > >> @@ -1105,11 +1105,19 @@ void kvmppc_xive_disable_vcpu_interrupts(struc= t kvm_vcpu *vcpu) > >> void kvmppc_xive_cleanup_vcpu(struct kvm_vcpu *vcpu) > >> { > >> struct kvmppc_xive_vcpu *xc =3D vcpu->arch.xive_vcpu; > >> - struct kvmppc_xive *xive =3D xc->xive; > >> + struct kvmppc_xive *xive; > >> int i; > >> =20 > >> + if (!kvmppc_xics_enabled(vcpu)) > >=20 > > This should be kvmppc_xive_enabled(), no? >=20 > This is the KVM XICS-on-XIVE device and its IRQ type is KVMPPC_IRQ_XICS. > So this is correct :/=20 Ah, right, sorry. > May be we should introduce a KVMPPC_IRQ_XICS_ON_XIVE macro to > clarify. Yeah, maybe. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --I/5syFLg1Ed7r+1G Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEdfRlhq5hpmzETofcbDjKyiDZs5IFAlyJu74ACgkQbDjKyiDZ s5LKFA/+LzfjqE5drr5Ofe5gBMqFYKtgh2EfJOxxudjr+clIpCccJZp3esC7sXLD JUHWUlPd+k+LRc99ekNtv0XiGqix2zovuKzeaMIVVouNEF/WZO/i1waZUG8jfMdN 7KW6JOQjxphIwXEmcbg0A7WXj8du7Uh6bpABjw2F+a6l8I1cxXrNyH1a97Ju2F99 2dUEdzBehptrgnDHRd/srNI9Q4BFLxSKw3xVpGCyHWP1i+KVhZvTDKQJZzdWnpc9 85X22q7WPSLG4Us8oJQ42vmVTygyDA2kIRZMQoLzvlFAfYEjUgrXiO/mroCezKmL 5vIW2SLIA5wPJYkWuCOaM2VT1/2CMeEdkKiN/CkYOrllsGUUTQzfgFLWxVl4yt1R B17sq1fVHqGQGYeNNJPfhnGFNTd0P4FqvlJSHx0eJO8LZxEM74NR16zGqlhqGZMC 96/jtP9Fj3YEYlwflmwh3QniXwqRaH78sJRjgJ88DdPQHqgFfRj8QSGrcAXDLAUH 26A5E58lIQSSHH21KPMqlKVhWWaXCRps5AfQC1jwb/TKXaB6sGsR1NBZr+jQIsWl /p7TR1VHTQc1cTQs1pfp0eR8cLKqZyzCr0S/1dTnQSs2u34jQLiYfDKxfNdYSg82 s6H2B5kJx6v1tRwWNJdVK81zxqpLnU/bEgZ2+XdtZA5fORy45Os= =rBt5 -----END PGP SIGNATURE----- --I/5syFLg1Ed7r+1G--