Re: [PATCH 2/4] KVM: arm64: Handle PSCI resets before userspace touches vCPU state

From: Marc Zyngier <maz@kernel.org>
To: Oliver Upton <oupton@google.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
	Peter Shier <pshier@google.com>,
	Ricardo Koller <ricarkol@google.com>,
	Jing Zhang <jingzhangos@google.com>,
	Raghavendra Rao Anata <rananta@google.com>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>
Subject: Re: [PATCH 2/4] KVM: arm64: Handle PSCI resets before userspace touches vCPU state
Date: Wed, 18 Aug 2021 11:38:02 +0100	[thread overview]
Message-ID: <87r1errqb9.wl-maz@kernel.org> (raw)
In-Reply-To: <20210818085047.1005285-3-oupton@google.com>

On Wed, 18 Aug 2021 09:50:45 +0100,
Oliver Upton <oupton@google.com> wrote:
> 
> The CPU_ON PSCI call takes a payload that KVM uses to configure a
> destination vCPU to run. This payload is non-architectural state and not
> exposed through any existing UAPI. Effectively, we have a race between
> CPU_ON and userspace saving/restoring a guest: if the target vCPU isn't
> ran again before the VMM saves its state, the requested PC and context
> ID are lost. When restored, the target vCPU will be runnable and start
> executing at its old PC.
> 
> We can avoid this race by making sure the reset payload is serviced
> before userspace can access a vCPU's state. This is, of course, a hairy
> ugly hack. A benefit of such a hack, though, is that we've managed to
> massage the reset state into the architected state, thereby making it
> migratable without forcing userspace to play our game with a UAPI
> addition.

I don't think it is that bad. In a way, it is similar to the "resync
pending exception state" dance that we do on vcpu exit to userspace.
One thing to note is that it only works because this is done from the
vcpu thread itself.

>
> Fixes: 358b28f09f0a ("arm/arm64: KVM: Allow a VCPU to fully reset itself")
> Signed-off-by: Oliver Upton <oupton@google.com>
> ---
> I really hate this, but my imagination is failing me on any other way to
> cure the race without cluing in userspace. Any ideas?
> 
>  arch/arm64/kvm/arm.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 0de4b41c3706..6b124c29c663 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -1216,6 +1216,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
>  		if (copy_from_user(&reg, argp, sizeof(reg)))
>  			break;
>  
> +		/*
> +		 * ugly hack. We could owe a reset due to PSCI and not yet
> +		 * serviced it. Prevent userspace from reading/writing state
> +		 * that will be clobbered by the eventual handling of the reset
> +		 * bit.

This reads a bit odd. You are taking care of two potential issues in
one go here:
- userspace writes won't be overwritten by a pending reset as they
will take place after said reset
- userspace reads will reflect the state of the freshly reset CPU
instead of some stale state

> +		 */
> +		if (kvm_check_request(KVM_REQ_VCPU_RESET, vcpu))
> +			kvm_reset_vcpu(vcpu);
> +
>  		if (ioctl == KVM_SET_ONE_REG)
>  			r = kvm_arm_set_reg(vcpu, &reg);
>  		else

Otherwise, well spotted.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.