[PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Jianyong Wu]

821d935c87b introduces support for userspace SMCCC filtering, but lack
of a way to tell userspace if we have this feature. Add a corresponding
feature extension can resolve this issue.

For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
As there is no way to check this feature, VMM will run into error when
it calls this feature on an old kernel. It's bad for backward compatible.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
---
 Documentation/virt/kvm/api.rst | 3 ++-
 arch/arm64/kvm/arm.c           | 1 +
 include/uapi/linux/kvm.h       | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 7025b3751027..559c6c531bfd 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6395,7 +6395,8 @@ For arm64:
 ----------
 
 SMCCC exits can be enabled depending on the configuration of the SMCCC
-filter. See the Documentation/virt/kvm/devices/vm.rst
+filter. This feature can be only available if KVM_CAP_ARM_VM_SMCCC is
+upported. See the Documentation/virt/kvm/devices/vm.rst
 ``KVM_ARM_SMCCC_FILTER`` for more details.
 
 ``nr`` contains the function ID of the guest's SMCCC call. Userspace is
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e5f75f1f1085..44583da440ae 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -241,6 +241,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_SYSTEM_SUSPEND:
 	case KVM_CAP_IRQFD_RESAMPLE:
 	case KVM_CAP_COUNTER_OFFSET:
+	case KVM_CAP_ARM_VM_SMCCC:
 		r = 1;
 		break;
 	case KVM_CAP_SET_GUEST_DEBUG2:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 211b86de35ac..e67fb1df508d 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1201,6 +1201,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE 228
 #define KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES 229
 #define KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES 230
+#define KVM_CAP_ARM_VM_SMCCC 231
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
2.34.1

Re: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Cornelia Huck]

On Thu, Nov 16 2023, Jianyong Wu <jianyong.wu@arm.com> wrote:

> 821d935c87b introduces support for userspace SMCCC filtering, but lack
> of a way to tell userspace if we have this feature. Add a corresponding
> feature extension can resolve this issue.
>
> For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> As there is no way to check this feature, VMM will run into error when
> it calls this feature on an old kernel. It's bad for backward compatible.

Can't you simply query via KVM_HAS_DEVICE_ATTR whether the SMCCC
filtering controls exist?

>
> Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> ---
>  Documentation/virt/kvm/api.rst | 3 ++-
>  arch/arm64/kvm/arm.c           | 1 +
>  include/uapi/linux/kvm.h       | 1 +
>  3 files changed, 4 insertions(+), 1 deletion(-)
>

RE: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Salil Mehta]

> From: Cornelia Huck <cohuck@redhat.com>
> Sent: Thursday, November 16, 2023 1:09 PM
> To: Jianyong Wu <jianyong.wu@arm.com>; maz@kernel.org; james.morse@arm.com;
> will@kernel.org
> 
> On Thu, Nov 16 2023, Jianyong Wu <jianyong.wu@arm.com> wrote:
> 
> > 821d935c87b introduces support for userspace SMCCC filtering, but lack
> > of a way to tell userspace if we have this feature. Add a corresponding
> > feature extension can resolve this issue.
> >
> > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > As there is no way to check this feature, VMM will run into error when
> > it calls this feature on an old kernel. It's bad for backward compatible.
> 
> Can't you simply query via KVM_HAS_DEVICE_ATTR whether the SMCCC
> filtering controls exist?


Agreed. In fact, this is what I had earlier intended to do but deferred this
change. As of now, RFC V2 of vCPU Hotplug series does not have this check yet
while installing the SMCCC filters in KVM Host.

Thanks

> > Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> > ---
> >  Documentation/virt/kvm/api.rst | 3 ++-
> >  arch/arm64/kvm/arm.c           | 1 +
> >  include/uapi/linux/kvm.h       | 1 +
> >  3 files changed, 4 insertions(+), 1 deletion(-)
> >

Re: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Marc Zyngier]

On Thu, 16 Nov 2023 13:08:58 +0000,
Cornelia Huck <cohuck@redhat.com> wrote:
> 
> On Thu, Nov 16 2023, Jianyong Wu <jianyong.wu@arm.com> wrote:
> 
> > 821d935c87b introduces support for userspace SMCCC filtering, but lack
> > of a way to tell userspace if we have this feature. Add a corresponding
> > feature extension can resolve this issue.
> >
> > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > As there is no way to check this feature, VMM will run into error when
> > it calls this feature on an old kernel. It's bad for backward compatible.
> 
> Can't you simply query via KVM_HAS_DEVICE_ATTR whether the SMCCC
> filtering controls exist?

Quite. Commit e0fc6b21616dd introduced it for that exact purpose,
specifically to prevent adding more of these capabilities when there
is a corresponding attribute that can be readily queried.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Russell King (Oracle)]

On Thu, Nov 16, 2023 at 11:41:52AM +0000, Jianyong Wu wrote:
> 821d935c87b introduces support for userspace SMCCC filtering, but lack
> of a way to tell userspace if we have this feature. Add a corresponding
> feature extension can resolve this issue.
> 
> For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> As there is no way to check this feature, VMM will run into error when
> it calls this feature on an old kernel. It's bad for backward compatible.

Can't you just attempt to use the SMCCC filtering, and if it errors out
with the appropriate error code, decide that SMCCC filtering is not
available?

That's how most things like kernel syscalls work - if they're not
implemented they return -ENOSYS. glibc can detect that and use a
fallback.

Imagine what it would be like if the kernel provided userspace with
a large bitmap of what syscalls were implemented...

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Oliver Upton]

On Thu, Nov 16, 2023 at 07:06:18PM +0000, Russell King (Oracle) wrote:
> On Thu, Nov 16, 2023 at 11:41:52AM +0000, Jianyong Wu wrote:
> > 821d935c87b introduces support for userspace SMCCC filtering, but lack
> > of a way to tell userspace if we have this feature. Add a corresponding
> > feature extension can resolve this issue.
> > 
> > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > As there is no way to check this feature, VMM will run into error when
> > it calls this feature on an old kernel. It's bad for backward compatible.
> 
> Can't you just attempt to use the SMCCC filtering, and if it errors out
> with the appropriate error code, decide that SMCCC filtering is not
> available?

That would also work, as we return ENXIO for the unsupported ioctl.

> That's how most things like kernel syscalls work - if they're not
> implemented they return -ENOSYS. glibc can detect that and use a
> fallback.

I generally agree, but KVM has gone in the other direction of providing
auxiliary interfaces for discovering new UAPI. ENXIO has been slightly
overloaded to imply that a given ioctl is non-existent or otherwise
unsupported due to some dynamic configuration.

Is it ideal? Of course not. With that said userspace may as well use the
preferred / documented discoverability mechanism. And in Jianyong's case
the KVM documentation is rather unambiguous (for once) about how you
discover device attributes.

https://docs.kernel.org/virt/kvm/api.html#kvm-has-device-attr

-- 
Thanks,
Oliver

RE: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Jianyong Wu]

> -----Original Message-----
> From: Salil Mehta <salil.mehta@huawei.com>
> Sent: 2023年11月16日 22:06
> To: Cornelia Huck <cohuck@redhat.com>; Jianyong Wu
> <Jianyong.Wu@arm.com>; maz@kernel.org; James Morse
> <James.Morse@arm.com>; will@kernel.org
> Cc: rmk@armlinux.org.uk; Suzuki Poulose <Suzuki.Poulose@arm.com>;
> oliver.upton@linux.dev; linux-arm-kernel@lists.infradead.org;
> kvmarm@lists.linux.dev; linux-kernel@vger.kernel.org; Justin He
> <Justin.He@arm.com>; Jianyong Wu <Jianyong.Wu@arm.com>
> Subject: RE: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter
> 
> > From: Cornelia Huck <cohuck@redhat.com>
> > Sent: Thursday, November 16, 2023 1:09 PM
> > To: Jianyong Wu <jianyong.wu@arm.com>; maz@kernel.org;
> > james.morse@arm.com; will@kernel.org
> >
> > On Thu, Nov 16 2023, Jianyong Wu <jianyong.wu@arm.com> wrote:
> >
> > > 821d935c87b introduces support for userspace SMCCC filtering, but
> > > lack of a way to tell userspace if we have this feature. Add a
> > > corresponding feature extension can resolve this issue.
> > >
> > > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > > As there is no way to check this feature, VMM will run into error
> > > when it calls this feature on an old kernel. It's bad for backward compatible.
> >
> > Can't you simply query via KVM_HAS_DEVICE_ATTR whether the SMCCC
> > filtering controls exist?
> 
> 
> Agreed. In fact, this is what I had earlier intended to do but deferred this change.
> As of now, RFC V2 of vCPU Hotplug series does not have this check yet while
> installing the SMCCC filters in KVM Host.
> 

Yes, we can use KVM_HAS_DEVICE_ATTR to do the check in userspace. Thanks Cornelia.

Thanks
Jianyong

> Thanks
> 
> > > Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
> > > ---
> > >  Documentation/virt/kvm/api.rst | 3 ++-
> > >  arch/arm64/kvm/arm.c           | 1 +
> > >  include/uapi/linux/kvm.h       | 1 +
> > >  3 files changed, 4 insertions(+), 1 deletion(-)
> > >

RE: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Jianyong Wu]

> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: 2023年11月16日 22:22
> To: Cornelia Huck <cohuck@redhat.com>
> Cc: Jianyong Wu <Jianyong.Wu@arm.com>; James Morse
> <James.Morse@arm.com>; will@kernel.org; rmk@armlinux.org.uk;
> salil.mehta@huawei.com; Suzuki Poulose <Suzuki.Poulose@arm.com>;
> oliver.upton@linux.dev; linux-arm-kernel@lists.infradead.org;
> kvmarm@lists.linux.dev; linux-kernel@vger.kernel.org; Justin He
> <Justin.He@arm.com>
> Subject: Re: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter
> 
> On Thu, 16 Nov 2023 13:08:58 +0000,
> Cornelia Huck <cohuck@redhat.com> wrote:
> >
> > On Thu, Nov 16 2023, Jianyong Wu <jianyong.wu@arm.com> wrote:
> >
> > > 821d935c87b introduces support for userspace SMCCC filtering, but
> > > lack of a way to tell userspace if we have this feature. Add a
> > > corresponding feature extension can resolve this issue.
> > >
> > > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > > As there is no way to check this feature, VMM will run into error
> > > when it calls this feature on an old kernel. It's bad for backward compatible.
> >
> > Can't you simply query via KVM_HAS_DEVICE_ATTR whether the SMCCC
> > filtering controls exist?
> 
> Quite. Commit e0fc6b21616dd introduced it for that exact purpose, specifically
> to prevent adding more of these capabilities when there is a corresponding
> attribute that can be readily queried.

Exactly. Commit e0fc6b21616dd has done for this. Ignore this patch.

Thanks
Jianyong
> 
> 	M.
> 
> --
> Without deviation from the norm, progress is not possible.

RE: [PATCH] arm64/kvm: Introduce feature extension for SMCCC filter

[Salil Mehta]

> From: Oliver Upton <oliver.upton@linux.dev>
> Sent: Thursday, November 16, 2023 11:22 PM
> To: Russell King (Oracle) <linux@armlinux.org.uk>
> 
> On Thu, Nov 16, 2023 at 07:06:18PM +0000, Russell King (Oracle) wrote:
> > On Thu, Nov 16, 2023 at 11:41:52AM +0000, Jianyong Wu wrote:
> > > 821d935c87b introduces support for userspace SMCCC filtering, but lack
> > > of a way to tell userspace if we have this feature. Add a corresponding
> > > feature extension can resolve this issue.
> > >
> > > For example, the incoming feature Vcpu Hotplug needs the SMCCC filter.
> > > As there is no way to check this feature, VMM will run into error when
> > > it calls this feature on an old kernel. It's bad for backward compatible.
> >
> > Can't you just attempt to use the SMCCC filtering, and if it errors out
> > with the appropriate error code, decide that SMCCC filtering is not
> > available?
> 
> That would also work, as we return ENXIO for the unsupported ioctl.
> 
> > That's how most things like kernel syscalls work - if they're not
> > implemented they return -ENOSYS. glibc can detect that and use a
> > fallback.
> 
> I generally agree, but KVM has gone in the other direction of providing
> auxiliary interfaces for discovering new UAPI. ENXIO has been slightly
> overloaded to imply that a given ioctl is non-existent or otherwise
> unsupported due to some dynamic configuration.


Agreed. We require this check for vCPU Hotplug series as well exactly
for the reason you stated above i.e. to clearly distinguish the case
when KVM host does not support SMCCC filter and when it does but an
error is purged out during configuration of this filter. In the later
we would like to abort the VM initialization (as being done in RFC V2)
but in former we would just continue without supporting vCPU Hotplug
feature. Handling is different in each.

Thanks
Salil.

> 
> Is it ideal? Of course not. With that said userspace may as well use the
> preferred / documented discoverability mechanism. And in Jianyong's case
> the KVM documentation is rather unambiguous (for once) about how you
> discover device attributes.
> 
> https://docs.kernel.org/virt/kvm/api.html#kvm-has-device-attr
> 
> --
> Thanks,
> Oliver