Current mainline kernel FTBFS in KVM SEV

Current mainline kernel FTBFS in KVM SEV

[Uros Bizjak]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

Current mainline kernel fails to build (on Fedora 31) with:

  GEN     .version
  CHK     include/generated/compile.h
  LD      vmlinux.o
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
  GEN     modules.builtin
  LD      .tmp_vmlinux.btf
ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
/hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
`sev_guest_df_flush'
ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
/hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
to `sev_platform_status'
  BTF     .btf.vmlinux.bin.o

.config is attached.

Uros.

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 53747 bytes --]

Re: Current mainline kernel FTBFS in KVM SEV

[Xu, Like]

Hi Bizjak,

would you mind telling us the top commit ID in your kernel tree ?
Or you may try the queue branch of 
https://git.kernel.org/pub/scm/virt/kvm/kvm.git
and check if this "undefined reference" issue gets fixed.

Thanks,
Like Xu

On 2020/4/9 16:20, Uros Bizjak wrote:
> Current mainline kernel fails to build (on Fedora 31) with:
>
>    GEN     .version
>    CHK     include/generated/compile.h
>    LD      vmlinux.o
>    MODPOST vmlinux.o
>    MODINFO modules.builtin.modinfo
>    GEN     modules.builtin
>    LD      .tmp_vmlinux.btf
> ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> `sev_guest_df_flush'
> ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> to `sev_platform_status'
>    BTF     .btf.vmlinux.bin.o
>
> .config is attached.
>
> Uros.

Re: Current mainline kernel FTBFS in KVM SEV

[Paolo Bonzini]

On 09/04/20 10:20, Uros Bizjak wrote:
> Current mainline kernel fails to build (on Fedora 31) with:
> 
>   GEN     .version
>   CHK     include/generated/compile.h
>   LD      vmlinux.o
>   MODPOST vmlinux.o
>   MODINFO modules.builtin.modinfo
>   GEN     modules.builtin
>   LD      .tmp_vmlinux.btf
> ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> `sev_guest_df_flush'
> ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> to `sev_platform_status'
>   BTF     .btf.vmlinux.bin.o

Strange, the functions are defined and exported with
CONFIG_CRYPTO_DEV_SP_PSP, which is "y" in your config.

Paolo

Re: Current mainline kernel FTBFS in KVM SEV

[Uros Bizjak]

On Thu, Apr 9, 2020 at 10:31 AM Xu, Like <like.xu@intel.com> wrote:
>
> Hi Bizjak,
>
> would you mind telling us the top commit ID in your kernel tree ?
> Or you may try the queue branch of
> https://git.kernel.org/pub/scm/virt/kvm/kvm.git
> and check if this "undefined reference" issue gets fixed.

Top of tree is at:

commit 5d30bcacd91af6874481129797af364a53cd9b46 (HEAD -> master,
origin/master, origin/HEAD)
Merge: fcc95f06403c c6f141412d24
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Apr 8 21:51:14 2020 -0700

    Merge tag '9p-for-5.7-2' of git://github.com/martinetd/linux

    Pull 9p documentation update from Dominique Martinet:
     "Document the new O_NONBLOCK short read behavior"

    * tag '9p-for-5.7-2' of git://github.com/martinetd/linux:
      9p: document short read behaviour with O_NONBLOCK

BR,
Uros.

> Thanks,
> Like Xu
>
> On 2020/4/9 16:20, Uros Bizjak wrote:
> > Current mainline kernel fails to build (on Fedora 31) with:
> >
> >    GEN     .version
> >    CHK     include/generated/compile.h
> >    LD      vmlinux.o
> >    MODPOST vmlinux.o
> >    MODINFO modules.builtin.modinfo
> >    GEN     modules.builtin
> >    LD      .tmp_vmlinux.btf
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> > `sev_guest_df_flush'
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> > to `sev_platform_status'
> >    BTF     .btf.vmlinux.bin.o
> >
> > .config is attached.
> >
> > Uros.
>

Re: Current mainline kernel FTBFS in KVM SEV

[Uros Bizjak]

On Thu, Apr 9, 2020 at 10:33 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 09/04/20 10:20, Uros Bizjak wrote:
> > Current mainline kernel fails to build (on Fedora 31) with:
> >
> >   GEN     .version
> >   CHK     include/generated/compile.h
> >   LD      vmlinux.o
> >   MODPOST vmlinux.o
> >   MODINFO modules.builtin.modinfo
> >   GEN     modules.builtin
> >   LD      .tmp_vmlinux.btf
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> > `sev_guest_df_flush'
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> > to `sev_platform_status'
> >   BTF     .btf.vmlinux.bin.o
>
> Strange, the functions are defined and exported with
> CONFIG_CRYPTO_DEV_SP_PSP, which is "y" in your config.

I tried to continue with a single make job (if there is something
wrong with dependencies). Still fails to build, but I can post the
continuation of errors from this single job:

[uros@localhost linux]$ make
  CALL    scripts/checksyscalls.sh
  CALL    scripts/atomic/check-atomics.sh
  DESCEND  objtool
  CHK     include/generated/compile.h
  CHK     kernel/kheaders_data.tar.xz
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  AR      init/built-in.a
  LD      vmlinux.o
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
  GEN     modules.builtin
  LD      .tmp_vmlinux.btf
ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
/hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
`sev_guest_df_flush'
ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
/hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
to `sev_platform_status'
  BTF     .btf.vmlinux.bin.o
tag__check_id_drift: subroutine_type id drift, core_id: 1644,
btf_type_id: 1642, type_id_off: 0
libbpf: Unsupported BTF_KIND:0
btf_elf__encode: btf__new failed!
free(): double free detected in tcache 2
scripts/link-vmlinux.sh: vrstica 114: 725221 Aborted
(izpis jedra) LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
  LD      .tmp_vmlinux.kallsyms1
.btf.vmlinux.bin.o: file not recognized: file format not recognized
make: *** [Makefile:1086: vmlinux] Error 1

Uros.

Re: Current mainline kernel FTBFS in KVM SEV

[Uros Bizjak]

On Thu, Apr 9, 2020 at 10:31 AM Xu, Like <like.xu@intel.com> wrote:
>
> Hi Bizjak,
>
> would you mind telling us the top commit ID in your kernel tree ?
> Or you may try the queue branch of
> https://git.kernel.org/pub/scm/virt/kvm/kvm.git
> and check if this "undefined reference" issue gets fixed.

Unfortunately no, the build breaks in the same way.

Uros.

> Thanks,
> Like Xu
>
> On 2020/4/9 16:20, Uros Bizjak wrote:
> > Current mainline kernel fails to build (on Fedora 31) with:
> >
> >    GEN     .version
> >    CHK     include/generated/compile.h
> >    LD      vmlinux.o
> >    MODPOST vmlinux.o
> >    MODINFO modules.builtin.modinfo
> >    GEN     modules.builtin
> >    LD      .tmp_vmlinux.btf
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> > `sev_guest_df_flush'
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> > to `sev_platform_status'
> >    BTF     .btf.vmlinux.bin.o
> >
> > .config is attached.
> >
> > Uros.
>

Re: Current mainline kernel FTBFS in KVM SEV

[Uros Bizjak]

On Thu, Apr 9, 2020 at 10:33 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 09/04/20 10:20, Uros Bizjak wrote:
> > Current mainline kernel fails to build (on Fedora 31) with:
> >
> >   GEN     .version
> >   CHK     include/generated/compile.h
> >   LD      vmlinux.o
> >   MODPOST vmlinux.o
> >   MODINFO modules.builtin.modinfo
> >   GEN     modules.builtin
> >   LD      .tmp_vmlinux.btf
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_flush_asids':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:48: undefined reference to
> > `sev_guest_df_flush'
> > ld: arch/x86/kvm/svm/sev.o: in function `sev_hardware_setup':
> > /hdd/uros/git/linux/arch/x86/kvm/svm/sev.c:1146: undefined reference
> > to `sev_platform_status'
> >   BTF     .btf.vmlinux.bin.o
>
> Strange, the functions are defined and exported with
> CONFIG_CRYPTO_DEV_SP_PSP, which is "y" in your config.

The problem is with

CONFIG_CRYPTO_DEV_CCP_DD=m

in combination with

CONFIG_KVM_AMD=y

in arch/x86/kvmKconfig, there is:

--cut here--
config KVM_AMD_SEV
    def_bool y
    bool "AMD Secure Encrypted Virtualization (SEV) support"
    depends on KVM_AMD && X86_64
    depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m)
    ---help---
    Provides support for launching Encrypted VMs on AMD processors.
--cut here--

which doesn't disable the compilation of sev.o. The missing functions
are actually in ccp.o *module*, called from built-in functions of
sev.o

Enabling CRYPTO_DEV_CCP_DD=y as a built-in instead of a module fixes the build.

Uros.

Re: Current mainline kernel FTBFS in KVM SEV

[Paolo Bonzini]

On 09/04/20 14:51, Uros Bizjak wrote:
> 
> --cut here--
> config KVM_AMD_SEV
>     def_bool y
>     bool "AMD Secure Encrypted Virtualization (SEV) support"
>     depends on KVM_AMD && X86_64
>     depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m)
>     ---help---
>     Provides support for launching Encrypted VMs on AMD processors.
> --cut here--
> 
> which doesn't disable the compilation of sev.o. The missing functions
> are actually in ccp.o *module*, called from built-in functions of
> sev.o

Yes, that's also what I was thinking but I confused SP_PSP with CCP_DD.

> Enabling CRYPTO_DEV_CCP_DD=y as a built-in instead of a module fixes the build.

What about this:

diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 0e3fc311d7da..364ffe32139c 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1117,7 +1117,7 @@ int __init sev_hardware_setup(void)
 	/* Maximum number of encrypted guests supported simultaneously */
 	max_sev_asid = cpuid_ecx(0x8000001F);
 
-	if (!max_sev_asid)
+	if (!svm_sev_enabled())
 		return 1;
 
 	/* Minimum ASID value that should be used for SEV guest */
@@ -1156,6 +1156,9 @@ int __init sev_hardware_setup(void)
 
 void sev_hardware_teardown(void)
 {
+	if (!svm_sev_enabled())
+		return;
+
 	bitmap_free(sev_asid_bitmap);
 	bitmap_free(sev_reclaim_asid_bitmap);
 
They should be the only places that call those functions and are not already
culled by svm_sev_enabled(), either directly or indirectly (most of sev.c
functions are static and the entry points reduce to just a return).

The two symbols go away for me with this change.

Paolo