Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files

From: Marc Zyngier <maz@kernel.org>
To: Auger Eric <eric.auger@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Alexander Graf <graf@amazon.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Andrew Scull <ascull@google.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Christoffer Dall <christoffer.dall@arm.com>,
	David Brazdil <dbrazdil@google.com>,
	Gavin Shan <gshan@redhat.com>, James Morse <james.morse@arm.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Peng Hao <richard.peng@oppo.com>,
	Quentin Perret <qperret@google.com>,
	Will Deacon <will@kernel.org>,
	Julien Thierry <julien.thierry.kdev@gmail.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	linux-arm-kernel@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
	kernel-team@android.com
Subject: Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Date: Wed, 05 May 2021 19:03:09 +0100	[thread overview]
Message-ID: <871ralnjmq.wl-maz@kernel.org> (raw)
In-Reply-To: <2ff3a1cb-a310-7963-4171-bd1e7d08e39b@redhat.com>

Hi Eric,

On Tue, 04 May 2021 15:47:36 +0100,
Auger Eric <eric.auger@redhat.com> wrote:
> 
> Hi David, Marc,
> 
> On 8/5/20 7:56 PM, Marc Zyngier wrote:
> > From: David Brazdil <dbrazdil@google.com>
> > 
> > Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building code
> > that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder for
> > hyp-specific header files which will include code common to VHE/nVHE.
> > 
> > Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
> > -D__KVM_VHE_HYPERVISOR__.
> > 
> > Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
> > prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
> > a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and nVHE
> > to share some source files, but compiled with different CFLAGS.
> > 
> > The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only symbols
> > will never appear in EL1 stack traces.
> > 
> > Due to symbol prefixing, add a section in image-vars.h for aliases of symbols
> > that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
> > 
> > Signed-off-by: David Brazdil <dbrazdil@google.com>
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > Link: https://lore.kernel.org/r/20200625131420.71444-4-dbrazdil@google.com
> > ---
> >  arch/arm64/kernel/image-vars.h   | 14 +++++++++++++
> >  arch/arm64/kvm/hyp/Makefile      | 10 +++++++---
> >  arch/arm64/kvm/hyp/nvhe/Makefile | 34 ++++++++++++++++++++++++++++++++
> >  arch/arm64/kvm/hyp/vhe/Makefile  | 17 ++++++++++++++++
> >  scripts/kallsyms.c               |  1 +
> >  5 files changed, 73 insertions(+), 3 deletions(-)
> >  create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
> >  create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile

[...]

> > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > index 6dc3078649fa..0096cd965332 100644
> > --- a/scripts/kallsyms.c
> > +++ b/scripts/kallsyms.c
> > @@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char type)
> >  		".LASANPC",		/* s390 kasan local symbols */
> >  		"__crc_",		/* modversions */
> >  		"__efistub_",		/* arm64 EFI stub namespace */
> > +		"__kvm_nvhe_",		/* arm64 non-VHE KVM namespace */
> The addition of this line seems to have introduced errors on the
> 'vmlinux symtab matches kallsyms' perf test (perf test -v 1) which fails
> on aarch64 for all __kvm_nvhe_ prefixed symbols, like
> 
> ERR : <addr> : __kvm_nvhe___invalid not on kallsyms
> ERR : <addr> : __kvm_nvhe___do_hyp_init not on kallsyms
> ERR : <addr> : __kvm_nvhe___kvm_handle_stub_hvc not on kallsyms
> ERR : <addr> : __kvm_nvhe_reset not on kallsyms
> ../..
> 
> I understand we willingly hided those symbols from /proc/kallsyms. Do
> you confirm the right fix is to upgrade the perf test suite accordingly?

Hmmm. This test always fail here, no matter whether I have this line
or not:

<quote>
maz@big-leg-emma:~$ sudo perf_5.10 test -v 1
 1: vmlinux symtab matches kallsyms                                 :
--- start ---
test child forked, pid 664
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
Looking at the vmlinux_path (8 entries long)
symsrc__init: cannot get elf header.
symsrc__init: cannot get elf header.
Couldn't find a vmlinux that matches the kernel running on this machine, skipping test
test child finished with -2
---- end ----
vmlinux symtab matches kallsyms: Skip
</quote>

Rookie question: How do you provide a kernel to the test framework?

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.