From: Marc Zyngier <maz@kernel.org>
To: Alexandru Elisei <alexandru.elisei@arm.com>
Cc: kvm@vger.kernel.org, pbonzini@redhat.com, drjones@redhat.com,
andre.przywara@arm.com
Subject: Re: [kvm-unit-tests RFC PATCH v3 5/7] lib: arm64: Add support for disabling and re-enabling VHE
Date: Thu, 30 Jan 2020 17:40:05 +0000 [thread overview]
Message-ID: <ad46bedcc585d03399576ecfce4c17c0@kernel.org> (raw)
In-Reply-To: <1577972806-16184-6-git-send-email-alexandru.elisei@arm.com>
Hi Alexandru,
On 2020-01-02 13:46, Alexandru Elisei wrote:
> Add a function to disable VHE and another one to re-enable VHE. Both
> functions work under the assumption that the CPU had VHE mode enabled
> at
> boot.
>
> Minimal support to run with VHE has been added to the TLB invalidate
> functions and to the exception handling code.
>
> Since we're touch the assembly enable/disable MMU code, let's take this
> opportunity to replace a magic number with the proper define.
I've been using this test case to debug my NV code... only to realize
after a few hours of banging my head on the wall that it is the test
that needed debugging, see below... ;-)
>
> Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
> ---
> lib/arm64/asm/mmu.h | 11 ++-
> lib/arm64/asm/pgtable-hwdef.h | 53 ++++++++---
> lib/arm64/asm/processor.h | 19 +++-
> lib/arm64/processor.c | 37 +++++++-
> arm/cstart64.S | 204
> ++++++++++++++++++++++++++++++++++++++++--
> 5 files changed, 300 insertions(+), 24 deletions(-)
[...]
> --- a/arm/cstart64.S
> +++ b/arm/cstart64.S
> @@ -104,6 +104,13 @@ exceptions_init:
>
> .text
>
> +exceptions_init_nvhe:
> + adrp x0, vector_table_nvhe
> + add x0, x0, :lo12:vector_table_nvhe
> + msr vbar_el2, x0
> + isb
> + ret
> +
> .globl get_mmu_off
> get_mmu_off:
> adrp x0, auxinfo
> @@ -203,7 +210,7 @@ asm_mmu_enable:
> TCR_IRGN_WBWA | TCR_ORGN_WBWA | \
> TCR_SHARED
> mrs x2, id_aa64mmfr0_el1
> - bfi x1, x2, #32, #3
> + bfi x1, x2, #TCR_EL1_IPS_SHIFT, #3
> msr tcr_el1, x1
>
> /* MAIR */
> @@ -228,6 +235,41 @@ asm_mmu_enable:
>
> ret
>
> +asm_mmu_enable_nvhe:
Note the "_nvhe" suffix, which implies that...
> + tlbi alle2
> + dsb nsh
> +
> + /* TCR */
> + ldr x1, =TCR_EL2_RES1 | \
> + TCR_T0SZ(VA_BITS) | \
> + TCR_TG0_64K | \
> + TCR_IRGN0_WBWA | TCR_ORGN0_WBWA | \
> + TCR_SH0_IS
> + mrs x2, id_aa64mmfr0_el1
> + bfi x1, x2, #TCR_EL2_PS_SHIFT, #3
> + msr tcr_el2, x1
> +
> + /* Same MAIR and TTBR0 as in VHE mode */
> + ldr x1, =MAIR(0x00, MT_DEVICE_nGnRnE) | \
> + MAIR(0x04, MT_DEVICE_nGnRE) | \
> + MAIR(0x0c, MT_DEVICE_GRE) | \
> + MAIR(0x44, MT_NORMAL_NC) | \
> + MAIR(0xff, MT_NORMAL)
> + msr mair_el1, x1
... this should be mair_el2...
> +
> + msr ttbr0_el1, x0
... and this should be ttbr0_el2.
> + isb
> +
> + /* SCTLR */
> + ldr x1, =SCTLR_EL2_RES1 | \
> + SCTLR_EL2_C | \
> + SCTLR_EL2_I | \
> + SCTLR_EL2_M
> + msr sctlr_el2, x1
> + isb
> +
> + ret
> +
> /* Taken with small changes from arch/arm64/incluse/asm/assembler.h */
> .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2
> adrp \tmp1, dcache_line_size
> @@ -242,21 +284,61 @@ asm_mmu_enable:
> dsb \domain
> .endm
>
> +clean_inval_cache:
> + adrp x0, __phys_offset
> + ldr x0, [x0, :lo12:__phys_offset]
> + adrp x1, __phys_end
> + ldr x1, [x1, :lo12:__phys_end]
> + dcache_by_line_op civac, sy, x0, x1, x2, x3
> + isb
> + ret
> +
> .globl asm_mmu_disable
> asm_mmu_disable:
> mrs x0, sctlr_el1
> bic x0, x0, SCTLR_EL1_M
> msr sctlr_el1, x0
> isb
> + b clean_inval_cache
>
> - /* Clean + invalidate the entire memory */
> - adrp x0, __phys_offset
> - ldr x0, [x0, :lo12:__phys_offset]
> - adrp x1, __phys_end
> - ldr x1, [x1, :lo12:__phys_end]
> - dcache_by_line_op civac, sy, x0, x1, x2, x3
> +asm_mmu_disable_nvhe:
> + mrs x0, sctlr_el2
> + bic x0, x0, SCTLR_EL2_M
> + msr sctlr_el2, x0
> + isb
> + b clean_inval_cache
> +
> +.globl asm_disable_vhe
> +asm_disable_vhe:
> + str x30, [sp, #-16]!
> +
> + bl asm_mmu_disable
> + msr hcr_el2, xzr
> + isb
At this stage, VHE is off...
> + bl exceptions_init_nvhe
> + /* Make asm_mmu_enable_nvhe happy by having TTBR0 value in x0. */
> + mrs x0, ttbr0_el1
... so this is going to sample the wrong TTBR. It really should be
TTBR0_EL2!
> + isb
nit: this ISB is useless, as you will have a dependency on x0 anyway.
With these fixes (and a few more terrible hacks to synchronize HCR_EL2
on ARMv8.4-NV), I can run this test reliably.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
next prev parent reply other threads:[~2020-01-30 17:40 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-02 13:46 [kvm-unit-tests RFC PATCH v3 0/7] arm64: Run at EL2 Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 1/7] lib: Add _UL and _ULL definitions to linux/const.h Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 2/7] lib: arm64: Run existing tests at EL2 Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 3/7] arm64: timer: Add test for EL2 timers Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 4/7] arm64: selftest: Add basic test for EL2 Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 5/7] lib: arm64: Add support for disabling and re-enabling VHE Alexandru Elisei
2020-01-30 17:40 ` Marc Zyngier [this message]
2020-01-31 9:52 ` Alexandru Elisei
2020-01-31 11:26 ` Marc Zyngier
2020-01-31 11:43 ` Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 6/7] arm64: selftest: Expand EL2 test to disable and re-enable VHE Alexandru Elisei
2020-01-02 13:46 ` [kvm-unit-tests RFC PATCH v3 7/7] arm64: timer: Run tests with VHE disabled Alexandru Elisei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ad46bedcc585d03399576ecfce4c17c0@kernel.org \
--to=maz@kernel.org \
--cc=alexandru.elisei@arm.com \
--cc=andre.przywara@arm.com \
--cc=drjones@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=pbonzini@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).