From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D65ACCA487 for ; Wed, 8 Jun 2022 01:08:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.343581.568936 (Exim 4.92) (envelope-from ) id 1nykBX-0006UD-WE; Wed, 08 Jun 2022 01:08:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 343581.568936; Wed, 08 Jun 2022 01:08:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nykBX-0006U6-TL; Wed, 08 Jun 2022 01:08:35 +0000 Received: by outflank-mailman (input) for mailman id 343581; Wed, 08 Jun 2022 01:08:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nykBW-0006U0-37 for xen-devel@lists.xenproject.org; Wed, 08 Jun 2022 01:08:34 +0000 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 83a19904-e6c7-11ec-bd2c-47488cf2e6aa; Wed, 08 Jun 2022 03:08:32 +0200 (CEST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 869DEB82489; Wed, 8 Jun 2022 01:08:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B27E3C34114; Wed, 8 Jun 2022 01:08:29 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 83a19904-e6c7-11ec-bd2c-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1654650510; bh=I4fcBOTQjhfyeFkrEprR88T4LuWZ2bbkS5qhS+uJu9I=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=qh+pJ2hLltIdqnZGKY6F8mA2dlapUDMDyLJN5xdVL7BhiMKssaYIJPHHrGAH1xbHV iFgzYDhn04dbKEhOw8VnK356yqIMcWpmecS12SSkkDco+GfbGvmx4bGuy1G2yUSAeb IdZ82ucOMdLiyvHAm5HNUdJR69l+x5oWryrh/86XcGHxNB16/RFvLSXx8tDGDZ2Ek5 d3Uy/ddeYb4sNKBevmXyzP3xS+srnNcibam8AaVO88iSGUEnMroQ+ZezRN6N1eOcnE QNi62z8AYqy/paRVBLY2U7U30J0HjYzRzV5VyfzF0WSGjH0tgqz7L32UBqjcsQCrF1 fTMwRyA59MuFg== Date: Tue, 7 Jun 2022 18:08:29 -0700 (PDT) From: Stefano Stabellini X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop To: Julien Grall cc: xen-devel@lists.xenproject.org, Wei Liu , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , George Dunlap , Hongyan Xia , Julien Grall , Jan Beulich , Wei Liu , Andrew Cooper , =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: Re: [PATCH 10/16] xen/arm: add Persistent Map (PMAP) infrastructure In-Reply-To: <20220520120937.28925-11-julien@xen.org> Message-ID: References: <20220520120937.28925-1-julien@xen.org> <20220520120937.28925-11-julien@xen.org> User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="8323329-2035065655-1654650510=:21215" This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-2035065655-1654650510=:21215 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT On Fri, 20 May 2022, Julien Grall wrote: > From: Wei Liu > > The basic idea is like Persistent Kernel Map (PKMAP) in Linux. We > pre-populate all the relevant page tables before the system is fully > set up. > > We will need it on Arm in order to rework the arm64 version of > xenheap_setup_mappings() as we may need to use pages allocated from > the boot allocator before they are effectively mapped. > > This infrastructure is not lock-protected therefore can only be used > before smpboot. After smpboot, map_domain_page() has to be used. > > This is based on the x86 version [1] that was originally implemented > by Wei Liu. > > The PMAP infrastructure is implemented in common code with some > arch helpers to set/clear the page-table entries and convertion > between a fixmap slot to a virtual address... > > As mfn_to_xen_entry() now needs to be exported, take the opportunity > to swich the parameter attr from unsigned to unsigned int. > > [1] > > Signed-off-by: Wei Liu > Signed-off-by: Hongyan Xia > [julien: Adapted for Arm] > Signed-off-by: Julien Grall > > --- > Changes in v4: > - Move xen_fixmap in fixmap.h and add a comment about its usage. > - Update comments > - Use DECLARE_BITMAP() > - Replace local_irq_{enable, disable} with an ASSERT() as there > should be no user of pmap() in interrupt context. > > Changes in v3: > - s/BITS_PER_LONG/BITS_PER_BYTE/ > - Move pmap to common code > > Changes in v2: > - New patch > > Cc: Jan Beulich > Cc: Wei Liu > Cc: Andrew Cooper > Cc: Roger Pau Monné > --- > xen/arch/arm/Kconfig | 1 + > xen/arch/arm/include/asm/fixmap.h | 24 +++++++++++ > xen/arch/arm/include/asm/lpae.h | 8 ++++ > xen/arch/arm/include/asm/pmap.h | 32 ++++++++++++++ > xen/arch/arm/mm.c | 7 +-- > xen/common/Kconfig | 3 ++ > xen/common/Makefile | 1 + > xen/common/pmap.c | 72 +++++++++++++++++++++++++++++++ > xen/include/xen/pmap.h | 16 +++++++ > 9 files changed, 158 insertions(+), 6 deletions(-) > create mode 100644 xen/arch/arm/include/asm/pmap.h > create mode 100644 xen/common/pmap.c > create mode 100644 xen/include/xen/pmap.h > > diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig > index ecfa6822e4d3..a89a67802aa9 100644 > --- a/xen/arch/arm/Kconfig > +++ b/xen/arch/arm/Kconfig > @@ -14,6 +14,7 @@ config ARM > select HAS_DEVICE_TREE > select HAS_PASSTHROUGH > select HAS_PDX > + select HAS_PMAP > select IOMMU_FORCE_PT_SHARE > > config ARCH_DEFCONFIG > diff --git a/xen/arch/arm/include/asm/fixmap.h b/xen/arch/arm/include/asm/fixmap.h > index 1cee51e52ab9..365a2385a087 100644 > --- a/xen/arch/arm/include/asm/fixmap.h > +++ b/xen/arch/arm/include/asm/fixmap.h > @@ -5,20 +5,44 @@ > #define __ASM_FIXMAP_H > > #include > +#include > > /* Fixmap slots */ > #define FIXMAP_CONSOLE 0 /* The primary UART */ > #define FIXMAP_MISC 1 /* Ephemeral mappings of hardware */ > #define FIXMAP_ACPI_BEGIN 2 /* Start mappings of ACPI tables */ > #define FIXMAP_ACPI_END (FIXMAP_ACPI_BEGIN + NUM_FIXMAP_ACPI_PAGES - 1) /* End mappings of ACPI tables */ > +#define FIXMAP_PMAP_BEGIN (FIXMAP_ACPI_END + 1) /* Start of PMAP */ > +#define FIXMAP_PMAP_END (FIXMAP_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of PMAP */ > + > +#define FIXMAP_LAST FIXMAP_PMAP_END > + > +#define FIXADDR_START FIXMAP_ADDR(0) > +#define FIXADDR_TOP FIXMAP_ADDR(FIXMAP_LAST) > > #ifndef __ASSEMBLY__ > > +/* > + * Direct access to xen_fixmap[] should only happen when {set, > + * clear}_fixmap() is unusable (e.g. where we would end up to > + * recursively call the helpers). > + */ > +extern lpae_t xen_fixmap[XEN_PT_LPAE_ENTRIES]; > + > /* Map a page in a fixmap entry */ > extern void set_fixmap(unsigned map, mfn_t mfn, unsigned attributes); > /* Remove a mapping from a fixmap entry */ > extern void clear_fixmap(unsigned map); > > +#define fix_to_virt(slot) ((void *)FIXMAP_ADDR(slot)) > + > +static inline unsigned int virt_to_fix(vaddr_t vaddr) > +{ > + BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); > + > + return ((vaddr - FIXADDR_START) >> PAGE_SHIFT); > +} > + > #endif /* __ASSEMBLY__ */ > > #endif /* __ASM_FIXMAP_H */ > diff --git a/xen/arch/arm/include/asm/lpae.h b/xen/arch/arm/include/asm/lpae.h > index aecb320dec45..fc19cbd84772 100644 > --- a/xen/arch/arm/include/asm/lpae.h > +++ b/xen/arch/arm/include/asm/lpae.h > @@ -4,6 +4,7 @@ > #ifndef __ASSEMBLY__ > > #include > +#include > > /* > * WARNING! Unlike the x86 pagetable code, where l1 is the lowest level and > @@ -168,6 +169,13 @@ static inline bool lpae_is_superpage(lpae_t pte, unsigned int level) > third_table_offset(addr) \ > } > > +/* > + * Standard entry type that we'll use to build Xen's own pagetables. > + * We put the same permissions at every level, because they're ignored > + * by the walker in non-leaf entries. > + */ > +lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); > + > #endif /* __ASSEMBLY__ */ > > /* > diff --git a/xen/arch/arm/include/asm/pmap.h b/xen/arch/arm/include/asm/pmap.h > new file mode 100644 > index 000000000000..74398b4c4fe6 > --- /dev/null > +++ b/xen/arch/arm/include/asm/pmap.h > @@ -0,0 +1,32 @@ > +#ifndef __ASM_PMAP_H__ > +#define __ASM_PMAP_H__ > + > +#include > + > +#include > + > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > +{ > + lpae_t *entry = &xen_fixmap[slot]; > + lpae_t pte; > + > + ASSERT(!lpae_is_valid(*entry)); > + > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > + pte.pt.table = 1; > + write_pte(entry, pte); Here we don't need a tlb flush because we never go from a valid mapping to another valid mapping. We also go through arch_pmap_unmap which clears the mapping and also flushes the tlb. Is that right? > +} > + > +static inline void arch_pmap_unmap(unsigned int slot) > +{ > + lpae_t pte = {}; > + > + write_pte(&xen_fixmap[slot], pte); > + > + flush_xen_tlb_range_va_local(FIXMAP_ADDR(slot), PAGE_SIZE); > +} > + > +void arch_pmap_map_slot(unsigned int slot, mfn_t mfn); > +void arch_pmap_clear_slot(void *ptr); What are these two? They are not defined anywhere? > +#endif /* __ASM_PMAP_H__ */ > diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c > index 52b2a0394047..bd1348a99716 100644 > --- a/xen/arch/arm/mm.c > +++ b/xen/arch/arm/mm.c > @@ -305,12 +305,7 @@ void dump_hyp_walk(vaddr_t addr) > dump_pt_walk(ttbr, addr, HYP_PT_ROOT_LEVEL, 1); > } > > -/* > - * Standard entry type that we'll use to build Xen's own pagetables. > - * We put the same permissions at every level, because they're ignored > - * by the walker in non-leaf entries. > - */ > -static inline lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned attr) > +lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) > { > lpae_t e = (lpae_t) { > .pt = { > diff --git a/xen/common/Kconfig b/xen/common/Kconfig > index d921c74d615e..5b6b2406c028 100644 > --- a/xen/common/Kconfig > +++ b/xen/common/Kconfig > @@ -49,6 +49,9 @@ config HAS_KEXEC > config HAS_PDX > bool > > +config HAS_PMAP > + bool > + > config HAS_SCHED_GRANULARITY > bool > > diff --git a/xen/common/Makefile b/xen/common/Makefile > index b1e076c30b81..3baf83d527d8 100644 > --- a/xen/common/Makefile > +++ b/xen/common/Makefile > @@ -29,6 +29,7 @@ obj-y += notifier.o > obj-y += page_alloc.o > obj-$(CONFIG_HAS_PDX) += pdx.o > obj-$(CONFIG_PERF_COUNTERS) += perfc.o > +obj-bin-$(CONFIG_HAS_PMAP) += pmap.init.o > obj-y += preempt.o > obj-y += random.o > obj-y += rangeset.o > diff --git a/xen/common/pmap.c b/xen/common/pmap.c > new file mode 100644 > index 000000000000..9355cacb7373 > --- /dev/null > +++ b/xen/common/pmap.c > @@ -0,0 +1,72 @@ > +#include > +#include > +#include > +#include > + > +#include > +#include > + > +/* > + * Simple mapping infrastructure to map / unmap pages in fixed map. > + * This is used to set the page table before the map domain page infrastructure > + * is initialized. > + * > + * This structure is not protected by any locks, so it must not be used after > + * smp bring-up. > + */ > + > +/* Bitmap to track which slot is used */ > +static __initdata DECLARE_BITMAP(inuse, NUM_FIX_PMAP); > + > +void *__init pmap_map(mfn_t mfn) > +{ > + unsigned int idx; > + unsigned int slot; > + > + ASSERT(system_state < SYS_STATE_smp_boot); > + ASSERT(!in_irq()); > + > + idx = find_first_zero_bit(inuse, NUM_FIX_PMAP); > + if ( idx == NUM_FIX_PMAP ) > + panic("Out of PMAP slots\n"); > + > + __set_bit(idx, inuse); > + > + slot = idx + FIXMAP_PMAP_BEGIN; > + ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END); > + > + /* > + * We cannot use set_fixmap() here. We use PMAP when the domain map > + * page infrastructure is not yet initialized, so map_pages_to_xen() called > + * by set_fixmap() needs to map pages on demand, which then calls pmap() > + * again, resulting in a loop. Modify the PTEs directly instead. The same > + * is true for pmap_unmap(). > + */ > + arch_pmap_map(slot, mfn); > + > + return fix_to_virt(slot); > +} > + > +void __init pmap_unmap(const void *p) > +{ > + unsigned int idx; > + unsigned int slot = virt_to_fix((unsigned long)p); > + > + ASSERT(system_state < SYS_STATE_smp_boot); > + ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END); > + ASSERT(in_irq()); > + > + idx = slot - FIXMAP_PMAP_BEGIN; > + > + __clear_bit(idx, inuse); > + arch_pmap_unmap(slot); > +} > + > +/* > + * Local variables: > + * mode: C > + * c-file-style: "BSD" > + * c-basic-offset: 4 > + * indent-tabs-mode: nil > + * End: > + */ > diff --git a/xen/include/xen/pmap.h b/xen/include/xen/pmap.h > new file mode 100644 > index 000000000000..93e61b10870e > --- /dev/null > +++ b/xen/include/xen/pmap.h > @@ -0,0 +1,16 @@ > +#ifndef __XEN_PMAP_H__ > +#define __XEN_PMAP_H__ > + > +/* Large enough for mapping 5 levels of page tables with some headroom */ > +#define NUM_FIX_PMAP 8 > + > +#ifndef __ASSEMBLY__ > + > +#include > + > +void *pmap_map(mfn_t mfn); > +void pmap_unmap(const void *p); > + > +#endif /* __ASSEMBLY__ */ > + > +#endif /* __XEN_PMAP_H__ */ > -- > 2.32.0 > --8323329-2035065655-1654650510=:21215--