xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Julien Grall <julien@xen.org>
To: "Xia, Hongyan" <hongyxia@amazon.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "Penny.Zheng@arm.com" <Penny.Zheng@arm.com>,
	"Bertrand.Marquis@arm.com" <Bertrand.Marquis@arm.com>,
	"Wei.Chen@arm.com" <Wei.Chen@arm.com>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"wei.liu2@citrix.com" <wei.liu2@citrix.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>,
	"Volodymyr_Babchuk@epam.com" <Volodymyr_Babchuk@epam.com>,
	"hongyax@amazon.com" <hongyax@amazon.com>,
	"Henry.Wang@arm.com" <Henry.Wang@arm.com>,
	"wl@xen.org" <wl@xen.org>, "Grall, Julien" <jgrall@amazon.co.uk>,
	"jbeulich@suse.com" <jbeulich@suse.com>
Subject: Re: [PATCH RFCv2 12/15] xen/arm: add Persistent Map (PMAP) infrastructure
Date: Sat, 3 Jul 2021 18:57:13 +0100	[thread overview]
Message-ID: <d3bdec7f-837e-a055-4fe5-a5ceff0c5a51@xen.org> (raw)
In-Reply-To: <baccb4ab6c71a655a4ac19cc27f2eacc116b8d4b.camel@amazon.com>

Hi Hongyan,

On 26/04/2021 10:41, Xia, Hongyan wrote:
> On Sun, 2021-04-25 at 21:13 +0100, Julien Grall wrote:
>> From: Wei Liu <wei.liu2@citrix.com>
>>
>> The basic idea is like Persistent Kernel Map (PKMAP) in Linux. We
>> pre-populate all the relevant page tables before the system is fully
>> set up.
>>
>> We will need it on Arm in order to rework the arm64 version of
>> xenheap_setup_mappings() as we may need to use pages allocated from
>> the boot allocator before they are effectively mapped.
>>
>> This infrastructure is not lock-protected therefore can only be used
>> before smpboot. After smpboot, map_domain_page() has to be used.
>>
>> This is based on the x86 version [1] that was originally implemented
>> by Wei Liu.
>>
>> Take the opportunity to switch the parameter attr from unsigned to
>> unsigned int.
>>
>> [1] <
>> e92da4ad6015b6089737fcccba3ec1d6424649a5.1588278317.git.hongyxia@amazon.com
>>>
>>
>> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
>> [julien: Adapted for Arm]
>> Signed-off-by: Julien Grall <jgrall@amazon.com>
> 
> [...]
> 
>> diff --git a/xen/arch/arm/pmap.c b/xen/arch/arm/pmap.c
>> new file mode 100644
>> index 000000000000..702b1bde982d
>> --- /dev/null
>> +++ b/xen/arch/arm/pmap.c
>> @@ -0,0 +1,101 @@
>> +#include <xen/init.h>
>> +#include <xen/mm.h>
>> +
>> +#include <asm/bitops.h>
>> +#include <asm/flushtlb.h>
>> +#include <asm/pmap.h>
>> +
>> +/*
>> + * To be able to use FIXMAP_PMAP_BEGIN.
>> + * XXX: move fixmap definition in a separate header
>> + */
>> +#include <xen/acpi.h>
>> +
>> +/*
>> + * Simple mapping infrastructure to map / unmap pages in fixed map.
>> + * This is used to set up the page table for mapcache, which is used
>> + * by map domain page infrastructure.
>> + *
>> + * This structure is not protected by any locks, so it must not be
>> used after
>> + * smp bring-up.
>> + */
>> +
>> +/* Bitmap to track which slot is used */
>> +static unsigned long __initdata inuse;
>> +
>> +/* XXX: Find an header to declare it */
>> +extern lpae_t xen_fixmap[LPAE_ENTRIES];
>> +
>> +void *__init pmap_map(mfn_t mfn)
>> +{
>> +    unsigned long flags;
>> +    unsigned int idx;
>> +    vaddr_t linear;
>> +    unsigned int slot;
>> +    lpae_t *entry, pte;
>> +
>> +    BUILD_BUG_ON(sizeof(inuse) * BITS_PER_LONG < NUM_FIX_PMAP);
> 
> This seems wrong to me. It should multiply with something like
> BITS_PER_BYTE.

Good spot! I have updated my tree.

> 
> I noticed this line was already present before the Arm version so
> probably my fault :(, which also needs to be fixed.

This should be taken care as the next version will create the pmap in 
common code :).

> 
>> +
>> +    ASSERT(system_state < SYS_STATE_smp_boot);
>> +
>> +    local_irq_save(flags);
>> +
>> +    idx = find_first_zero_bit(&inuse, NUM_FIX_PMAP);
>> +    if ( idx == NUM_FIX_PMAP )
>> +        panic("Out of PMAP slots\n");
>> +
>> +    __set_bit(idx, &inuse);
>> +
>> +    slot = idx + FIXMAP_PMAP_BEGIN;
>> +    ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END);
>> +
> 
>  From here...
> 
>> +    linear = FIXMAP_ADDR(slot);
>> +    /*
>> +     * We cannot use set_fixmap() here. We use PMAP when there is no
>> direct map,
>> +     * so map_pages_to_xen() called by set_fixmap() needs to map
>> pages on
>> +     * demand, which then calls pmap() again, resulting in a loop.
>> Modify the
>> +     * PTEs directly instead. The same is true for pmap_unmap().
>> +     */
>> +    entry = &xen_fixmap[third_table_offset(linear)];
>> +
>> +    ASSERT(!lpae_is_valid(*entry));
>> +
>> +    pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW);
>> +    pte.pt.table = 1;
>> +    write_pte(entry, pte);
>> +
> 
> ...to here, I wonder if we can move this chunk into arch (like void
> *arch_write_pmap_slot(slot)). Such an arch function hides how fixmap is
> handled and how page table entry is written behind arch, and the rest
> can just be common.

This is similar to what I had in mind. Let me give a try for the next 
version.

> 
>> +    local_irq_restore(flags);
>> +
>> +    return (void *)linear;
>> +}
>> +
>> +void __init pmap_unmap(const void *p)
>> +{
>> +    unsigned long flags;
>> +    unsigned int idx;
>> +    lpae_t *entry;
>> +    lpae_t pte = { 0 };
>> +    unsigned int slot = third_table_offset((vaddr_t)p);
>> +
>> +    ASSERT(system_state < SYS_STATE_smp_boot);
>> +    ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END);
>> +
>> +    idx = slot - FIXMAP_PMAP_BEGIN;
>> +    local_irq_save(flags);
>> +
>> +    __clear_bit(idx, &inuse);
>> +    entry = &xen_fixmap[third_table_offset((vaddr_t)p)];
>> +    write_pte(entry, pte);
>> +    flush_xen_tlb_range_va_local((vaddr_t)p, PAGE_SIZE);
> 
> and the same for the above, something like arch_clear_pmap(void *) and
> the rest into common.
> 
>  From a quick glance, I don't think x86 and Arm share any useful TLB
> flush helpers? So the TLB flush probably should be behind arch as well.

We could potential define flush_tlb_one_local() on Arm. But, I am not 
sure this is worth it because the page table manipulation is mainly 
happening in arch code so far. Although, this might change in the future.

> 
>> +
>> +    local_irq_restore(flags);
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
> 
> [...]
> 
>> diff --git a/xen/include/asm-arm/pmap.h b/xen/include/asm-arm/pmap.h
>> new file mode 100644
>> index 000000000000..8e1dce93f8e4
>> --- /dev/null
>> +++ b/xen/include/asm-arm/pmap.h
>> @@ -0,0 +1,10 @@
>> +#ifndef __ASM_PMAP_H__
>> +#define __ARM_PMAP_H__
> 
> This line doesn't seem to match the #ifndef, but if the functions are
> moved to common, this header can be moved to common as well.

Stefano pointed out the same. I have fixed it in my tree but I will 
likely rename the guard as the header will be moved in common.

Cheers,

-- 
Julien Grall


  reply	other threads:[~2021-07-03 17:57 UTC|newest]

Thread overview: 59+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-25 20:13 [PATCH RFCv2 00/15] xen/arm: mm: Remove open-coding mappings Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 01/15] xen/arm: lpae: Rename LPAE_ENTRIES_MASK_GS to LPAE_ENTRY_MASK_GS Julien Grall
2021-05-11 22:12   ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 02/15] xen/arm: lpae: Use the generic helpers to defined the Xen PT helpers Julien Grall
2021-05-11 22:26   ` Stefano Stabellini
2021-05-12 14:26     ` Julien Grall
2021-05-12 21:30       ` Stefano Stabellini
2021-05-12 22:16         ` Julien Grall
2021-05-13 22:44           ` Stefano Stabellini
2021-07-03 17:32             ` Julien Grall
2021-07-13 20:53               ` Stefano Stabellini
2021-07-14 17:40                 ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 03/15] xen/arm: p2m: Replace level_{orders, masks} arrays with LEVEL_{ORDER, MASK} Julien Grall
2021-05-11 22:33   ` Stefano Stabellini
2021-05-12 14:39     ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 04/15] xen/arm: mm: Allow other mapping size in xen_pt_update_entry() Julien Grall
2021-05-11 22:42   ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 05/15] xen/arm: mm: Avoid flushing the TLBs when mapping are inserted Julien Grall
2021-05-11 22:51   ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 06/15] xen/arm: mm: Don't open-code Xen PT update in remove_early_mappings() Julien Grall
2021-05-11 22:58   ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 07/15] xen/arm: mm: Re-implement early_fdt_map() using map_pages_to_xen() Julien Grall
2021-05-12 21:41   ` Stefano Stabellini
2021-05-12 22:18     ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 08/15] xen/arm32: mm: Check if the virtual address is shared before updating it Julien Grall
2021-05-12 22:00   ` Stefano Stabellini
2021-05-12 22:23     ` Julien Grall
2021-05-13 22:32       ` Stefano Stabellini
2021-05-13 22:59         ` Julien Grall
2021-05-14  1:04           ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 09/15] xen/arm32: mm: Re-implement setup_xenheap_mappings() using map_pages_to_xen() Julien Grall
2021-05-12 22:07   ` Stefano Stabellini
2021-05-13 17:55     ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 10/15] xen/arm: mm: Allocate xen page tables in domheap rather than xenheap Julien Grall
2021-05-12 22:44   ` Stefano Stabellini
2021-05-13 18:09     ` Julien Grall
2021-05-13 22:27       ` Stefano Stabellini
2021-05-15  8:48         ` Julien Grall
2021-05-18  0:37           ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 11/15] xen/arm: mm: Allow page-table allocation from the boot allocator Julien Grall
2021-05-13 22:58   ` Stefano Stabellini
2021-04-25 20:13 ` [PATCH RFCv2 12/15] xen/arm: add Persistent Map (PMAP) infrastructure Julien Grall
2021-04-26  9:41   ` Xia, Hongyan
2021-07-03 17:57     ` Julien Grall [this message]
2021-04-28 21:47   ` Wei Liu
2021-05-14 23:25   ` Stefano Stabellini
2021-05-15  8:54     ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 13/15] xen/arm: mm: Use the PMAP helpers in xen_{,un}map_table() Julien Grall
2021-05-14 23:35   ` Stefano Stabellini
2021-05-15  9:03     ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 14/15] xen/arm: mm: Rework setup_xenheap_mappings() Julien Grall
2021-05-14 23:51   ` Stefano Stabellini
2021-05-15  9:21     ` Julien Grall
2021-05-18  0:50       ` Stefano Stabellini
2022-02-12 19:16         ` Julien Grall
2021-04-25 20:13 ` [PATCH RFCv2 15/15] xen/arm: mm: Re-implement setup_frame_table_mappings() with map_pages_to_xen() Julien Grall
2021-05-15  0:02   ` Stefano Stabellini
2021-05-15  9:25     ` Julien Grall
2021-05-18  0:51       ` Stefano Stabellini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d3bdec7f-837e-a055-4fe5-a5ceff0c5a51@xen.org \
    --to=julien@xen.org \
    --cc=Bertrand.Marquis@arm.com \
    --cc=Henry.Wang@arm.com \
    --cc=Penny.Zheng@arm.com \
    --cc=Volodymyr_Babchuk@epam.com \
    --cc=Wei.Chen@arm.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=hongyax@amazon.com \
    --cc=hongyxia@amazon.com \
    --cc=jbeulich@suse.com \
    --cc=jgrall@amazon.co.uk \
    --cc=roger.pau@citrix.com \
    --cc=sstabellini@kernel.org \
    --cc=wei.liu2@citrix.com \
    --cc=wl@xen.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).