From: Stefano Stabellini <sstabellini@kernel.org>
To: Julien Grall <julien.grall@arm.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
Wei Liu <wei.liu2@citrix.com>,
Steve Capper <Steve.Capper@arm.com>,
ian.jackson@eu.citrix.com, peter.huangpeng@huawei.com,
xen-devel@lists.xen.org, Shannon Zhao <shannon.zhao@linaro.org>,
Shannon Zhao <zhaoshenglong@huawei.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>
Subject: Re: [PATCH v2 16/17] libxc/xc_dom_arm: Copy ACPI tables to guest space
Date: Thu, 21 Jul 2016 11:54:51 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.10.1607211151480.6667@sstabellini-ThinkPad-X260> (raw)
In-Reply-To: <2ec4e8a6-28bd-6c7e-1751-a0af235eb636@arm.com>
On Thu, 21 Jul 2016, Julien Grall wrote:
> Hi Stefano,
>
> On 21/07/16 18:53, Stefano Stabellini wrote:
> > On Wed, 20 Jul 2016, Boris Ostrovsky wrote:
> > > On 07/20/2016 01:28 PM, Stefano Stabellini wrote:
> > > > On Wed, 20 Jul 2016, Boris Ostrovsky wrote:
> > > > > On 07/20/2016 09:41 AM, Julien Grall wrote:
> > > > > >
> > > > > > On 20/07/2016 14:33, Boris Ostrovsky wrote:
> > > > > > > On 07/20/2016 08:33 AM, Julien Grall wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > On 14/07/16 14:37, Stefano Stabellini wrote:
> > > > > > > > > On Wed, 13 Jul 2016, Julien Grall wrote:
> > > > > > > > > > Hello,
> > > > > > > > > >
> > > > > > > > > > On 12/07/2016 17:58, Boris Ostrovsky wrote:
> > > > > > > > > > > On 07/12/2016 12:10 PM, Julien Grall wrote:
> > > > > > > > > > > > On 12/07/2016 16:08, Boris Ostrovsky wrote:
> > > > > > > > > > > > > On 07/12/2016 10:57 AM, Shannon Zhao wrote:
> > > > > > > > > > > > It will affect some others part of the guest if we don't
> > > > > > > > > > > > increment
> > > > > > > > > > > > the
> > > > > > > > > > > > "maxmem" requested by the user. For ARM the ACPI blob
> > > > > > > > > > > > will be
> > > > > > > > > > > > exposed
> > > > > > > > > > > > at a specific address that is outside of the guest RAM
> > > > > > > > > > > > (see the
> > > > > > > > > > > > guest
> > > > > > > > > > > > memory layout in public/arch-arm.h).
> > > > > > > > > > > >
> > > > > > > > > > > > We chose this solution over putting in the RAM because
> > > > > > > > > > > > the ACPI
> > > > > > > > > > > > tables
> > > > > > > > > > > > are not easily relocatable (compare to the device tree,
> > > > > > > > > > > > initrd and
> > > > > > > > > > > > kernel) so we could not take advantage of superpage in
> > > > > > > > > > > > both stage-2
> > > > > > > > > > > > (hypervisor) and stage-1 (kernel) page table.
> > > > > > > > > > > Maybe this is something ARM-specific then. For x86 we will
> > > > > > > > > > > want to
> > > > > > > > > > > keep
> > > > > > > > > > > maxmem unchanged.
> > > > > > > > > > I don't think what I described in my previous mail is
> > > > > > > > > > ARM-specific. The
> > > > > > > > > > pressure will be more important on the TLBs, if Xen does not
> > > > > > > > > > use
> > > > > > > > > > superpage in
> > > > > > > > > > the stage 2 page tables (i.e EPT for x86) no matter the
> > > > > > > > > > architecture.
> > > > > > > > > >
> > > > > > > > > > IHMO, this seems to be a bigger drawback compare to add few
> > > > > > > > > > more
> > > > > > > > > > kilobytes to
> > > > > > > > > > maxmem in the toolstack for the ACPI blob. You will loose
> > > > > > > > > > them when
> > > > > > > > > > creating
> > > > > > > > > > the intermediate page table in any case.
> > > > > > > > > I agree with Julien. On ARM we have to increase maxmem because
> > > > > > > > > I don't
> > > > > > > > > think that shattering a superpage is acceptable for just a few
> > > > > > > > > KBs. In
> > > > > > > > > fact, it's not much about increasing maxmem, but it's about
> > > > > > > > > keeping
> > > > > > > > > the
> > > > > > > > > allocation of guest memory to the value passed by the user in
> > > > > > > > > "memory",
> > > > > > > > > so that it can be done in the most efficient way possible. (I
> > > > > > > > > am
> > > > > > > > > assuming users are going to allocate VMs of 2048MB, rather
> > > > > > > > > than
> > > > > > > > > 2049MB.)
> > > > > > > > >
> > > > > > > > > I wouldn't want to end up adding to the performance tuning
> > > > > > > > > page on the
> > > > > > > > > wiki "Make sure to add 1 more MB to the memory of your VM to
> > > > > > > > > get the
> > > > > > > > > most out of the system."
> > > > > > > > >
> > > > > > > > > I know that the location of the ACPI blob on x86 is different
> > > > > > > > > in guest
> > > > > > > > > memory space, but it seems to me that the problem would be the
> > > > > > > > > same. Do
> > > > > > > > > you have 1 gigabyte pages in stage-2 on x86? If so, I would
> > > > > > > > > think
> > > > > > > > > twice
> > > > > > > > > about this. Otherwise, if you only have 4K and 2MB
> > > > > > > > > allocations,
> > > > > > > > > then it
> > > > > > > > > might not make that much of a difference.
> > > > > > > > Looking at the x86 code, 1 gigabyte pages seems to be supported.
> > > > > > > >
> > > > > > > > Boris, do you have any opinions on this?
> > > > > > >
> > > > > > > I don't think I understand the superpage shattering argument. In
> > > > > > > x86
> > > > > > > the tables live in regular RAM and a guest is free to use those
> > > > > > > addresses as regular memory.
> > > > > > >
> > > > > > > This apparently is different from how ARM manages the tables (you
> > > > > > > said
> > > > > > > in an earlier message that they are not part of RAM) so I can see
> > > > > > > that
> > > > > > > taking memory from RAM allocation to store the tables may affect
> > > > > > > how
> > > > > > > mapping is done, potentially causing GB pages to be broken.
> > > > > > >
> > > > > > > In fact (and I am totally speculating here) padding memory for x86
> > > > > > > may
> > > > > > > actually *cause* shattering because we will have (for example)
> > > > > > > 2049MB of
> > > > > > > RAM to deal with.
> > > > > > Correct me if I am wrong. On your series you are populating the page
> > > > > > at a specific address for the ACPI tables separately to the RAM
> > > > > > allocation. So you will shatter GB pages if the user provides 2048MB
> > > > > > because the ACPI tables is accounted in the 2048MB.
> > > > > And to be honest I am not convinced this was a well selected address
> > > > > (0xfc000000). I am actually thinking about moving it down (this may
> > > > > require changing dsdt.asl). I don't know whether I will actually do it
> > > > > in this version but it is certainly a possibility.
> > > > I don't understand how this statement fits in the discussion.
> > > >
> > > > The memory allocation for the ACPI blob is done by the toolstack
> > > > separately from the rest of guest memory, leading to two separate
> > > > stage-2 pagetable allocations of less than 1-gigabyte pages. Is that
> > > > correct?
> > >
> > >
> > > If I move the table lower into memory we won't have to do any extra
> > > allocation. The memory will have been already allocated for the guest,
> > > we just map it and copy the tables.
> >
> > I see, thanks for the explanation. I think this could work for ARM too
> > and should avoid the stage-2 shattering issue described above.
>
> But you will end up to have stage-1 shattering issue if you put the ACPI
> tables lower into the guest RAM, reducing the overall performance. It is why I
> first asked Shannon to put the ACPI outside of the guest RAM.
I am not sure about this actually: even with the ACPI blob in the middle
of guest memory, the guest OS could still use a single superpage for its
own stage-1 memory mappings. I don't know if Linux does it that way, but
it should be possible.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-07-21 18:54 UTC|newest]
Thread overview: 130+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-23 3:16 [PATCH v2 00/17] Xen ARM DomU ACPI support Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 01/17] libxl/arm: Factor out codes for generating DTB Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 02/17] libxc: Add placeholders for ACPI tables blob and size Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 03/17] libxl/arm: Add a configuration option for ARM DomU ACPI Shannon Zhao
2016-06-23 13:39 ` Stefano Stabellini
2016-06-23 14:34 ` Shannon Zhao
2016-06-27 10:40 ` Julien Grall
2016-07-07 15:30 ` Wei Liu
2016-07-12 3:40 ` Shannon Zhao
2016-07-12 9:22 ` Julien Grall
2016-07-12 11:33 ` Wei Liu
2016-07-12 14:17 ` Shannon Zhao
2016-07-12 14:33 ` Wei Liu
2016-07-12 14:45 ` Shannon Zhao
2016-07-13 7:54 ` Shannon Zhao
2016-07-13 9:20 ` Julien Grall
2016-07-13 9:48 ` Shannon Zhao
2016-07-13 10:03 ` Julien Grall
2016-07-15 8:00 ` Shannon Zhao
2016-07-15 8:07 ` Shannon Zhao
2016-07-18 19:40 ` Stefano Stabellini
2016-07-19 10:40 ` Wei Liu
2016-07-19 10:44 ` Ian Jackson
2016-06-23 15:53 ` Julien Grall
2016-06-23 3:16 ` [PATCH v2 04/17] libxl/arm: prepare for constructing ACPI tables Shannon Zhao
2016-06-23 13:37 ` Stefano Stabellini
2016-06-23 14:23 ` Shannon Zhao
2016-06-23 14:27 ` Stefano Stabellini
2016-06-23 16:18 ` Julien Grall
2016-06-23 3:16 ` [PATCH v2 05/17] libxl/arm: Construct ACPI RSDP table Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 06/17] libxl/arm: Construct ACPI XSDT table Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 07/17] libxl/arm: Construct ACPI GTDT table Shannon Zhao
2016-06-23 15:00 ` Stefano Stabellini
2016-06-23 16:19 ` Julien Grall
2016-06-23 16:26 ` Julien Grall
2016-06-23 16:58 ` Julien Grall
2016-06-27 1:44 ` Shannon Zhao
2016-06-27 10:17 ` Julien Grall
2016-06-29 9:29 ` Shannon Zhao
2016-06-29 9:42 ` Julien Grall
2016-06-29 13:41 ` Shannon Zhao
2016-07-05 16:42 ` Stefano Stabellini
2016-07-06 9:50 ` Julien Grall
2016-07-06 10:16 ` Stefano Stabellini
2016-06-23 3:16 ` [PATCH v2 08/17] libxl/arm: Factor MPIDR computing codes out as a helper Shannon Zhao
2016-06-23 16:29 ` Julien Grall
2016-06-23 3:16 ` [PATCH v2 09/17] libxl/arm: Construct ACPI MADT table Shannon Zhao
2016-06-23 16:36 ` Julien Grall
2016-06-23 3:16 ` [PATCH v2 10/17] libxl/arm: Construct ACPI FADT table Shannon Zhao
2016-06-23 3:16 ` [PATCH v2 11/17] libxl/arm: Construct ACPI DSDT table Shannon Zhao
2016-06-23 14:50 ` Stefano Stabellini
2016-06-23 16:42 ` Julien Grall
2016-06-27 1:50 ` Shannon Zhao
2016-06-23 17:03 ` Julien Grall
2016-06-27 6:01 ` Shannon Zhao
2016-06-27 10:29 ` Julien Grall
2016-06-27 12:05 ` Boris Ostrovsky
2016-06-28 11:03 ` Shannon Zhao
2016-06-28 13:41 ` Boris Ostrovsky
2016-06-29 18:58 ` Boris Ostrovsky
2016-07-01 7:58 ` Shannon Zhao
2016-07-01 10:18 ` Julien Grall
2016-07-01 14:42 ` Boris Ostrovsky
2016-07-01 15:14 ` Julien Grall
2016-06-23 3:16 ` [PATCH v2 12/17] libxl/arm: Add a helper to calculate the ACPI table checksum Shannon Zhao
2016-06-23 17:05 ` Julien Grall
2016-06-23 3:17 ` [PATCH v2 13/17] libxl/arm: Link all ACPI tables into one buffer Shannon Zhao
2016-06-23 17:10 ` Julien Grall
2016-06-23 3:17 ` [PATCH v2 14/17] libxl/arm: Factor finalise_one_memory_node as a gerneric function Shannon Zhao
2016-06-23 3:17 ` [PATCH v2 15/17] libxl/arm: Add ACPI module Shannon Zhao
2016-06-23 18:35 ` Julien Grall
2016-06-25 3:22 ` Shannon Zhao
2016-06-27 9:48 ` Julien Grall
2016-06-23 3:17 ` [PATCH v2 16/17] libxc/xc_dom_arm: Copy ACPI tables to guest space Shannon Zhao
2016-06-23 18:46 ` Julien Grall
2016-06-27 6:25 ` Shannon Zhao
2016-06-27 10:49 ` Julien Grall
2016-06-27 12:11 ` Boris Ostrovsky
2016-07-05 17:13 ` Stefano Stabellini
2016-07-06 9:46 ` Julien Grall
2016-07-06 10:12 ` Stefano Stabellini
2016-07-12 3:47 ` Shannon Zhao
2016-07-12 9:25 ` Julien Grall
2016-07-12 11:35 ` Wei Liu
2016-07-12 14:42 ` Shannon Zhao
2016-07-12 14:50 ` Wei Liu
2016-07-12 14:57 ` Shannon Zhao
2016-07-12 15:08 ` Boris Ostrovsky
2016-07-12 15:13 ` Wei Liu
2016-07-12 15:21 ` Boris Ostrovsky
2016-07-12 16:05 ` Wei Liu
2016-07-12 16:10 ` Julien Grall
2016-07-12 16:58 ` Boris Ostrovsky
2016-07-13 15:22 ` Julien Grall
2016-07-13 17:08 ` Boris Ostrovsky
2016-07-14 11:15 ` Wei Liu
2016-07-15 9:39 ` Shannon Zhao
2016-07-19 10:38 ` Wei Liu
2016-07-20 6:52 ` Shannon Zhao
2016-07-20 9:32 ` Wei Liu
2016-07-25 7:56 ` Shannon Zhao
2016-07-28 11:10 ` Julien Grall
2016-07-14 11:29 ` Julien Grall
2016-07-14 13:37 ` Stefano Stabellini
2016-07-20 12:33 ` Julien Grall
2016-07-20 13:33 ` Boris Ostrovsky
2016-07-20 13:41 ` Julien Grall
2016-07-20 14:09 ` Boris Ostrovsky
2016-07-20 17:28 ` Stefano Stabellini
2016-07-20 19:51 ` Boris Ostrovsky
2016-07-21 17:53 ` Stefano Stabellini
2016-07-21 18:23 ` Julien Grall
2016-07-21 18:54 ` Stefano Stabellini [this message]
2016-07-21 19:14 ` Julien Grall
2016-07-21 21:15 ` Stefano Stabellini
2016-07-25 8:38 ` George Dunlap
2016-07-25 9:46 ` Julien Grall
2016-07-25 22:06 ` Stefano Stabellini
2016-07-25 22:46 ` Boris Ostrovsky
2016-07-25 23:40 ` Stefano Stabellini
2016-07-26 1:17 ` Boris Ostrovsky
2016-07-28 11:06 ` Julien Grall
2016-07-28 12:42 ` Shannon Zhao
2016-08-02 11:01 ` Wei Liu
2016-08-03 19:20 ` Julien Grall
2016-08-04 10:17 ` Wei Liu
2016-08-02 11:01 ` Wei Liu
2016-07-07 15:35 ` Wei Liu
2016-06-23 3:17 ` [PATCH v2 17/17] libxl/arm: Initialize domain param HVM_PARAM_CALLBACK_IRQ Shannon Zhao
2016-06-23 18:48 ` Julien Grall
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.10.1607211151480.6667@sstabellini-ThinkPad-X260 \
--to=sstabellini@kernel.org \
--cc=Steve.Capper@arm.com \
--cc=boris.ostrovsky@oracle.com \
--cc=ian.jackson@eu.citrix.com \
--cc=julien.grall@arm.com \
--cc=peter.huangpeng@huawei.com \
--cc=shannon.zhao@linaro.org \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
--cc=zhaoshenglong@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).