From: Julien Grall <Julien.Grall@arm.com> To: Stefano Stabellini <sstabellini@kernel.org> Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, nd <nd@arm.com>, Stefano Stabellini <stefanos@xilinx.com> Subject: Re: [PATCH v2 10/10] xen/arm: add reserved-memory regions to the dom0 memory node Date: Mon, 20 May 2019 22:38:32 +0000 [thread overview] Message-ID: <c94041e2-8f85-6fd6-21cd-3dd082882854@arm.com> (raw) In-Reply-To: <alpine.DEB.2.21.1905201407410.16404@sstabellini-ThinkPad-T480s> Hi Stefano, On 20/05/2019 22:26, Stefano Stabellini wrote: > On Sat, 11 May 2019, Julien Grall wrote: >>>>> But I am still not happy with the approach taken for the reserved-memory >>>>> regions in this series. As I pointed out before, they are just normal >>>>> memory >>>>> that was reserved for other purpose (CMA, framebuffer...). >>>>> >>>>> Treating them as "device" from Xen POV is a clear abuse of the meaning >>>>> and I >>>>> don't believe it is a viable solution long term. >>>> >>>> If we don't consider "reusable" memory regions as part of the >>>> discussion, the distinction becomes more philosophical than practical: >>>> >>>> - Xen is not supposed to use them for anything >>>> - only given them to the VM configured for it >>>> >>>> I don't see much of a difference with MMIO regions, except for the >>>> expected pagetable attributes: i.e. cacheable, not-cacheable. But even >>>> in that case, there could be reasonable use cases for non-cacheable >>>> mappings of reserved-memory regions, even if reserved-memory regions are >>>> "normal" memory. >>>> >>>> Could you please help me understand why you see them so differently, as >>>> far as to say that "treating them as "device" from Xen POV is a clear >>>> abuse of the meaning"? >>> >>> Obviously if you take half of the picture, then it makes things easier. >>> However, we are not here to discuss half of the picture but the full one >>> (even if at the end you only implement half of it). >>> >>>>> Indeed, some of the regions may have a property "reusable" allowing the >>>>> the OS >>>>> to use them until they are claimed by the device driver owning the >>>>> region. I >>>>> don't know how Linux (or any other OS) is using it today, but I don't >>>>> see what >>>>> would prevent it to use them as hypercall buffer. This would obviously >>>>> not >>>>> work because they are not actual RAM from Xen POV. >>>> >>>> I haven't attempted at handling "reusable" reserved-memory regions >>>> because I don't have a test environment and/or a use-case for them. In >>>> other words, I don't have any "reusable" reserved-memory regions in any >>>> of the boards (Xilinx and not Xilinx) I have access to. I could add a >>>> warning if we find a "reusable" reserved-memory region at boot. >>> >>> Don't get me wrong, I don't ask for the implementation now, so a warning >>> would be fine here. However, you need at least to show me some ground that >>> re-usable memory can be implemented with your solution or they are not a >>> concern for Xen at all. >>> >>>> >>>> Nonetheless, if you have a concrete suggestion which doesn't require a >>>> complete rework of this series, I can try to put extra effort to handle >>>> this case even if it is not a benefit to my employer. I am also open to >>>> the possibility of dropping patches 6-10 from the series. >>> I don't think the series as it is would allow us to support re-usable >>> memory. However as I haven't spent enough time to understand how this could >>> be possibly dealt. So I am happy to be proved wrong. >> >> I thought a bit more about this series during the night. I do agree that we >> need to improve the support of the reserved-memory today as we may give memory >> to the allocator that are could be exposed to a guest via a different method >> (iomem). So carving out the reserved-memory region from the memory allocator >> is the first step to go. >> >> Now we have to differentiate the hardware domain from the other guests. I >> don't have any objection regarding the way to map reserved-memory region to >> the hardware domain because this is completely internal to Xen. However, I >> have some objections with the current interface for DomU: >> 1) It is still unclear how "reusable" property would fit in that story >> 2) It is definitely not possible for a user to use 'iomem' for >> reserved-memory region today because the partial Device-Tree doesn't allow you >> to create /reserved-memory node nor /memory >> 3) AFAIK, there are no way for to prevent the hardware domain to use the >> reserved-region (status = "disabled" would not work). >> So, IHMO, the guest support for reserved-memory is not in shape. So I think it >> would be best if we don't permit the reserved-memory region in the iomem >> rangeset. This would avoid us to tie us in an interface until we figure out >> the correct plan for guest. > > Wouldn't be proper documentation be enough? (See below for where the > documentation should live.) > > This is not about privilege over the system: whoever will make the > decision to ask the hypervisor to map the page will have all the > necessary rights to do it. If the user wants to map a given region, > either because she knows what she is doing, because she is > experimenting, or for whatever reason, I think she should be allowed. In > fact, she can always do it by reverting the patch. So why make it > inconvenient for her? TBH, I am getting very frustrated on reviewing this series. We spent our previous f2f meetings discussing reserved-memory in lengthy way. We also agreed on a plan (see below), but now we are back on square one again... Yes, a user will need to revert the patch. But then as you said the user would know what he/she is doing. So reverting a patch is not going to be a complication. However, I already pointed out multiple time that giving permission is not going to be enough. So I still don't see the value of having that in Xen without an easy way to use it. For reminder, you agreed on the following splitting the series in 3 parts: - Part 1: Extend iomem to support cacheability - Part 2: Partially support reserved-memory for Dom0 and don't give iomem permission on them - Part 3: reserved-memory for guest I agreed to merge part 1 and 2. Part 3 will be a start for a discussion how this should be supported for guest. I also pointed out that Xilinx can carry part 3 in their tree if they feel like too. > > >> With that in place, I don't have a strong objection with patches 6-10. >> >> In any case I think you should clearly spell out in the commit message what >> kind of reserved-memory region is supported. > > Yes, this makes sense. I am thinking of adding a note to SUPPORT.md. Any > other places where I should write it down aside from commit messages? > > >> For instance, by just going through the binding, I have the feeling >> that those properties are not actually supported: >> 1) "no-map" - It is used to tell the OS to not create a virtual memory of >> the region as part of its standard mapping of system memory, nor permit >> speculative access to it under any circumstances other than under the control >> of the device driver using the region. On Arm64, Xen will map reserved-memory >> as part of xenheap (i.e the direct mapping), but carving out from xenheap >> would not be sufficient as we use 1GB block for the mapping. So they may still >> be covered. I would assume this is used for memory that needs to be mapped >> non-cacheable, so it is potentially critical as Xen would map them cacheable >> in the stage-1 hypervisor page-tables. >> 2) "alloc-ranges": it is used to specify regions of memory where it is >> acceptable to allocate memory from. This may not play well with the Dom0 >> memory allocator. >> 3) "reusable": I mention here only for completeness. My understanding is >> it could potentially be used for hypercall buffer. This needs to be >> investigated. > > Yes, you are right about these properties not being properly supported. > Do you think that I should list them in SUPPORT.md under a new iomem > section? Or do you prefer a longer document under docs/? Or both? The properties have nothing to do with iomem. So it would be clearly the wrong place to put under. Instead this should be a separate sections. Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
WARNING: multiple messages have this Message-ID (diff)
From: Julien Grall <Julien.Grall@arm.com> To: Stefano Stabellini <sstabellini@kernel.org> Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, nd <nd@arm.com>, Stefano Stabellini <stefanos@xilinx.com> Subject: Re: [Xen-devel] [PATCH v2 10/10] xen/arm: add reserved-memory regions to the dom0 memory node Date: Mon, 20 May 2019 22:38:32 +0000 [thread overview] Message-ID: <c94041e2-8f85-6fd6-21cd-3dd082882854@arm.com> (raw) Message-ID: <20190520223832.NluKyXzqPmqRegaxjOA9v7Ppu4AjCVjBheg1CeRXL0c@z> (raw) In-Reply-To: <alpine.DEB.2.21.1905201407410.16404@sstabellini-ThinkPad-T480s> Hi Stefano, On 20/05/2019 22:26, Stefano Stabellini wrote: > On Sat, 11 May 2019, Julien Grall wrote: >>>>> But I am still not happy with the approach taken for the reserved-memory >>>>> regions in this series. As I pointed out before, they are just normal >>>>> memory >>>>> that was reserved for other purpose (CMA, framebuffer...). >>>>> >>>>> Treating them as "device" from Xen POV is a clear abuse of the meaning >>>>> and I >>>>> don't believe it is a viable solution long term. >>>> >>>> If we don't consider "reusable" memory regions as part of the >>>> discussion, the distinction becomes more philosophical than practical: >>>> >>>> - Xen is not supposed to use them for anything >>>> - only given them to the VM configured for it >>>> >>>> I don't see much of a difference with MMIO regions, except for the >>>> expected pagetable attributes: i.e. cacheable, not-cacheable. But even >>>> in that case, there could be reasonable use cases for non-cacheable >>>> mappings of reserved-memory regions, even if reserved-memory regions are >>>> "normal" memory. >>>> >>>> Could you please help me understand why you see them so differently, as >>>> far as to say that "treating them as "device" from Xen POV is a clear >>>> abuse of the meaning"? >>> >>> Obviously if you take half of the picture, then it makes things easier. >>> However, we are not here to discuss half of the picture but the full one >>> (even if at the end you only implement half of it). >>> >>>>> Indeed, some of the regions may have a property "reusable" allowing the >>>>> the OS >>>>> to use them until they are claimed by the device driver owning the >>>>> region. I >>>>> don't know how Linux (or any other OS) is using it today, but I don't >>>>> see what >>>>> would prevent it to use them as hypercall buffer. This would obviously >>>>> not >>>>> work because they are not actual RAM from Xen POV. >>>> >>>> I haven't attempted at handling "reusable" reserved-memory regions >>>> because I don't have a test environment and/or a use-case for them. In >>>> other words, I don't have any "reusable" reserved-memory regions in any >>>> of the boards (Xilinx and not Xilinx) I have access to. I could add a >>>> warning if we find a "reusable" reserved-memory region at boot. >>> >>> Don't get me wrong, I don't ask for the implementation now, so a warning >>> would be fine here. However, you need at least to show me some ground that >>> re-usable memory can be implemented with your solution or they are not a >>> concern for Xen at all. >>> >>>> >>>> Nonetheless, if you have a concrete suggestion which doesn't require a >>>> complete rework of this series, I can try to put extra effort to handle >>>> this case even if it is not a benefit to my employer. I am also open to >>>> the possibility of dropping patches 6-10 from the series. >>> I don't think the series as it is would allow us to support re-usable >>> memory. However as I haven't spent enough time to understand how this could >>> be possibly dealt. So I am happy to be proved wrong. >> >> I thought a bit more about this series during the night. I do agree that we >> need to improve the support of the reserved-memory today as we may give memory >> to the allocator that are could be exposed to a guest via a different method >> (iomem). So carving out the reserved-memory region from the memory allocator >> is the first step to go. >> >> Now we have to differentiate the hardware domain from the other guests. I >> don't have any objection regarding the way to map reserved-memory region to >> the hardware domain because this is completely internal to Xen. However, I >> have some objections with the current interface for DomU: >> 1) It is still unclear how "reusable" property would fit in that story >> 2) It is definitely not possible for a user to use 'iomem' for >> reserved-memory region today because the partial Device-Tree doesn't allow you >> to create /reserved-memory node nor /memory >> 3) AFAIK, there are no way for to prevent the hardware domain to use the >> reserved-region (status = "disabled" would not work). >> So, IHMO, the guest support for reserved-memory is not in shape. So I think it >> would be best if we don't permit the reserved-memory region in the iomem >> rangeset. This would avoid us to tie us in an interface until we figure out >> the correct plan for guest. > > Wouldn't be proper documentation be enough? (See below for where the > documentation should live.) > > This is not about privilege over the system: whoever will make the > decision to ask the hypervisor to map the page will have all the > necessary rights to do it. If the user wants to map a given region, > either because she knows what she is doing, because she is > experimenting, or for whatever reason, I think she should be allowed. In > fact, she can always do it by reverting the patch. So why make it > inconvenient for her? TBH, I am getting very frustrated on reviewing this series. We spent our previous f2f meetings discussing reserved-memory in lengthy way. We also agreed on a plan (see below), but now we are back on square one again... Yes, a user will need to revert the patch. But then as you said the user would know what he/she is doing. So reverting a patch is not going to be a complication. However, I already pointed out multiple time that giving permission is not going to be enough. So I still don't see the value of having that in Xen without an easy way to use it. For reminder, you agreed on the following splitting the series in 3 parts: - Part 1: Extend iomem to support cacheability - Part 2: Partially support reserved-memory for Dom0 and don't give iomem permission on them - Part 3: reserved-memory for guest I agreed to merge part 1 and 2. Part 3 will be a start for a discussion how this should be supported for guest. I also pointed out that Xilinx can carry part 3 in their tree if they feel like too. > > >> With that in place, I don't have a strong objection with patches 6-10. >> >> In any case I think you should clearly spell out in the commit message what >> kind of reserved-memory region is supported. > > Yes, this makes sense. I am thinking of adding a note to SUPPORT.md. Any > other places where I should write it down aside from commit messages? > > >> For instance, by just going through the binding, I have the feeling >> that those properties are not actually supported: >> 1) "no-map" - It is used to tell the OS to not create a virtual memory of >> the region as part of its standard mapping of system memory, nor permit >> speculative access to it under any circumstances other than under the control >> of the device driver using the region. On Arm64, Xen will map reserved-memory >> as part of xenheap (i.e the direct mapping), but carving out from xenheap >> would not be sufficient as we use 1GB block for the mapping. So they may still >> be covered. I would assume this is used for memory that needs to be mapped >> non-cacheable, so it is potentially critical as Xen would map them cacheable >> in the stage-1 hypervisor page-tables. >> 2) "alloc-ranges": it is used to specify regions of memory where it is >> acceptable to allocate memory from. This may not play well with the Dom0 >> memory allocator. >> 3) "reusable": I mention here only for completeness. My understanding is >> it could potentially be used for hypercall buffer. This needs to be >> investigated. > > Yes, you are right about these properties not being properly supported. > Do you think that I should list them in SUPPORT.md under a new iomem > section? Or do you prefer a longer document under docs/? Or both? The properties have nothing to do with iomem. So it would be clearly the wrong place to put under. Instead this should be a separate sections. Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-05-20 22:38 UTC|newest] Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-30 21:02 [PATCH v2 0/10] iomem memory policy Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-04-30 21:02 ` [PATCH v2 01/10] xen: add a p2mt parameter to map_mmio_regions Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-02 14:59 ` Jan Beulich 2019-05-02 14:59 ` [Xen-devel] " Jan Beulich 2019-05-02 18:49 ` Stefano Stabellini 2019-05-02 18:49 ` [Xen-devel] " Stefano Stabellini 2019-05-15 13:39 ` Oleksandr 2019-05-15 13:39 ` [Xen-devel] " Oleksandr 2019-04-30 21:02 ` [PATCH v2 02/10] xen: rename un/map_mmio_regions to un/map_regions Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-01 9:22 ` Julien Grall 2019-05-01 9:22 ` [Xen-devel] " Julien Grall 2019-06-17 21:24 ` Stefano Stabellini 2019-06-18 11:05 ` Julien Grall 2019-06-18 20:19 ` Stefano Stabellini 2019-05-02 15:03 ` Jan Beulich 2019-05-02 15:03 ` [Xen-devel] " Jan Beulich 2019-05-02 18:55 ` Stefano Stabellini 2019-05-02 18:55 ` [Xen-devel] " Stefano Stabellini 2019-04-30 21:02 ` [PATCH v2 03/10] xen: extend XEN_DOMCTL_memory_mapping to handle memory policy Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-02 15:12 ` Jan Beulich 2019-05-02 15:12 ` [Xen-devel] " Jan Beulich 2019-06-17 21:28 ` Stefano Stabellini 2019-06-18 8:59 ` Jan Beulich 2019-06-18 20:32 ` Stefano Stabellini 2019-06-18 23:15 ` Stefano Stabellini 2019-06-19 6:53 ` Jan Beulich 2019-05-07 16:41 ` Julien Grall 2019-05-07 16:41 ` [Xen-devel] " Julien Grall 2019-06-17 22:43 ` Stefano Stabellini 2019-06-18 11:13 ` Julien Grall 2019-05-15 14:40 ` Oleksandr 2019-05-15 14:40 ` [Xen-devel] " Oleksandr 2019-04-30 21:02 ` [PATCH v2 04/10] libxc: introduce xc_domain_mem_map_policy Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-04-30 21:02 ` [PATCH v2 05/10] libxl/xl: add memory policy option to iomem Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-01 9:42 ` Julien Grall 2019-05-01 9:42 ` [Xen-devel] " Julien Grall 2019-06-17 22:32 ` Stefano Stabellini 2019-06-18 11:09 ` Julien Grall 2019-06-18 11:15 ` Julien Grall 2019-06-18 22:07 ` Stefano Stabellini 2019-06-18 22:20 ` Julien Grall 2019-06-18 22:46 ` Stefano Stabellini 2019-04-30 21:02 ` [PATCH v2 06/10] xen/arm: extend device_tree_for_each_node Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-07 17:12 ` Julien Grall 2019-05-07 17:12 ` [Xen-devel] " Julien Grall 2019-04-30 21:02 ` [PATCH v2 07/10] xen/arm: make process_memory_node a device_tree_node_func Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-01 9:47 ` Julien Grall 2019-05-01 9:47 ` [Xen-devel] " Julien Grall 2019-04-30 21:02 ` [PATCH v2 08/10] xen/arm: keep track of reserved-memory regions Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-01 10:03 ` Julien Grall 2019-05-01 10:03 ` [Xen-devel] " Julien Grall 2019-06-21 23:47 ` Stefano Stabellini 2019-05-07 17:21 ` Julien Grall 2019-05-07 17:21 ` [Xen-devel] " Julien Grall 2019-04-30 21:02 ` [PATCH v2 09/10] xen/arm: map reserved-memory regions as normal memory in dom0 Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-07 19:52 ` Julien Grall 2019-05-07 19:52 ` [Xen-devel] " Julien Grall 2019-04-30 21:02 ` [PATCH v2 10/10] xen/arm: add reserved-memory regions to the dom0 memory node Stefano Stabellini 2019-04-30 21:02 ` [Xen-devel] " Stefano Stabellini 2019-05-07 20:15 ` Julien Grall 2019-05-07 20:15 ` [Xen-devel] " Julien Grall 2019-05-10 20:51 ` Stefano Stabellini 2019-05-10 20:51 ` [Xen-devel] " Stefano Stabellini 2019-05-10 21:43 ` Julien Grall 2019-05-10 21:43 ` [Xen-devel] " Julien Grall 2019-05-11 12:40 ` Julien Grall 2019-05-11 12:40 ` [Xen-devel] " Julien Grall 2019-05-20 21:26 ` Stefano Stabellini 2019-05-20 21:26 ` [Xen-devel] " Stefano Stabellini 2019-05-20 22:38 ` Julien Grall [this message] 2019-05-20 22:38 ` Julien Grall 2019-06-05 16:30 ` Julien Grall 2019-06-21 23:47 ` Stefano Stabellini 2019-05-16 16:52 ` [PATCH v2 0/10] iomem memory policy Oleksandr 2019-05-16 16:52 ` [Xen-devel] " Oleksandr 2019-06-21 23:48 ` Stefano Stabellini
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=c94041e2-8f85-6fd6-21cd-3dd082882854@arm.com \ --to=julien.grall@arm.com \ --cc=nd@arm.com \ --cc=sstabellini@kernel.org \ --cc=stefanos@xilinx.com \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).