From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>,
"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
"sstabellini@kernel.org" <sstabellini@kernel.org>,
"julien@xen.org" <julien@xen.org>
Subject: RE: [PATCH 11/37] xen/x86: abstract neutral code from acpi_numa_memory_affinity_init
Date: Wed, 26 Jan 2022 10:39:14 +0000 [thread overview]
Message-ID: <PAXPR08MB7420F9BF695574EBD2AE57029E209@PAXPR08MB7420.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <46759708-36a8-fcde-d0dc-f73401491cbc@suse.com>
Hi Jan,
> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 2022年1月25日 0:51
> To: Wei Chen <Wei.Chen@arm.com>
> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>; xen-
> devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.org
> Subject: Re: [PATCH 11/37] xen/x86: abstract neutral code from
> acpi_numa_memory_affinity_init
>
> On 23.09.2021 14:02, Wei Chen wrote:
> > There is some code in acpi_numa_memory_affinity_init to update node
> > memory range and update node_memblk_range array. This code is not
> > ACPI specific, it can be shared by other NUMA implementation, like
> > device tree based NUMA implementation.
> >
> > So in this patch, we abstract this memory range and blocks relative
> > code to a new function. This will avoid exporting static variables
> > like node_memblk_range. And the PXM in neutral code print messages
> > have been replaced by NODE, as PXM is ACPI specific.
> >
> > Signed-off-by: Wei Chen <wei.chen@arm.com>
>
> SRAT is an ACPI concept, which I assume has no meaning with DT. Hence
> any generically usable logic here wants, I think, separating out into
> a file which is not SRAT-specific (peeking ahead, specifically not a
> file named "numa_srat.c"). This may in turn require some more though
When I created the file, I wanted to place non-ACPI/DT specific code in
a new file. But I was confused about how to name it. I chose numa_srat.c
as the file name because I thought the device tree is also a static
resource table. But it seems this name is still misleading, because
ACPI SRAT is well known.
> regarding the proper split between the stuff remaining in srat.c and
> the stuff becoming kind of library code. In particular this may mean
> moving some of the static variables as well, and with them perhaps
> some further functions (while I did peek ahead, I didn't look closely
> at the later patch doing the actual movement). And it is then hard to
> see why the separation needs to happen in two steps - you could move
> the generically usable code to a new file right away.
>
OK, I will reduce the steps. And I think the "new file" can be common/numa.c.
Because the generically usable code are some logical functions to check numa
memory blocks/ranges and update nodes, we don't need a "numa_srat.c".
> > --- a/xen/arch/x86/srat.c
> > +++ b/xen/arch/x86/srat.c
> > @@ -104,6 +104,14 @@ nodeid_t setup_node(unsigned pxm)
> > return node;
> > }
> >
> > +bool __init numa_memblks_available(void)
> > +{
> > + if (num_node_memblks < NR_NODE_MEMBLKS)
> > + return true;
> > +
> > + return false;
> > +}
>
> Please can you avoid expressing things in more complex than necessary
> ways? Here I don't see why it can't just be
OK, I will simplify it.
>
> bool __init numa_memblks_available(void)
> {
> return num_node_memblks < NR_NODE_MEMBLKS;
> }
>
> > @@ -301,69 +309,35 @@ static bool __init
> is_node_memory_continuous(nodeid_t nid,
> > return true;
> > }
> >
> > -/* Callback for parsing of the Proximity Domain <-> Memory Area
> mappings */
> > -void __init
> > -acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> > +/* Neutral NUMA memory affinity init function for ACPI and DT */
> > +int __init numa_update_node_memblks(nodeid_t node,
> > + paddr_t start, paddr_t size, bool hotplug)
>
> Indentation.
OK.
>
> > {
> > - paddr_t start, end;
> > - unsigned pxm;
> > - nodeid_t node;
> > + paddr_t end = start + size;
> > int i;
> >
> > - if (srat_disabled())
> > - return;
> > - if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
> > - bad_srat();
> > - return;
> > - }
> > - if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
> > - return;
> > -
> > - start = ma->base_address;
> > - end = start + ma->length;
> > - /* Supplement the heuristics in l1tf_calculations(). */
> > - l1tf_safe_maddr = max(l1tf_safe_maddr, ROUNDUP(end, PAGE_SIZE));
> > -
> > - if (num_node_memblks >= NR_NODE_MEMBLKS)
> > - {
> > - dprintk(XENLOG_WARNING,
> > - "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
> > - bad_srat();
> > - return;
> > - }
> > -
> > - pxm = ma->proximity_domain;
> > - if (srat_rev < 2)
> > - pxm &= 0xff;
> > - node = setup_node(pxm);
> > - if (node == NUMA_NO_NODE) {
> > - bad_srat();
> > - return;
> > - }
> > - /* It is fine to add this area to the nodes data it will be used
> later*/
> > + /* It is fine to add this area to the nodes data it will be used
> later */
> > i = conflicting_memblks(start, end);
> > if (i < 0)
> > /* everything fine */;
> > else if (memblk_nodeid[i] == node) {
> > - bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
> > - !test_bit(i, memblk_hotplug);
> > + bool mismatch = !hotplug != !test_bit(i, memblk_hotplug);
> >
> > - printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with
> itself (%"PRIpaddr"-%"PRIpaddr")\n",
> > - mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
> > + printk("%sSRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps
> with itself (%"PRIpaddr"-%"PRIpaddr")\n",
>
> Nit: Unlike PXM, which is an acronym, "node" doesn't want to be all upper
> case.
>
OK.
> Also did you check that the node <-> PXM association is known to a reader
> of a log at this point in time?
>
Yes, I read your comment below. The original log contains node <-> PXM mapping.
Because PXM is ACPI specific, I think in neutral code, we just need node in
log. But this change removed the log of node <-> PXM association, it was my
mistake. I will add them back in next version. But I don't think they will stay in
neutal code, it would be in ACPI specific code.
I also had wanted to keep PXM <-> node mapping in the log, so I set up PXM to
node 1-1 mapping for the device tree. But then I thought it was an unnecessary
burden on the device tree, so I selected to remove PXM in log.
> > + mismatch ? KERN_ERR : KERN_WARNING, node, start, end,
> > node_memblk_range[i].start, node_memblk_range[i].end);
> > if (mismatch) {
> > - bad_srat();
> > - return;
> > + return -1;
> > }
> > } else {
> > printk(KERN_ERR
> > - "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with
> PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
> > - pxm, start, end, node_to_pxm(memblk_nodeid[i]),
> > + "SRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with
> NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
> > + node, start, end, memblk_nodeid[i],
> > node_memblk_range[i].start, node_memblk_range[i].end);
> > - bad_srat();
> > - return;
> > + return -1;
>
> Please no -1 return values. Either a function means to return boolean,
> in which case it should use bool / true / false, or it means to return
> a proper errno-style error code.
>
Ok, I will do it.
> > @@ -375,26 +349,69 @@ acpi_numa_memory_affinity_init(const struct
> acpi_srat_mem_affinity *ma)
> > if (nd->end < end)
> > nd->end = end;
> >
> > - /* Check whether this range contains memory for other
> nodes */
> > - if (!is_node_memory_continuous(node, nd->start, nd->end))
> {
> > - bad_srat();
> > - return;
> > - }
> > + if (!is_node_memory_continuous(node, nd->start, nd->end))
> > + return -1;
> > }
> > }
> > - printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> > - node, pxm, start, end,
> > - ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
> > +
> > + printk(KERN_INFO "SRAT: Node %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> > + node, start, end, hotplug ? " (hotplug)" : "");
>
> Continuing from a comment further up: Here you remove an instance of
> logging the node <-> PXM association.
>
see my above comments.
> > node_memblk_range[num_node_memblks].start = start;
> > node_memblk_range[num_node_memblks].end = end;
> > memblk_nodeid[num_node_memblks] = node;
> > - if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) {
> > + if (hotplug) {
> > __set_bit(num_node_memblks, memblk_hotplug);
> > if (end > mem_hotplug_boundary())
> > mem_hotplug_update_boundary(end);
> > }
> > num_node_memblks++;
> > +
> > + return 0;
> > +}
> > +
> > +/* Callback for parsing of the Proximity Domain <-> Memory Area
> mappings */
> > +void __init
> > +acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> > +{
> > + unsigned pxm;
> > + nodeid_t node;
> > + int ret;
> > +
> > + if (srat_disabled())
> > + return;
> > + if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
> > + bad_srat();
> > + return;
> > + }
> > + if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
> > + return;
> > +
> > + /* Supplement the heuristics in l1tf_calculations(). */
> > + l1tf_safe_maddr = max(l1tf_safe_maddr,
> > + ROUNDUP((ma->base_address + ma->length), PAGE_SIZE));
>
> Indentation and unnecessary pair of parentheses.
>
OK.
> > + if (!numa_memblks_available())
> > + {
>
> For code you touch, please try to bring it into consistent style. Here
> the brace wants to move to the previous line, seeing that the file is
> using Linux style.
>
OK.
> > + dprintk(XENLOG_WARNING,
> > + "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
>
> Here you want to fix indentation and ideally also format and grammar of
> the actual log message.
>
I will fix it, thanks.
> > + bad_srat();
> > + return;
> > + }
> > +
> > + pxm = ma->proximity_domain;
> > + if (srat_rev < 2)
> > + pxm &= 0xff;
> > + node = setup_node(pxm);
> > + if (node == NUMA_NO_NODE) {
> > + bad_srat();
> > + return;
> > + }
> > +
> > + ret = numa_update_node_memblks(node, ma->base_address, ma->length,
> > + ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE);
>
> Indentation again.
Ok.
>
> Jan
next prev parent reply other threads:[~2022-01-26 10:39 UTC|newest]
Thread overview: 192+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-23 12:01 [PATCH 00/37] Add device tree based NUMA support to Arm Wei Chen
2021-09-23 12:02 ` [PATCH 01/37] xen/arm: Print a 64-bit number in hex from early uart Wei Chen
2021-09-23 12:02 ` [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA nodes number Wei Chen
2021-09-23 23:45 ` Stefano Stabellini
2021-09-24 1:24 ` Wei Chen
2021-09-24 8:55 ` Jan Beulich
2021-09-24 10:33 ` Wei Chen
2021-09-24 10:47 ` Jan Beulich
2021-09-23 12:02 ` [PATCH 03/37] xen/x86: Initialize memnodemapsize while faking NUMA node Wei Chen
2021-09-24 8:57 ` Jan Beulich
2021-09-24 10:34 ` Wei Chen
2021-09-23 12:02 ` [PATCH 04/37] xen: introduce an arch helper for default dma zone status Wei Chen
2021-09-23 23:55 ` Stefano Stabellini
2021-09-24 1:50 ` Wei Chen
2022-01-17 16:10 ` Jan Beulich
2022-01-18 7:51 ` Wei Chen
2022-01-18 8:16 ` Jan Beulich
2022-01-18 9:20 ` Wei Chen
2022-01-18 14:16 ` Jan Beulich
2022-01-19 2:49 ` Wei Chen
2022-01-19 7:50 ` Jan Beulich
2022-01-19 8:33 ` Wei Chen
2021-09-23 12:02 ` [PATCH 05/37] xen: decouple NUMA from ACPI in Kconfig Wei Chen
2021-09-23 12:02 ` [PATCH 06/37] xen/arm: use !CONFIG_NUMA to keep fake NUMA API Wei Chen
2021-09-24 0:05 ` Stefano Stabellini
2021-09-24 10:21 ` Wei Chen
2021-09-23 12:02 ` [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node structure Wei Chen
2021-09-24 0:11 ` Stefano Stabellini
2021-09-24 0:13 ` Stefano Stabellini
2021-09-24 3:00 ` Wei Chen
2022-01-18 15:22 ` Jan Beulich
2022-01-19 6:33 ` Wei Chen
2022-01-19 7:55 ` Jan Beulich
2022-01-19 8:36 ` Wei Chen
2021-09-23 12:02 ` [PATCH 08/37] xen/x86: add detection of discontinous node memory range Wei Chen
2021-09-24 0:25 ` Stefano Stabellini
2021-09-24 4:28 ` Wei Chen
2021-09-24 19:52 ` Stefano Stabellini
2021-09-26 10:11 ` Wei Chen
2021-09-27 3:13 ` Stefano Stabellini
2021-09-27 5:05 ` Stefano Stabellini
2021-09-27 9:50 ` Wei Chen
2021-09-27 17:19 ` Stefano Stabellini
2021-09-28 4:41 ` Wei Chen
2021-09-28 4:59 ` Stefano Stabellini
2022-01-18 16:13 ` Jan Beulich
2022-01-19 7:33 ` Wei Chen
2022-01-19 8:01 ` Jan Beulich
2022-01-19 8:24 ` Wei Chen
2021-09-23 12:02 ` [PATCH 09/37] xen/x86: introduce two helpers to access memory hotplug end Wei Chen
2021-09-24 0:29 ` Stefano Stabellini
2021-09-24 4:21 ` Wei Chen
2022-01-24 16:24 ` Jan Beulich
2022-01-26 7:53 ` Wei Chen
2021-09-23 12:02 ` [PATCH 10/37] xen/x86: use helpers to access/update mem_hotplug Wei Chen
2021-09-24 0:31 ` Stefano Stabellini
2021-09-24 4:29 ` Wei Chen
2022-01-24 16:29 ` Jan Beulich
2022-01-26 7:58 ` Wei Chen
2021-09-23 12:02 ` [PATCH 11/37] xen/x86: abstract neutral code from acpi_numa_memory_affinity_init Wei Chen
2021-09-24 0:38 ` Stefano Stabellini
2022-01-24 16:50 ` Jan Beulich
2022-01-26 10:39 ` Wei Chen [this message]
2021-09-23 12:02 ` [PATCH 12/37] xen/x86: decouple nodes_cover_memory from E820 map Wei Chen
2021-09-24 0:39 ` Stefano Stabellini
2022-01-24 16:59 ` Jan Beulich
2022-01-27 8:03 ` Wei Chen
2022-01-27 8:08 ` Jan Beulich
2022-01-27 9:03 ` Wei Chen
2022-01-27 9:22 ` Jan Beulich
2022-01-27 9:27 ` Wei Chen
2021-09-23 12:02 ` [PATCH 13/37] xen/x86: decouple processor_nodes_parsed from acpi numa functions Wei Chen
2021-09-24 0:40 ` Stefano Stabellini
2022-01-25 9:49 ` Jan Beulich
2022-01-27 8:06 ` Wei Chen
2021-09-23 12:02 ` [PATCH 14/37] xen/x86: use name fw_numa to replace acpi_numa Wei Chen
2021-09-24 0:40 ` Stefano Stabellini
2022-01-25 10:12 ` Jan Beulich
2022-01-27 8:09 ` Wei Chen
2021-09-23 12:02 ` [PATCH 15/37] xen/x86: rename acpi_scan_nodes to numa_scan_nodes Wei Chen
2021-09-24 0:40 ` Stefano Stabellini
2022-01-25 10:17 ` Jan Beulich
2022-01-27 8:14 ` Wei Chen
2021-09-23 12:02 ` [PATCH 16/37] xen/x86: export srat_bad to external Wei Chen
2021-09-24 0:41 ` Stefano Stabellini
2022-01-25 10:22 ` Jan Beulich
2022-01-27 8:35 ` Wei Chen
2022-01-27 8:37 ` Jan Beulich
2022-01-27 8:47 ` Wei Chen
2021-09-23 12:02 ` [PATCH 17/37] xen/x86: use CONFIG_NUMA to gate numa_scan_nodes Wei Chen
2021-09-24 0:41 ` Stefano Stabellini
2022-01-25 10:26 ` Jan Beulich
2022-01-27 8:37 ` Wei Chen
2021-09-23 12:02 ` [PATCH 18/37] xen: move NUMA common code from x86 to common Wei Chen
2021-09-23 12:02 ` [PATCH 19/37] xen/x86: promote VIRTUAL_BUG_ON to ASSERT in Wei Chen
2022-01-17 16:21 ` Jan Beulich
2022-01-18 7:52 ` Wei Chen
2021-09-23 12:02 ` [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI architecture Wei Chen
2021-09-24 1:15 ` Stefano Stabellini
2021-09-24 4:34 ` Wei Chen
2021-09-24 7:58 ` Jan Beulich
2021-09-24 10:31 ` Wei Chen
2021-09-24 10:49 ` Jan Beulich
2021-09-26 10:25 ` Wei Chen
2021-09-27 10:28 ` Wei Chen
2021-09-28 0:59 ` Stefano Stabellini
2021-09-28 4:16 ` Wei Chen
2021-09-28 5:01 ` Stefano Stabellini
2021-09-28 8:02 ` Jan Beulich
2021-10-03 23:28 ` Wei Chen
2022-01-25 10:34 ` Jan Beulich
2022-01-27 8:44 ` Wei Chen
2022-01-27 8:51 ` Wei Chen
2022-01-27 9:00 ` Jan Beulich
2022-01-27 9:09 ` Wei Chen
2022-01-27 9:16 ` Jan Beulich
2022-01-27 9:25 ` Wei Chen
2022-01-27 9:27 ` Jan Beulich
2022-01-27 10:00 ` Julien Grall
2022-01-28 4:35 ` Wei Chen
2021-09-23 12:02 ` [PATCH 21/37] xen/arm: Keep memory nodes in dtb for NUMA when boot from EFI Wei Chen
2021-09-24 1:23 ` Stefano Stabellini
2021-09-24 4:36 ` Wei Chen
2022-01-25 10:38 ` Jan Beulich
2022-01-27 8:45 ` Wei Chen
2021-09-23 12:02 ` [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS Wei Chen
2021-09-24 1:34 ` Stefano Stabellini
2021-09-26 13:13 ` Wei Chen
2021-09-27 3:25 ` Stefano Stabellini
2021-09-27 4:18 ` Wei Chen
2021-09-27 4:59 ` Stefano Stabellini
2021-09-27 6:25 ` Julien Grall
2021-09-27 6:46 ` Wei Chen
2021-09-27 6:53 ` Wei Chen
2021-09-27 7:35 ` Julien Grall
2021-09-27 10:21 ` Wei Chen
2021-09-27 10:39 ` Julien Grall
2021-09-27 16:58 ` Stefano Stabellini
2021-09-28 2:57 ` Wei Chen
2021-09-23 12:02 ` [PATCH 23/37] xen/arm: implement node distance helpers for Arm Wei Chen
2021-09-24 1:46 ` Stefano Stabellini
2021-09-24 4:41 ` Wei Chen
2021-09-24 19:36 ` Stefano Stabellini
2021-09-26 10:15 ` Wei Chen
2021-09-23 12:02 ` [PATCH 24/37] xen/arm: implement two arch helpers to get memory map info Wei Chen
2021-09-24 2:06 ` Stefano Stabellini
2021-09-24 4:42 ` Wei Chen
2021-09-23 12:02 ` [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA initialization Wei Chen
2021-09-24 2:09 ` Stefano Stabellini
2021-09-24 4:45 ` Wei Chen
2021-09-24 8:07 ` Jan Beulich
2021-09-24 19:33 ` Stefano Stabellini
2021-09-23 12:02 ` [PATCH 26/37] xen/arm: build NUMA cpu_to_node map in dt_smp_init_cpus Wei Chen
2021-09-24 2:26 ` Stefano Stabellini
2021-09-24 4:25 ` Wei Chen
2021-09-23 12:02 ` [PATCH 27/37] xen/arm: Add boot and secondary CPU to NUMA system Wei Chen
2021-09-23 12:02 ` [PATCH 28/37] xen/arm: stub memory hotplug access helpers for Arm Wei Chen
2021-09-24 2:33 ` Stefano Stabellini
2021-09-24 4:26 ` Wei Chen
2021-09-23 12:02 ` [PATCH 29/37] xen/arm: introduce a helper to parse device tree processor node Wei Chen
2021-09-24 2:44 ` Stefano Stabellini
2021-09-24 4:46 ` Wei Chen
2021-09-23 12:02 ` [PATCH 30/37] xen/arm: introduce a helper to parse device tree memory node Wei Chen
2021-09-24 3:05 ` Stefano Stabellini
2021-09-24 7:54 ` Wei Chen
2021-09-23 12:02 ` [PATCH 31/37] xen/arm: introduce a helper to parse device tree NUMA distance map Wei Chen
2021-09-24 3:05 ` Stefano Stabellini
2021-09-24 5:23 ` Wei Chen
2021-09-23 12:02 ` [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from device tree Wei Chen
2021-09-24 3:16 ` Stefano Stabellini
2021-09-24 7:58 ` Wei Chen
2021-09-24 19:42 ` Stefano Stabellini
2021-09-23 12:02 ` [PATCH 33/37] xen/arm: keep guest still be NUMA unware Wei Chen
2021-09-24 3:19 ` Stefano Stabellini
2021-09-24 10:23 ` Wei Chen
2021-09-23 12:02 ` [PATCH 34/37] xen/arm: enable device tree based NUMA in system init Wei Chen
2021-09-24 3:28 ` Stefano Stabellini
2021-09-24 9:52 ` Wei Chen
2021-09-23 12:02 ` [PATCH 35/37] xen/arm: use CONFIG_NUMA to gate node_online_map in smpboot Wei Chen
2021-09-23 12:02 ` [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable NUMA Wei Chen
2021-09-24 3:31 ` Stefano Stabellini
2021-09-24 10:13 ` Wei Chen
2021-09-24 19:39 ` Stefano Stabellini
2021-09-27 8:33 ` Jan Beulich
2021-09-27 8:45 ` Julien Grall
2021-09-27 9:17 ` Jan Beulich
2021-09-27 17:17 ` Stefano Stabellini
2021-09-28 2:59 ` Wei Chen
2021-09-28 3:30 ` Stefano Stabellini
2021-09-24 10:25 ` Jan Beulich
2021-09-24 10:37 ` Wei Chen
2021-09-23 12:02 ` [PATCH 37/37] docs: update numa command line to support Arm Wei Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PAXPR08MB7420F9BF695574EBD2AE57029E209@PAXPR08MB7420.eurprd08.prod.outlook.com \
--to=wei.chen@arm.com \
--cc=Bertrand.Marquis@arm.com \
--cc=jbeulich@suse.com \
--cc=julien@xen.org \
--cc=sstabellini@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).