From: Stefano Stabellini <sstabellini@kernel.org>
To: Wei Chen <Wei.Chen@arm.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
"xen-devel@lists.xenproject.org"
<xen-devel@lists.xenproject.org>,
"julien@xen.org" <julien@xen.org>,
Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device tree NUMA distance map
Date: Wed, 1 Sep 2021 09:21:42 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s> (raw)
In-Reply-To: <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
[-- Attachment #1: Type: text/plain, Size: 9596 bytes --]
On Wed, 1 Sep 2021, Wei Chen wrote:
> Hi Stefano,
>
> > -----Original Message-----
> > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > Stefano Stabellini
> > Sent: 2021年9月1日 5:36
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>
> > Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
> > device tree NUMA distance map
> >
> > On Tue, 31 Aug 2021, Wei Chen wrote:
> > > Hi Stefano,
> > >
> > > > -----Original Message-----
> > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > Sent: 2021年8月31日 8:48
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > julien@xen.org;
> > > > jbeulich@suse.com; Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > Subject: Re: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to
> > parse
> > > > device tree NUMA distance map
> > > >
> > > > On Wed, 11 Aug 2021, Wei Chen wrote:
> > > > > A NUMA aware device tree will provide a "distance-map" node to
> > > > > describe distance between any two nodes. This patch introduce a
> > > > > new helper to parse this distance map.
> > > > >
> > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > ---
> > > > > xen/arch/arm/numa_device_tree.c | 67
> > +++++++++++++++++++++++++++++++++
> > > > > 1 file changed, 67 insertions(+)
> > > > >
> > > > > diff --git a/xen/arch/arm/numa_device_tree.c
> > > > b/xen/arch/arm/numa_device_tree.c
> > > > > index bbe081dcd1..6e0d1d3d9f 100644
> > > > > --- a/xen/arch/arm/numa_device_tree.c
> > > > > +++ b/xen/arch/arm/numa_device_tree.c
> > > > > @@ -200,3 +200,70 @@ device_tree_parse_numa_memory_node(const void
> > *fdt,
> > > > int node,
> > > > >
> > > > > return 0;
> > > > > }
> > > > > +
> > > > > +/* Parse NUMA distance map v1 */
> > > > > +int __init
> > > > > +device_tree_parse_numa_distance_map_v1(const void *fdt, int node)
> > > > > +{
> > > > > + const struct fdt_property *prop;
> > > > > + const __be32 *matrix;
> > > > > + int entry_count, len, i;
> > > > > +
> > > > > + printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n");
> > > > > +
> > > > > + prop = fdt_get_property(fdt, node, "distance-matrix", &len);
> > > > > + if ( !prop )
> > > > > + {
> > > > > + printk(XENLOG_WARNING
> > > > > + "NUMA: No distance-matrix property in distance-
> > map\n");
> > > > > +
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + if ( len % sizeof(uint32_t) != 0 )
> > > > > + {
> > > > > + printk(XENLOG_WARNING
> > > > > + "distance-matrix in node is not a multiple of
> > u32\n");
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + entry_count = len / sizeof(uint32_t);
> > > > > + if ( entry_count <= 0 )
> > > > > + {
> > > > > + printk(XENLOG_WARNING "NUMA: Invalid distance-matrix\n");
> > > > > +
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + matrix = (const __be32 *)prop->data;
> > > > > + for ( i = 0; i + 2 < entry_count; i += 3 )
> > > > > + {
> > > > > + uint32_t from, to, distance;
> > > > > +
> > > > > + from = dt_read_number(matrix, 1);
> > > > > + matrix++;
> > > > > + to = dt_read_number(matrix, 1);
> > > > > + matrix++;
> > > > > + distance = dt_read_number(matrix, 1);
> > > > > + matrix++;
> > > > > +
> > > > > + if ( (from == to && distance != NUMA_LOCAL_DISTANCE) ||
> > > > > + (from != to && distance <= NUMA_LOCAL_DISTANCE) )
> > > > > + {
> > > > > + printk(XENLOG_WARNING
> > > > > + "Invalid nodes' distance from node#%d to node#%d
> > > > = %d\n",
> > > > > + from, to, distance);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + printk(XENLOG_INFO "NUMA: distance from node#%d to node#%d
> > > > = %d\n",
> > > > > + from, to, distance);
> > > > > + numa_set_distance(from, to, distance);
> > > > > +
> > > > > + /* Set default distance of node B->A same as A->B */
> > > > > + if (to > from)
> > > > > + numa_set_distance(to, from, distance);
> > > >
> > > > I am a bit unsure about this last 2 lines: why calling
> > numa_set_distance
> > > > in the opposite direction only when to > from? Wouldn't it be OK to
> > > > always do both:
> > > >
> > > > numa_set_distance(from, to, distance);
> > > > numa_set_distance(to, from, distance);
> > > >
> > > > ?
> > > >
> > > I borrowed this code from Linux, but here is my understanding:
> > >
> > > First, I read some notes in Documentation/devicetree/bindings/numa.txt
> > > 1. Each entry represents distance from first node to second node.
> > > The distances are equal in either direction.
> > > 2. distance-matrix should have entries in lexicographical ascending
> > > order of nodes.
> > >
> > > Here is an example of distance-map node in DTB:
> > > Sample#1, full list:
> > > distance-map {
> > > compatible = "numa-distance-map-v1";
> > > distance-matrix = <0 0 10>,
> > > <0 1 20>,
> > > <0 2 40>,
> > > <0 3 20>,
> > > <1 0 20>,
> > > <1 1 10>,
> > > <1 2 20>,
> > > <1 3 40>,
> > > <2 0 40>,
> > > <2 1 20>,
> > > <2 2 10>,
> > > <2 3 20>,
> > > <3 0 20>,
> > > <3 1 40>,
> > > <3 2 20>,
> > > <3 3 10>;
> > > };
> > >
> > > Call numa_set_distance when "to > from" will prevent Xen to call
> > > numa_set_distance(0, 1, 20) again when it's setting distance for <1 0
> > 20>.
> > > But, numa_set_distance(1, 0, 20) will be call twice.
> > >
> > > Normally, distance-map node will be optimized in following sample#2,
> > > all redundant entries are removed:
> > > Sample#2, partial list:
> > > distance-map {
> > > compatible = "numa-distance-map-v1";
> > > distance-matrix = <0 0 10>,
> > > <0 1 20>,
> > > <0 2 40>,
> > > <0 3 20>,
> > > <1 1 10>,
> > > <1 2 20>,
> > > <1 3 40>,
> > > <2 2 10>,
> > > <2 3 20>,
> > > <3 3 10>;
> > > };
> > >
> > > There is not any "from > to" entry in the map. But using this partial
> > map
> > > still can set all distances for all pairs. And numa_set_distance(1, 0,
> > 20)
> > > will be only once.
> >
> > I see. I can't find in Documentation/devicetree/bindings/numa.txt where
> > it says that "from > to" nodes can be omitted. If it is not written
> > down, then somebody could easily optimize it the opposite way:
> >
> > distance-matrix = <0 0 10>,
> > <1 0 20>,
> > <2 0 40>,
> > <3 0 20>,
> > <1 1 10>,
> > <2 1 20>,
> > <3 1 40>,
> > <2 2 10>,
> > <3 2 20>,
> > <3 3 10>;
> >
>
> Yes, you're right. Spec doesn't say opposite way is unallowed.
>
> > I think the code in Xen should be resilient and able to cope with a
> > device tree like the one you wrote or the one I wrote. From a code
> > perspective, it should be very easy to do. If nothing else it would make
> > Xen more resilient against buggy firmware.
> >
> >
>
> I don't disagree with that.
>
> > > > But in any case, I have a different suggestion. The binding states
> > that
> > > > "distances are equal in either direction". Also it has an example
> > where
> > > > only one direction is expressed unfortunately (at the end of the
> > > > document).
> > > >
> > >
> > > Oh, I should see this comment first, then I will not post above
> > > comment : )
> > >
> > > > So my suggestion is to parse it as follows:
> > > >
> > > > - call numa_set_distance just once from
> > > > device_tree_parse_numa_distance_map_v1
> > > >
> > > > - in numa_set_distance:
> > > > - set node_distance_map[from][to] = distance;
> > > > - check node_distance_map[to][from]
> > > > - if unset, node_distance_map[to][from] = distance;
> > > > - if already set to the same value, return success;
> > > > - if already set to a different value, return error;
> > >
> > > I don't really like this implementation. I want the behavior of
> > > numa_set_distance just like the function name, do not include
> > > implicit operations. Otherwise, except the user read this function
> > > implementation before he use it, he probably doesn't know this
> > > function has done so many things.
> >
> > You can leave numa_set_distance as-is without any implicit operations.
> >
> > In that case, just call numa_set_distance twice from numa_set_distance
> > for both from/to and to/from. numa_set_distance could return error is
>
> I am OK for the first sentence. But...
>
> > the entry was already set to a different value or success otherwise
> > (also in the case it was already set to the same value). This would
>
> ... I prefer not to check the previous value. Subsequent numa_set_distance
> call will override previous calls. Keep numa_set_distance as simple as
> it can. And when you pass new data to numa_set_distance, it doesn't
> know whether the previous data was correct or the new data is correct.
> Only caller may have known.
That might be OK but if not numa_set_distance then somebody else needs
to check against overwriting previous values. That is to be able to spot
bad device tree cases like:
0 1 20
1 0 40
next prev parent reply other threads:[~2021-09-01 16:22 UTC|newest]
Thread overview: 196+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-11 10:23 [XEN RFC PATCH 00/40] Add device tree based NUMA support to Arm64 Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 01/40] tools: Fix -Werror=maybe-uninitialized for xlu_pci_parse_bdf Wei Chen
2021-08-11 10:49 ` Jan Beulich
2021-08-13 6:28 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 02/40] xen/arm: Print a 64-bit number in hex from early uart Wei Chen
2021-08-19 13:05 ` Julien Grall
2021-08-20 1:13 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 03/40] xen/x86: Initialize memnodemapsize while faking NUMA node Wei Chen
2021-08-12 15:32 ` Jan Beulich
2021-08-13 7:26 ` Wei Chen
2021-08-13 8:29 ` Jan Beulich
2021-08-11 10:23 ` [XEN RFC PATCH 04/40] xen/arm: return default DMA bit width when platform is not set Wei Chen
2021-08-11 10:54 ` Jan Beulich
2021-08-13 6:54 ` Wei Chen
2021-08-13 6:56 ` Jan Beulich
2021-08-19 13:28 ` Julien Grall
2021-08-20 2:04 ` Wei Chen
2021-08-20 8:20 ` Julien Grall
2021-08-20 9:37 ` Wei Chen
2021-08-20 11:18 ` Julien Grall
2021-08-20 11:58 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 05/40] xen/arm: Fix lowmem_bitsize when arch_get_dma_bitsize return 0 Wei Chen
2021-08-19 13:32 ` Julien Grall
2021-08-20 2:05 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 06/40] xen: decouple NUMA from ACPI in Kconfig Wei Chen
2021-08-12 15:36 ` Jan Beulich
2021-08-13 7:27 ` Wei Chen
2021-08-12 16:54 ` Julien Grall
2021-08-13 7:28 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 07/40] xen/arm: use !CONFIG_NUMA to keep fake NUMA API Wei Chen
2021-08-19 13:34 ` Julien Grall
2021-08-20 2:08 ` Wei Chen
2021-08-20 8:23 ` Julien Grall
2021-08-20 10:24 ` Wei Chen
2021-08-20 11:24 ` Julien Grall
2021-08-20 12:23 ` Wei Chen
2021-08-20 14:41 ` Julien Grall
2021-08-11 10:23 ` [XEN RFC PATCH 08/40] xen/x86: Move NUMA memory node map functions to common Wei Chen
2021-08-23 17:47 ` Julien Grall
2021-08-24 4:07 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 09/40] xen/x86: Move numa_add_cpu_node " Wei Chen
2021-08-23 17:54 ` Julien Grall
2021-08-24 4:18 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 10/40] xen/x86: Move NR_NODE_MEMBLKS macro " Wei Chen
2021-08-23 17:58 ` Julien Grall
2021-08-11 10:23 ` [XEN RFC PATCH 11/40] xen/x86: Move NUMA nodes and memory block ranges " Wei Chen
2021-08-24 17:40 ` Julien Grall
2021-08-25 0:57 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 12/40] xen/x86: Move numa_initmem_init " Wei Chen
2021-08-25 10:21 ` Julien Grall
2021-08-25 11:15 ` Wei Chen
2021-08-25 13:26 ` Julien Grall
2021-08-11 10:23 ` [XEN RFC PATCH 13/40] xen/arm: introduce numa_set_node for Arm Wei Chen
2021-08-25 10:36 ` Julien Grall
2021-08-25 12:07 ` Wei Chen
2021-08-25 13:24 ` Julien Grall
2021-08-26 5:13 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 14/40] xen/arm: set NUMA nodes max number to 64 by default Wei Chen
2021-08-25 13:28 ` Julien Grall
2021-08-25 13:36 ` Jan Beulich
2021-08-26 2:26 ` Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 15/40] xen/x86: move NUMA API from x86 header to common header Wei Chen
2021-08-11 10:23 ` [XEN RFC PATCH 16/40] xen/arm: Create a fake NUMA node to use common code Wei Chen
2021-08-26 23:10 ` Stefano Stabellini
2021-08-27 1:15 ` Wei Chen
2021-08-27 6:18 ` Jan Beulich
2021-08-27 9:32 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 17/40] xen/arm: Introduce DEVICE_TREE_NUMA Kconfig for arm64 Wei Chen
2021-08-19 13:38 ` Julien Grall
2021-08-20 2:30 ` Wei Chen
2021-08-20 8:41 ` Julien Grall
2021-08-20 10:49 ` Wei Chen
2021-08-20 11:28 ` Julien Grall
2021-08-20 12:25 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 18/40] xen/arm: Keep memory nodes in dtb for NUMA when boot from EFI Wei Chen
2021-08-19 17:35 ` Julien Grall
2021-08-20 2:18 ` Wei Chen
2021-08-26 23:24 ` Stefano Stabellini
2021-08-27 7:41 ` Julien Grall
2021-08-27 23:10 ` Stefano Stabellini
2021-08-27 9:23 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 19/40] xen: fdt: Introduce a helper to check fdt node type Wei Chen
2021-08-25 13:39 ` Julien Grall
2021-08-26 6:00 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 20/40] xen/arm: implement node distance helpers for Arm64 Wei Chen
2021-08-26 23:52 ` Stefano Stabellini
2021-08-27 9:30 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 21/40] xen/arm: introduce device_tree_numa as a switch for device tree NUMA Wei Chen
2021-08-19 17:45 ` Julien Grall
2021-08-20 2:21 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 22/40] xen/arm: introduce a helper to parse device tree processor node Wei Chen
2021-08-19 18:09 ` Julien Grall
2021-08-23 8:42 ` Wei Chen
2021-08-19 18:10 ` Julien Grall
2021-08-23 8:47 ` Wei Chen
2021-08-23 10:59 ` Julien Grall
2021-08-24 4:09 ` Wei Chen
2021-08-19 18:13 ` Julien Grall
2021-08-20 2:23 ` Wei Chen
2021-08-20 8:44 ` Julien Grall
2021-08-20 11:53 ` Wei Chen
2021-08-27 0:06 ` Stefano Stabellini
2021-08-27 9:31 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse device tree memory node Wei Chen
2021-08-25 13:48 ` Julien Grall
2021-08-26 6:35 ` Wei Chen
2021-08-26 8:21 ` Julien Grall
2021-08-26 11:54 ` Wei Chen
2021-08-28 1:06 ` Stefano Stabellini
2021-08-28 3:56 ` Wei Chen
2021-08-28 10:33 ` Julien Grall
2021-08-28 13:58 ` Wei Chen
2021-09-08 7:34 ` Wei Chen
2021-09-08 22:31 ` Stefano Stabellini
2021-09-09 3:54 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device tree NUMA distance map Wei Chen
2021-08-25 13:56 ` Julien Grall
2021-08-26 7:01 ` Wei Chen
2021-08-31 0:48 ` Stefano Stabellini
2021-08-31 10:17 ` Wei Chen
2021-08-31 21:36 ` Stefano Stabellini
2021-09-01 11:04 ` Wei Chen
2021-09-01 16:21 ` Stefano Stabellini [this message]
2021-09-02 2:30 ` Wei Chen
2021-09-02 15:19 ` Stefano Stabellini
2021-09-02 6:00 ` Jan Beulich
2021-09-02 14:14 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 25/40] xen/arm: unified entry to parse all NUMA data from device tree Wei Chen
2021-08-31 0:54 ` Stefano Stabellini
2021-08-31 17:47 ` Julien Grall
2021-09-01 18:30 ` Stefano Stabellini
2021-09-02 2:48 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 26/40] xen/arm: Add boot and secondary CPU to NUMA system Wei Chen
2021-08-25 16:58 ` Julien Grall
2021-08-26 7:24 ` Wei Chen
2021-08-26 8:49 ` Julien Grall
2021-08-26 9:39 ` Jan Beulich
2021-08-26 12:08 ` Wei Chen
2021-08-26 12:26 ` Jan Beulich
2021-08-11 10:24 ` [XEN RFC PATCH 27/40] xen/arm: build CPU NUMA node map while creating cpu_logical_map Wei Chen
2021-08-25 17:06 ` Julien Grall
2021-08-26 7:26 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 28/40] xen/x86: decouple nodes_cover_memory with E820 map Wei Chen
2021-08-31 1:07 ` Stefano Stabellini
2021-08-31 10:19 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 29/40] xen/arm: implement Arm arch helpers Arm to get memory map info Wei Chen
2021-08-25 17:09 ` Julien Grall
2021-08-26 7:27 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 30/40] xen: move NUMA memory and CPU parsed nodemasks to common Wei Chen
2021-08-25 17:16 ` Julien Grall
2021-08-26 7:29 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 31/40] xen/x86: move nodes_cover_memory " Wei Chen
2021-08-31 1:16 ` Stefano Stabellini
2021-08-31 13:43 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 32/40] xen/x86: make acpi_scan_nodes to be neutral Wei Chen
2021-08-27 14:08 ` Julien Grall
2021-08-28 2:11 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 33/40] xen: export bad_srat and srat_disabled to extern Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 34/40] xen: move numa_scan_nodes from x86 to common Wei Chen
2021-08-27 14:14 ` Julien Grall
2021-08-28 2:12 ` Wei Chen
2021-08-31 1:26 ` Stefano Stabellini
2021-08-31 13:43 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 35/40] xen: enable numa_scan_nodes for device tree based NUMA Wei Chen
2021-08-27 14:19 ` Julien Grall
2021-08-28 2:13 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 36/40] xen/arm: keep guest still be NUMA unware Wei Chen
2021-08-27 14:28 ` Julien Grall
2021-08-28 2:19 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 37/40] xen: introduce an arch helper to do NUMA init failed fallback Wei Chen
2021-08-27 14:30 ` Julien Grall
2021-08-28 3:09 ` Wei Chen
2021-08-28 3:45 ` Wei Chen
2021-08-30 9:52 ` Jan Beulich
2021-08-30 10:38 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 38/40] xen/arm: enable device tree based NUMA in system init Wei Chen
2021-08-27 14:32 ` Julien Grall
2021-08-28 3:17 ` Wei Chen
2021-08-28 10:45 ` Julien Grall
2021-08-28 14:02 ` Wei Chen
2021-08-31 1:50 ` Stefano Stabellini
2021-08-31 13:43 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 39/40] xen/x86: move numa_setup to common to support NUMA switch in command line Wei Chen
2021-08-27 14:37 ` Julien Grall
2021-08-28 3:22 ` Wei Chen
2021-08-31 1:53 ` Stefano Stabellini
2021-08-31 13:44 ` Wei Chen
2021-08-11 10:24 ` [XEN RFC PATCH 40/40] xen/x86: move dump_numa info hotkey to common Wei Chen
2021-08-11 10:41 ` [XEN RFC PATCH 00/40] Add device tree based NUMA support to Arm64 Jan Beulich
2021-08-13 2:33 ` Wei Chen
2021-08-13 6:53 ` Jan Beulich
2021-08-19 13:42 ` Julien Grall
2021-08-19 14:05 ` Bertrand Marquis
2021-08-19 17:11 ` Julien Grall
2021-08-26 0:09 ` Stefano Stabellini
2021-08-26 7:31 ` Wei Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s \
--to=sstabellini@kernel.org \
--cc=Bertrand.Marquis@arm.com \
--cc=Wei.Chen@arm.com \
--cc=julien@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).