* Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.
[not found] <1410944189-3608-1-git-send-email-ganapatrao.kulkarni@caviumnetworks.com>
@ 2014-09-17 15:37 ` Kumar Gala
2014-09-17 19:34 ` Mark Rutland
[not found] ` <541A01B7.8030601@mentor.com>
1 sibling, 1 reply; 4+ messages in thread
From: Kumar Gala @ 2014-09-17 15:37 UTC (permalink / raw)
To: Ganapatrao Kulkarni
Cc: Mark Rutland, open list:OPEN FIRMWARE AND...,
Pawel Moll, Ian Campbell, Rob Herring, linuxppc-dev,
linux-arm-kernel, gpkulkarni
On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni =
<ganapatrao.kulkarni@caviumnetworks.com> wrote:
> From: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>=20
> This patch adds property "nid" to memory node to provide the memory =
range to
> numa node id mapping.
>=20
> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>=20
> =97
Adding the PPC guys as they=92ve been doing NUMA on IBM Power Servers =
for years with OF/DT. So we should really try and follow what they=92ve =
done.
> Documentation/devicetree/bindings/numa.txt | 58 =
++++++++++++++++++++++++++++++
> 1 file changed, 58 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/numa.txt
>=20
> diff --git a/Documentation/devicetree/bindings/numa.txt =
b/Documentation/devicetree/bindings/numa.txt
> new file mode 100644
> index 0000000..c4a94f2
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/numa.txt
> @@ -0,0 +1,58 @@
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +numa id binding description
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +1 - Introduction
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +The device node property "nid(numa node id)" can be added to memory
> +device node to map the range of memory addresses as defined in =
property "reg".
> +The property "nid" maps the memory range to the numa node id, which =
is used to
> +find the local and remory pages on numa aware systems.
> +
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +2 - nid property
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> +Numa node id, "nid" is required property of memory device node for=20
> +numa enabled platforms.
> +
> +|------------------------------------------------------|
> +|Property Type | Usage | Value Type | Definition |
> +|------------------------------------------------------|
> +| nid | R | <u32> | Numa Node id |
> +| | | | for this memory |
> +|------------------------------------------------------|
> +
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> +4 - Example memory nodes with numa node id mapping
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> +
> +Example 1 (2 memory nodes, each mapped to a numa node.):
> +
> + memory@00000000 {
> + device_type =3D "memory";
> + reg =3D <0x0 0x00000000 0x0 0x80000000>;
> + nid =3D <0x0>;
> + };
> +
> + memory@10000000000 {
> + device_type =3D "memory";
> + reg =3D <0x100 0x00000000 0x0 0x80000000>;
> + nid =3D <0x1>;
> + };
> +
> +Example 2 (multiple memory ranges in each memory node and mapped to =
numa node):
> +
> + memory@00000000 {
> + device_type =3D "memory";
> + reg =3D <0x0 0x00000000 0x0 0x80000000>,
> + <0x0 0x80000000 0x0 0x80000000>;
> + nid =3D <0x0>;
> + };
> +
> + memory@10000000000 {
> + device_type =3D "memory";
> + reg =3D <0x100 0x00000000 0x0 0x80000000>,
> + <0x100 0x80000000 0x0 0x80000000>;
> + nid =3D <0x1>;
> + };
> --=20
> 1.8.1.4
>=20
--=20
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, =
hosted by The Linux Foundation
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.
2014-09-17 15:37 ` [RFC PATCH] dt:numa: adding numa node mapping for memory nodes Kumar Gala
@ 2014-09-17 19:34 ` Mark Rutland
2014-10-28 7:27 ` Hanjun Guo
0 siblings, 1 reply; 4+ messages in thread
From: Mark Rutland @ 2014-09-17 19:34 UTC (permalink / raw)
To: Kumar Gala
Cc: open list:OPEN FIRMWARE AND...,
Pawel Moll, Ian Campbell, Rob Herring, Ganapatrao Kulkarni,
linuxppc-dev, linux-arm-kernel, gpkulkarni
On Wed, Sep 17, 2014 at 04:37:30PM +0100, Kumar Gala wrote:
>
> On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni <ganapatrao.kulkarni@caviumnetworks.com> wrote:
>
> > From: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
> >
> > This patch adds property "nid" to memory node to provide the memory range to
> > numa node id mapping.
> >
> > Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
> >
> > —
>
> Adding the PPC guys as they’ve been doing NUMA on IBM Power Servers
> for years with OF/DT. So we should really try and follow what they’ve
> done.
Agreed.
> > Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/numa.txt
> >
> > diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
> > new file mode 100644
> > index 0000000..c4a94f2
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/numa.txt
> > @@ -0,0 +1,58 @@
> > +======================================================
> > +numa id binding description
> > +======================================================
> > +
> > +======================================================
> > +1 - Introduction
> > +======================================================
> > +The device node property "nid(numa node id)" can be added to memory
Why the quotes?
> > +device node to map the range of memory addresses as defined in property "reg".
> > +The property "nid" maps the memory range to the numa node id, which is used to
> > +find the local and remory pages on numa aware systems.
What is a "numa node id", exactly, and how is the OS intended to use it?
I don't see how this can be of any use as-is.
Mark.
> > +
> > +======================================================
> > +2 - nid property
> > +======================================================
> > +Numa node id, "nid" is required property of memory device node for
> > +numa enabled platforms.
> > +
> > +|------------------------------------------------------|
> > +|Property Type | Usage | Value Type | Definition |
> > +|------------------------------------------------------|
> > +| nid | R | <u32> | Numa Node id |
> > +| | | | for this memory |
> > +|------------------------------------------------------|
> > +
> > +========================================================
> > +4 - Example memory nodes with numa node id mapping
> > +========================================================
> > +
> > +Example 1 (2 memory nodes, each mapped to a numa node.):
> > +
> > + memory@00000000 {
> > + device_type = "memory";
> > + reg = <0x0 0x00000000 0x0 0x80000000>;
> > + nid = <0x0>;
> > + };
> > +
> > + memory@10000000000 {
> > + device_type = "memory";
> > + reg = <0x100 0x00000000 0x0 0x80000000>;
> > + nid = <0x1>;
> > + };
> > +
> > +Example 2 (multiple memory ranges in each memory node and mapped to numa node):
> > +
> > + memory@00000000 {
> > + device_type = "memory";
> > + reg = <0x0 0x00000000 0x0 0x80000000>,
> > + <0x0 0x80000000 0x0 0x80000000>;
> > + nid = <0x0>;
> > + };
> > +
> > + memory@10000000000 {
> > + device_type = "memory";
> > + reg = <0x100 0x00000000 0x0 0x80000000>,
> > + <0x100 0x80000000 0x0 0x80000000>;
> > + nid = <0x1>;
> > + };
> > --
> > 1.8.1.4
> >
>
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.
[not found] ` <CAFpQJXUs+6Qvj0SqgajA8_D8UdEh_EQVgTQDz0bPSn-VpEty9g@mail.gmail.com>
@ 2014-09-18 14:30 ` Ganapatrao Kulkarni
0 siblings, 0 replies; 4+ messages in thread
From: Ganapatrao Kulkarni @ 2014-09-18 14:30 UTC (permalink / raw)
To: Zi Shen Lim
Cc: Mark Rutland, devicetree, Pawel Moll, Ian Campbell, Nathan Lynch,
Rob Herring, linux-arm-kernel, Kumar Gala, linuxppc-dev,
Ganapatrao Kulkarni
looks like previous email not had the email ids added by kumar gala.
adding the missing ids and sending again.
On Thu, Sep 18, 2014 at 9:32 AM, Ganapatrao Kulkarni
<gpkulkarni@gmail.com> wrote:
> On Thu, Sep 18, 2014 at 4:42 AM, Zi Shen Lim <zlim.lnx@gmail.com> wrote:
>> On Wed, Sep 17, 2014 at 2:48 PM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>>> On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
>>>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>>>>
>>>> This patch adds property "nid" to memory node to provide the memory range to
>>>> numa node id mapping.
>>>>
>>>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>>>>
>>>> ---
>>>> Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>>> 1 file changed, 58 insertions(+)
>>>> create mode 100644 Documentation/devicetree/bindings/numa.txt
>>>>
>>>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>>>> new file mode 100644
>>>> index 0000000..c4a94f2
>>>> --- /dev/null
>>>> +++ b/Documentation/devicetree/bindings/numa.txt
>>>> @@ -0,0 +1,58 @@
>>>> +======================================================
>>>> +numa id binding description
>>>> +======================================================
>>>> +
>>>> +======================================================
>>>> +1 - Introduction
>>>> +======================================================
>>>> +The device node property "nid(numa node id)" can be added to memory
>>>> +device node to map the range of memory addresses as defined in property "reg".
>>>> +The property "nid" maps the memory range to the numa node id, which is used to
>>>> +find the local and remory pages on numa aware systems.
>>>
>>> "Local" and "remote" memory are notions that relate to some other
>>> resource -- typically a CPU, but also I/O resources on some systems. It
>>> seems to me that a useful NUMA binding would at least specify a "nid"
>>> property, or something like it, for both cpu and memory nodes. But this
>>> document speaks only of memory nodes.
> IMO, nid can be extended to cpu node also if the arch dont have any
> other way to do mapping, in fact this can be scaled to any node.
> for ARM arch, cpu mapping can be fetched from topology parsing and
> more over cpu mapping is required much later when secondary cpus comes
> online.
> i have added for memory, since this is parsed very early in the boot
> sequence and there is no support for dt based map at this moment(at
> least i am not aware off).
> numa distance matrix can also be defined, if the architecture has
> variable node distance other than just local/remote.
>>>
>>> As Kumar said, the device tree on powerpc server systems already has
>>> properties that express NUMA information. If you can get hold of a copy
>>> of the PAPR (not ePAPR) from power.org, refer to the description of
>>> "ibm,associativity" and related properties. I recall that it's a bit
>>> more complex than this proposal, though.
>>
>> I'm not able to find a link to the actual PAPR (not ePAPR)
>> specification. Anyone has a linky?
>>
>> I did find some code in tree, but couldn't find the bindings in
>> Documentation/devicetree/.
> yes, there is nothing in Documentation, if PPC are using since several
> years, then i humbly request to publish the documentation for numa
> bindings.
> No point in reinventing the wheel. Also i did not find any spec named
> PAPR, is it open spec, available to all like ePAPR?
>
>>
>> Seems like we'd care about form1?
>> -----8<-----
>> commit 41eab6f88f24124df89e38067b3766b7bef06ddb
>> Author: Anton Blanchard <anton@samba.org>
>> Date: Sun May 16 20:22:31 2010 +0000
>>
>> powerpc/numa: Use form 1 affinity to setup node distance
>>
>> Form 1 affinity allows multiple entries in
>> ibm,associativity-reference-points
>> which represent affinity domains in decreasing order of importance. The
>> Linux concept of a node is always the first entry, but using the other
>> values as an input to node_distance() allows the memory allocator to make
>> better decisions on which node to go first when local memory has been
>> exhausted.
>>
>> We keep things simple and create an array indexed by NUMA node, capped at
>> 4 entries. Each time we lookup an associativity property we initialise
>> the array which is overkill, but since we should only hit this path during
>> boot it didn't seem worth adding a per node valid bit.
>>
>> Signed-off-by: Anton Blanchard <anton@samba.org>
>> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> ----->8-----
>>
>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> linux-arm-kernel@lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.
2014-09-17 19:34 ` Mark Rutland
@ 2014-10-28 7:27 ` Hanjun Guo
0 siblings, 0 replies; 4+ messages in thread
From: Hanjun Guo @ 2014-10-28 7:27 UTC (permalink / raw)
To: Mark Rutland, Kumar Gala, Ganapatrao Kulkarni
Cc: open list:OPEN FIRMWARE AND...,
Pawel Moll, Ian Campbell, Rob Herring, linuxppc-dev,
linux-arm-kernel, gpkulkarni
On 2014-9-18 3:34, Mark Rutland wrote:
> On Wed, Sep 17, 2014 at 04:37:30PM +0100, Kumar Gala wrote:
>>
>> On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni <ganapatrao.kulkarni@caviumnetworks.com> wrote:
>>
>>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>>>
>>> This patch adds property "nid" to memory node to provide the memory range to
>>> numa node id mapping.
>>>
>>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
>>>
>>> —
>>
>> Adding the PPC guys as they’ve been doing NUMA on IBM Power Servers
>> for years with OF/DT. So we should really try and follow what they’ve
>> done.
>
> Agreed.
>
>>> Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>> 1 file changed, 58 insertions(+)
>>> create mode 100644 Documentation/devicetree/bindings/numa.txt
>>>
>>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>>> new file mode 100644
>>> index 0000000..c4a94f2
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/numa.txt
>>> @@ -0,0 +1,58 @@
>>> +======================================================
>>> +numa id binding description
>>> +======================================================
>>> +
>>> +======================================================
>>> +1 - Introduction
>>> +======================================================
>>> +The device node property "nid(numa node id)" can be added to memory
>
> Why the quotes?
>
>>> +device node to map the range of memory addresses as defined in property "reg".
>>> +The property "nid" maps the memory range to the numa node id, which is used to
>>> +find the local and remory pages on numa aware systems.
>
> What is a "numa node id", exactly, and how is the OS intended to use it?
I think "Proximity Domain" would be more suitably, processors and memory or IOs
in the same domain will have better performance than crossing other domains.
Thanks
Hanjun
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-10-28 7:29 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1410944189-3608-1-git-send-email-ganapatrao.kulkarni@caviumnetworks.com>
2014-09-17 15:37 ` [RFC PATCH] dt:numa: adding numa node mapping for memory nodes Kumar Gala
2014-09-17 19:34 ` Mark Rutland
2014-10-28 7:27 ` Hanjun Guo
[not found] ` <541A01B7.8030601@mentor.com>
[not found] ` <CAMDttNfSr5BrnEwLXqgy676jOZbFHhXcMtC0n7gBSniwhWCDRQ@mail.gmail.com>
[not found] ` <CAFpQJXUs+6Qvj0SqgajA8_D8UdEh_EQVgTQDz0bPSn-VpEty9g@mail.gmail.com>
2014-09-18 14:30 ` Ganapatrao Kulkarni
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).