linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* 3 ways to represent cpu affinity in /sys and counting
@ 2004-12-26  0:27 Anton Blanchard
  2004-12-30 19:40 ` Jesse Barnes
  2005-01-03 15:15 ` Andi Kleen
  0 siblings, 2 replies; 3+ messages in thread
From: Anton Blanchard @ 2004-12-26  0:27 UTC (permalink / raw)
  To: linux-kernel; +Cc: ak, greg, miltonm


Hi,

We have a patch to change pcibus_to_cpumask to pcibus_to_node. This makes
it more consistent with cpu_to_node, and when you want a cpumask you
use node_to_cpumask.

The one real user of pcibus_to_cpumask in arch/x86-64/kernel/pci-gart.c
highlights this:

	cpumask_t mask;
	mask = pcibus_to_cpumask(to_pci_dev(dev)->bus->number);
	node = cpu_to_node(first_cpu(mask));

We go around in circles to get back to the node. I then started looking
at how we export cpu affinity and node information in sysfs:

A pci device has a local_cpus property:

/sys/devices/pci000a:00/000a:00:02.6/local_cpus

A pci_bus has a cpuaffinity property:

/sys/class/pci_bus/000d:d8/cpuaffinity

A node has a cpumap property:

/sys/devices/system/node/node3/cpumap

Can we standardize on a single property name for this? :)

Furthermore, looking at node linkages:

A node has symlinks to cpus:

/sys/devices/system/node/node0/cpu0 -> /sys/devices/system/cpu/cpu0

But doesnt have symlinks to pci devices.

A cpu doesnt have a cpumask of its node, but a pci device and pci bus
both do. Is there some way we can stardardize this too? 

Ideally we want to be able to lookup a device -> node -> cpumask
relationship in a consistent way. Assuming we fix up the 3 different
names for cpu affinity properties, we still have 2 methods of looking
that up, and no easy way of going from pci devices to nodes.

Anton

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: 3 ways to represent cpu affinity in /sys and counting
  2004-12-26  0:27 3 ways to represent cpu affinity in /sys and counting Anton Blanchard
@ 2004-12-30 19:40 ` Jesse Barnes
  2005-01-03 15:15 ` Andi Kleen
  1 sibling, 0 replies; 3+ messages in thread
From: Jesse Barnes @ 2004-12-30 19:40 UTC (permalink / raw)
  To: Anton Blanchard; +Cc: linux-kernel, ak, greg, miltonm

On Saturday, December 25, 2004 4:27 pm, Anton Blanchard wrote:
> Hi,
>
> We have a patch to change pcibus_to_cpumask to pcibus_to_node. This makes
> it more consistent with cpu_to_node, and when you want a cpumask you
> use node_to_cpumask.

Great!  I think Matt Dobson said he was going to do something similar, but he 
got sidetracked (and iirc, ak also didn't like the idea).  FWIW, I like it, 
with the caveat that the node returned from pcibus_to_node may not have any 
memory or CPUs associated with it.

> A pci device has a local_cpus property:
>
> /sys/devices/pci000a:00/000a:00:02.6/local_cpus
>
> A pci_bus has a cpuaffinity property:
>
> /sys/class/pci_bus/000d:d8/cpuaffinity

I don't know how these two got different names...

> A node has a cpumap property:
>
> /sys/devices/system/node/node3/cpumap
>
> Can we standardize on a single property name for this? :)

Seems like nodes should have a cpumap and PCI busses should have a node.

> Furthermore, looking at node linkages:
>
> A node has symlinks to cpus:
>
> /sys/devices/system/node/node0/cpu0 -> /sys/devices/system/cpu/cpu0
>
> But doesnt have symlinks to pci devices.

Yep, this would be nice to have.

Jesse

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: 3 ways to represent cpu affinity in /sys and counting
  2004-12-26  0:27 3 ways to represent cpu affinity in /sys and counting Anton Blanchard
  2004-12-30 19:40 ` Jesse Barnes
@ 2005-01-03 15:15 ` Andi Kleen
  1 sibling, 0 replies; 3+ messages in thread
From: Andi Kleen @ 2005-01-03 15:15 UTC (permalink / raw)
  To: Anton Blanchard; +Cc: linux-kernel, ak, greg, miltonm

On Sun, Dec 26, 2004 at 11:27:44AM +1100, Anton Blanchard wrote:
> 
> A pci device has a local_cpus property:
> 
> /sys/devices/pci000a:00/000a:00:02.6/local_cpus

We should have both node and cpus here because both
are needed in user space (one for CPU affinity, the other
for memory affinity). While it's possible to convert
in user space it's quite cumbersome. 

I plan to add functions to libnuma to make this easy,
but I'm not sure everybody will want to use libnuma just
for this.

> 
> A pci_bus has a cpuaffinity property:
> 
> /sys/class/pci_bus/000d:d8/cpuaffinity

I'm not sure what that is anyways. I don't think x86-64 sets it up
and does anybody really use it?
> 
> A node has a cpumap property:
> 
> /sys/devices/system/node/node3/cpumap
> 
> Can we standardize on a single property name for this? :)

I suspect it's already too late. 

> 
> Furthermore, looking at node linkages:
> 
> A node has symlinks to cpus:
> 
> /sys/devices/system/node/node0/cpu0 -> /sys/devices/system/cpu/cpu0
> 
> But doesnt have symlinks to pci devices.

I'm not sure how useful that would be anyways.

> 
> A cpu doesnt have a cpumask of its node, but a pci device and pci bus
> both do. Is there some way we can stardardize this too? 

You mean a nodemask?

> 
> Ideally we want to be able to lookup a device -> node -> cpumask
> relationship in a consistent way. Assuming we fix up the 3 different
> names for cpu affinity properties, we still have 2 methods of looking
> that up, and no easy way of going from pci devices to nodes.

You mean the user space interface?

Getting it from sysfs will be always a mess, best is to have a standard
library that offers this in nice functions and a command whose output
can be parsed by shell scripts. I plan to add this eventually to 
numactl/libnuma.

-Andi

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2005-01-03 15:16 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-12-26  0:27 3 ways to represent cpu affinity in /sys and counting Anton Blanchard
2004-12-30 19:40 ` Jesse Barnes
2005-01-03 15:15 ` Andi Kleen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).