[rfc][patch] driverfs multi-node(board) patch [1/2]

[rfc][patch] driverfs multi-node(board) patch [1/2]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Patrick,
	Here's the pre-reqs to my patch.  I put in a memblk_online_map (just like 
cpu_online_map) and associated functions (memblk_online(), etc).
	Also there's a include/linux/topology.h that has some wrappers for the 
in-kernel macros.  Just does bound checking, etc.

	This patch sits on top of 2.5.39-mm1 (which includes my in-kernel 
topology API).  The *real* driverfs additions will sit on top of this 
patch...  Coming momentarily...

Cheers!

-Matt

[-- Attachment #2: driverfs-pre_req-2.5.39.patch --]
[-- Type: text/plain, Size: 8050 bytes --]

diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/arch/i386/kernel/smpboot.c linux-2.5.39-driverfs_pre-reqs/arch/i386/kernel/smpboot.c
--- linux-2.5.39-vanilla/arch/i386/kernel/smpboot.c	Fri Sep 27 14:49:54 2002
+++ linux-2.5.39-driverfs_pre-reqs/arch/i386/kernel/smpboot.c	Mon Sep 30 15:57:51 2002
@@ -62,6 +62,9 @@
 /* Bitmask of currently online CPUs */
 unsigned long cpu_online_map;
 
+/* Bitmask of currently online memory blocks */
+unsigned long memblk_online_map = 0UL;
+
 static volatile unsigned long cpu_callin_map;
 volatile unsigned long cpu_callout_map;
 static unsigned long smp_commenced_mask;
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/drivers/base/core.c linux-2.5.39-driverfs_pre-reqs/drivers/base/core.c
--- linux-2.5.39-vanilla/drivers/base/core.c	Fri Sep 27 14:49:03 2002
+++ linux-2.5.39-driverfs_pre-reqs/drivers/base/core.c	Mon Sep 30 15:57:44 2002
@@ -68,7 +68,7 @@
 }
 
 /**
- * device_attach - try to associated device with a driver
+ * do_device_attach - try to associate device with a driver
  * @drv:	current driver to try
  * @data:	device in disguise
  *
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/include/asm-i386/smp.h linux-2.5.39-driverfs_pre-reqs/include/asm-i386/smp.h
--- linux-2.5.39-vanilla/include/asm-i386/smp.h	Fri Sep 27 14:49:02 2002
+++ linux-2.5.39-driverfs_pre-reqs/include/asm-i386/smp.h	Mon Sep 30 15:57:51 2002
@@ -54,6 +54,7 @@
 extern void smp_alloc_memory(void);
 extern unsigned long phys_cpu_present_map;
 extern unsigned long cpu_online_map;
+extern unsigned long memblk_online_map;
 extern volatile unsigned long smp_invalidate_needed;
 extern int pic_mode;
 extern int smp_num_siblings;
@@ -102,6 +103,13 @@
 	return -1;
 }
 
+#define memblk_online(memblk) (memblk_online_map & (1<<(memblk)))
+ 
+extern inline unsigned int num_online_memblks(void)
+{
+	return hweight32(memblk_online_map);
+}
+
 static __inline int hard_smp_processor_id(void)
 {
 	/* we don't want to mark this access volatile - bad code generation */
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/include/linux/smp.h linux-2.5.39-driverfs_pre-reqs/include/linux/smp.h
--- linux-2.5.39-vanilla/include/linux/smp.h	Fri Sep 27 14:50:17 2002
+++ linux-2.5.39-driverfs_pre-reqs/include/linux/smp.h	Mon Sep 30 15:57:51 2002
@@ -94,7 +94,10 @@
 #define cpu_online(cpu)				({ BUG_ON((cpu) != 0); 1; })
 #define num_online_cpus()			1
 #define num_booting_cpus()			1
-#define cpu_possible(cpu)				({ BUG_ON((cpu) != 0); 1; })
+#define cpu_possible(cpu)			({ BUG_ON((cpu) != 0); 1; })
+#define memblk_online_map			1
+#define memblk_online(memblk)			({ BUG_ON((memblk) != 0); 1; })
+#define num_online_memblks()			1
 
 struct notifier_block;
 
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/include/linux/topology.h linux-2.5.39-driverfs_pre-reqs/include/linux/topology.h
--- linux-2.5.39-vanilla/include/linux/topology.h	Wed Dec 31 16:00:00 1969
+++ linux-2.5.39-driverfs_pre-reqs/include/linux/topology.h	Mon Sep 30 15:57:48 2002
@@ -0,0 +1,155 @@
+/*
+ * linux/include/linux/topology.h
+ *
+ * Written by: Matthew Dobson, IBM Corporation
+ *
+ * Copyright (C) 2002, IBM Corp.
+ *
+ * All rights reserved.          
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT.  See the GNU General Public License for more
+ * details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ * Send feedback to <colpatch@us.ibm.com>
+ */
+#include <linux/kernel.h>
+#include <linux/unistd.h>
+#include <linux/config.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/smp.h>
+#include <linux/mmzone.h>
+
+#include <asm/topology.h>
+
+extern int numnodes;
+
+/*
+ * cpu_to_node(cpu): Returns the number of the most specific Node 
+ * containing CPU 'cpu'.
+ */
+static inline int cpu_to_node(int cpu)
+{
+	int node;
+
+	if (cpu == -1)     /* returns highest numbered node */
+		return (numnodes - 1);
+
+	if ((cpu < 0) || (cpu >= NR_CPUS) || (!cpu_online(cpu)) || 
+	    ((node = __cpu_to_node(cpu)) < 0))  /* invalid cpu */
+		return -ENODEV;
+
+	return node;
+}
+
+/*
+ * memblk_to_node(memblk): Returns the number of the most specific Node 
+ * containing Memory Block 'memblk'.
+ */
+static inline int memblk_to_node(int memblk)
+{
+	int node;
+
+	if (memblk == -1)  /* return highest numbered node */
+		return (numnodes - 1);
+
+	if ((memblk < 0) || (memblk >= MAX_NR_MEMBLKS) || (!memblk_online(memblk)) || 
+	    ((node = __memblk_to_node(memblk)) < 0))  /* invalid memblk # */
+		return -ENODEV;
+
+	return node;
+}
+
+/*
+ * parent_node(node): Returns the number of the of the most specific Node that
+ * encompasses Node 'node'.  Some may call this the parent Node of 'node'.
+ */
+static inline int parent_node(int node)
+{
+	if ((node < 0) || (node >= MAX_NR_NODES))  /* invalid node # */
+		return -ENODEV;
+
+	return __parent_node(node);
+}
+
+/*
+ * node_to_first_cpu(node): Returns the lowest numbered CPU on Node 'node'
+ */
+static inline int node_to_first_cpu(int node)
+{
+	int cpu;
+
+	if (node == -1)  /* return highest numbered cpu */
+		return (num_online_cpus() - 1);
+
+	if ((node < 0) || (node >= MAX_NR_NODES) ||
+	    ((cpu = __node_to_first_cpu(node)) < 0))  /* invalid node # */
+		return -ENODEV;
+
+	return cpu;
+}
+
+/*
+ * node_to_cpu_mask(node): Returns a bitmask of CPUs on Node 'node'
+ */
+static inline unsigned long node_to_cpu_mask(int node)
+{
+	if (node == -1)  /* return highest numbered cpu */
+		return (num_online_cpus() - 1);
+
+	if ((node < 0) || (node >= MAX_NR_NODES))  /* invalid node # */
+		return -ENODEV;
+
+	return __node_to_cpu_mask(node);
+}
+
+/*
+ * node_to_memblk(node): Returns the lowest numbered MemBlk on Node 'node'
+ */
+static inline int node_to_memblk(int node)
+{
+	int memblk;
+
+	if (node == -1)  /* return highest numbered memblk */
+		return (num_online_memblks() - 1);
+
+	if ((node < 0) || (node >= MAX_NR_NODES) ||
+	    ((memblk = __node_to_memblk(node)) < 0))  /* invalid node # */
+		return -ENODEV;
+
+	return memblk;
+}
+
+/*
+ * get_curr_cpu(): Returns the currently executing CPU number.
+ * For now, this has only mild usefulness, as this information could
+ * change on the return from syscall (which automatically calls schedule()).
+ * Due to this, the data could be stale by the time it gets back to the user.
+ * It will have to do, until a better method is found.
+ */
+static inline int get_curr_cpu(void)
+{
+	return smp_processor_id();
+}
+
+/*
+ * get_curr_node(): Returns the number of the Node containing 
+ * the currently executing CPU.  Subject to the same caveat
+ * as the get_curr_cpu() call.
+ */
+static inline int get_curr_node(void)
+{
+	return cpu_to_node(get_curr_cpu());
+}
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-vanilla/mm/numa.c linux-2.5.39-driverfs_pre-reqs/mm/numa.c
--- linux-2.5.39-vanilla/mm/numa.c	Fri Sep 27 14:50:29 2002
+++ linux-2.5.39-driverfs_pre-reqs/mm/numa.c	Mon Sep 30 15:57:51 2002
@@ -37,6 +37,7 @@
 	}
 	contig_page_data.node_mem_map = pmap;
 	free_area_init_core(&contig_page_data, zones_size, zholes_size);
+	memblk_online_map = 1UL;
 	mem_map = contig_page_data.node_mem_map;
 }
 
@@ -74,6 +75,10 @@
 	}
 	pgdat->node_mem_map = pmap;
 	free_area_init_core(pgdat, zones_size, zholes_size);
+	if (test_and_set_bit(num_online_memblks(), &memblk_online_map)){
+		printk("free_area_init_core: memblk alread counted?!?!\n");
+		BUG();
+	}
 
 	/*
 	 * Get space for the valid bitmap.

[rfc][patch] driverfs multi-node(board) patch [2/2]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 391 bytes --]

Patrick,
	Ok..  here are the real changes.  I'd really like to get some feedback on 
what you (or anyone else) thinks of these proposed changes.  This sets 
up a generic topology initialization routine which should discover all 
online nodes (boards), CPUs, and Memory Blocks at boot time.  It also 
makes the CPUs and memblks it discovers children of the appropriate nodes.

Cheers!

-Matt

[-- Attachment #2: driverfs-additions-2.5.39.patch --]
[-- Type: text/plain, Size: 9836 bytes --]

diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/arch/i386/kernel/cpu/common.c linux-2.5.39-mm1+pre_reqs+additions/arch/i386/kernel/cpu/common.c
--- linux-2.5.39-mm1+pre_reqs/arch/i386/kernel/cpu/common.c	Fri Sep 27 14:49:02 2002
+++ linux-2.5.39-mm1+pre_reqs+additions/arch/i386/kernel/cpu/common.c	Mon Sep 30 14:37:50 2002
@@ -1,7 +1,6 @@
 #include <linux/init.h>
 #include <linux/string.h>
 #include <linux/delay.h>
-#include <linux/cpu.h>
 #include <linux/smp.h>
 #include <asm/semaphore.h>
 #include <asm/processor.h>
@@ -507,37 +506,3 @@
 	current->used_math = 0;
 	stts();
 }
-
-/*
- * Bulk registration of the cpu devices with the system.
- * Some of this stuff could possibly be moved into a shared 
- * location..
- * Also, these devices should be integrated with other CPU data..
- */
-
-static struct cpu cpu_devices[NR_CPUS];
-
-static struct device_driver cpu_driver = {
-	.name		= "cpu",
-	.bus		= &system_bus_type,
-	.devclass	= &cpu_devclass,
-};
-
-static int __init register_cpus(void)
-{
-	int i;
-
-	driver_register(&cpu_driver);
-
-	for (i = 0; i < NR_CPUS; i++) {
-		struct sys_device * sysdev = &cpu_devices[i].sysdev;
-		sysdev->name = "cpu";
-		sysdev->id = i;
-		sysdev->dev.driver = &cpu_driver;
-		if (cpu_possible(i))
-			sys_device_register(sysdev);
-	}
-	return 0;
-}
-
-subsys_initcall(register_cpus);
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/Makefile linux-2.5.39-mm1+pre_reqs+additions/drivers/base/Makefile
--- linux-2.5.39-mm1+pre_reqs/drivers/base/Makefile	Fri Sep 27 14:49:17 2002
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/Makefile	Mon Sep 30 14:37:50 2002
@@ -2,13 +2,14 @@
 
 obj-y		:= core.o sys.o interface.o power.o bus.o \
 			driver.o class.o intf.o platform.o \
-			cpu.o
+			node.o cpu.o memblk.o topology.o
 
 obj-y		+= fs/
 
 obj-$(CONFIG_HOTPLUG)	+= hotplug.o
 
 export-objs	:= core.o power.o sys.o bus.o driver.o \
-			class.o intf.o platform.o cpu.o 
+			class.o intf.o platform.o node.o \
+			cpu.o memblk.o
 
 include $(TOPDIR)/Rules.make
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/cpu.c linux-2.5.39-mm1+pre_reqs+additions/drivers/base/cpu.c
--- linux-2.5.39-mm1+pre_reqs/drivers/base/cpu.c	Fri Sep 27 14:49:42 2002
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/cpu.c	Mon Sep 30 14:37:50 2002
@@ -5,7 +5,6 @@
 #include <linux/device.h>
 #include <linux/module.h>
 #include <linux/init.h>
-#include <linux/cpu.h>
 
 static int cpu_add_device(struct device * dev)
 {
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/memblk.c linux-2.5.39-mm1+pre_reqs+additions/drivers/base/memblk.c
--- linux-2.5.39-mm1+pre_reqs/drivers/base/memblk.c	Wed Dec 31 16:00:00 1969
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/memblk.c	Mon Sep 30 14:37:50 2002
@@ -0,0 +1,27 @@
+/*
+ * memblk.c - basic memblk class support
+ */
+
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/init.h>
+
+static int memblk_add_device(struct device * dev)
+{
+	return 0;
+}
+
+struct device_class memblk_devclass = {
+	.name		= "memblk",
+	.add_device	= memblk_add_device,
+};
+
+
+static int __init memblk_devclass_init(void)
+{
+	return devclass_register(&memblk_devclass);
+}
+
+postcore_initcall(memblk_devclass_init);
+
+EXPORT_SYMBOL(memblk_devclass);
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/node.c linux-2.5.39-mm1+pre_reqs+additions/drivers/base/node.c
--- linux-2.5.39-mm1+pre_reqs/drivers/base/node.c	Wed Dec 31 16:00:00 1969
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/node.c	Mon Sep 30 14:37:50 2002
@@ -0,0 +1,27 @@
+/*
+ * node.c - basic node class support
+ */
+
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/init.h>
+
+static int node_add_device(struct device * dev)
+{
+	return 0;
+}
+
+struct device_class node_devclass = {
+	.name		= "node",
+	.add_device	= node_add_device,
+};
+
+
+static int __init node_devclass_init(void)
+{
+	return devclass_register(&node_devclass);
+}
+
+postcore_initcall(node_devclass_init);
+
+EXPORT_SYMBOL(node_devclass);
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/sys.c linux-2.5.39-mm1+pre_reqs+additions/drivers/base/sys.c
--- linux-2.5.39-mm1+pre_reqs/drivers/base/sys.c	Fri Sep 27 14:48:33 2002
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/sys.c	Mon Sep 30 14:37:50 2002
@@ -55,6 +55,9 @@
 
 	pr_debug("Registering system board %d\n",root->id);
 
+	if (!root->dev.parent)
+		root->dev.parent = &system_bus;
+
 	error = device_register(&root->dev);
 	if (!error) {
 		strncpy(root->sysdev.bus_id,"sys",BUS_ID_SIZE);
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.39-mm1+pre_reqs/drivers/base/topology.c linux-2.5.39-mm1+pre_reqs+additions/drivers/base/topology.c
--- linux-2.5.39-mm1+pre_reqs/drivers/base/topology.c	Wed Dec 31 16:00:00 1969
+++ linux-2.5.39-mm1+pre_reqs+additions/drivers/base/topology.c	Mon Sep 30 14:37:50 2002
@@ -0,0 +1,175 @@
+/*
+ * drivers/base/topology.c - Populate driverfs with topology information
+ *
+ * Written by: Matthew Dobson, IBM Corporation
+ * Original Code: Paul Dorwin, IBM Corporation, Patrick Mochel, OSDL
+ *
+ * Copyright (C) 2002, IBM Corp.
+ *
+ * All rights reserved.          
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT.  See the GNU General Public License for more
+ * details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ * Send feedback to <colpatch@us.ibm.com>
+ */
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/topology.h>
+
+extern struct device_class node_devclass;
+extern struct device_class cpu_devclass;
+extern struct device_class memblk_devclass;
+
+/*
+ * Node devices
+ */
+struct node {
+	unsigned long cpumask;	/* Mask of CPUs on the node */
+	struct sys_root sysroot;
+};
+static struct node node_devices[MAX_NR_NODES];
+
+static struct device_driver node_driver = {
+	.name		= "node",
+	.bus		= &system_bus_type,
+	.devclass	= &node_devclass,
+};
+
+/*
+ * register_node - Setup a driverfs device for a node.
+ * @num - Node number to use when creating the device.
+ *
+ * Check to see if the node is already initialized.
+ * Initialize, and register the node device.
+ */
+static void __init register_node(int num)
+{
+	int parent;
+	struct sys_root *node = &node_devices[num].sysroot;
+
+	node_devices[num].cpumask = node_to_cpu_mask(num);
+	node->id = num;
+	if ((parent = parent_node(num)) != num)
+		node->dev.parent = &node_devices[parent].sysroot.sysdev;
+	snprintf(node->dev.name, DEVICE_NAME_SIZE, "Node %u", num);
+	snprintf(node->dev.bus_id, BUS_ID_SIZE, "node%u", num);
+	node->dev.bus = &system_bus_type;
+	node->dev.driver = &node_driver;
+	sys_register_root(node);
+}
+
+
+/*
+ * CPU devices
+ */
+struct cpu {
+	int node_id;	/* The node which contains the CPU */
+	struct sys_device sysdev;
+};
+static struct cpu cpu_devices[NR_CPUS];
+
+static struct device_driver cpu_driver = {
+	.name		= "cpu",
+	.bus		= &system_bus_type,
+	.devclass	= &cpu_devclass,
+};
+
+/*
+ * register_cpu - Setup a driverfs device for a CPU.
+ * @num - CPU number to use when creating the device.
+ *
+ * Check to see if the CPU is already initialized.
+ * Initialize, and register the CPU device.
+ * Also, these devices should be integrated with other CPU data..
+ */
+static void __init register_cpu(int num)
+{
+	int node_id;
+	struct sys_device *cpu = &cpu_devices[num].sysdev;
+
+	if ((node_id = cpu_to_node(num)) < 0)
+		BUG();
+	cpu_devices[num].node_id = node_id;
+	cpu->name = "cpu";
+	cpu->id = num;
+	cpu->root = &node_devices[node_id].sysroot;
+	snprintf(cpu->dev.name, DEVICE_NAME_SIZE, "CPU %u", num);
+	cpu->dev.driver = &cpu_driver;
+	sys_register_device(cpu);
+}
+
+
+/*
+ * MemBlk devices
+ */
+struct memblk {
+	int node_id;	/* The node which contains the MemBlk */
+	struct sys_device sysdev;
+};
+static struct memblk memblk_devices[MAX_NR_MEMBLKS];
+
+static struct device_driver memblk_driver = {
+	.name		= "memblk",
+	.bus		= &system_bus_type,
+	.devclass	= &memblk_devclass,
+};
+
+/*
+ * register_memblk - Setup a driverfs device for a MemBlk
+ * @num: MemBlk number to use when creating the device.
+ *
+ * Initialize, and register the MemBlk device.
+ */
+static __init void register_memblk(int num)
+{
+	int node_id;
+	struct sys_device *memblk = &memblk_devices[num].sysdev;
+
+	if ((node_id = memblk_to_node(num)) < 0)
+		BUG();
+	memblk_devices[num].node_id = node_id;
+	memblk->name = "memblk";
+	memblk->id = num;
+	memblk->root = &node_devices[node_id].sysroot;
+	snprintf(memblk->dev.name, DEVICE_NAME_SIZE, "Memory Block %u", num);
+	memblk->dev.driver = &memblk_driver;
+	sys_register_device(memblk);
+}
+
+
+/*
+ * Populate driverfs with the system topology
+ */
+static int __init topology_init(void)
+{
+	int id;
+
+	driver_register(&node_driver);
+	driver_register(&cpu_driver);
+	driver_register(&memblk_driver);
+
+	for (id = 0; id < numnodes; id++)
+		register_node(id);
+	for (id = 0; id < num_online_cpus(); id++)
+		register_cpu(id);
+	for (id = 0; id < num_online_memblks(); id++)
+		register_memblk(id);
+	return 0;
+}
+
+subsys_initcall(topology_init);

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Greg KH]

On Mon, Sep 30, 2002 at 06:03:12PM -0700, Matthew Dobson wrote:
> Patrick,
> 	Ok..  here are the real changes.  I'd really like to get some 
> 	feedback on what you (or anyone else) thinks of these proposed changes.  
> This sets up a generic topology initialization routine which should 
> discover all online nodes (boards), CPUs, and Memory Blocks at boot time.  
> It also makes the CPUs and memblks it discovers children of the appropriate 
> nodes.

Can you show an example output of what the directory structure now looks
like with this patch?

Curious,

greg k-h

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Matthew Dobson]

Greg KH wrote:
> Can you show an example output of what the directory structure now looks
> like with this patch?
> 
> Curious,
> 
> greg k-h
Surely, Greg!  Something I definitely should have put in the original 
post...  here's the before:

*****************BEFORE****************************
[root@elm3b79 devices]# tree -d bus/system/devices/
bus/system/devices/
|-- cpu0 -> ../../../root/sys/cpu0
|-- cpu1 -> ../../../root/sys/cpu1
|-- cpu2 -> ../../../root/sys/cpu2
|-- cpu3 -> ../../../root/sys/cpu3
|-- cpu4 -> ../../../root/sys/cpu4
|-- cpu5 -> ../../../root/sys/cpu5
|-- cpu6 -> ../../../root/sys/cpu6
|-- cpu7 -> ../../../root/sys/cpu7
|-- pic0 -> ../../../root/sys/pic0
`-- rtc0 -> ../../../root/sys/rtc0

10 directories
[root@elm3b79 devices]# tree -d class/
class/
|-- cpu
|   |-- devices
|   |   |-- 0 -> ../../../root/sys/cpu0
|   |   |-- 1 -> ../../../root/sys/cpu1
|   |   |-- 2 -> ../../../root/sys/cpu2
|   |   |-- 3 -> ../../../root/sys/cpu3
|   |   |-- 4 -> ../../../root/sys/cpu4
|   |   |-- 5 -> ../../../root/sys/cpu5
|   |   |-- 6 -> ../../../root/sys/cpu6
|   |   `-- 7 -> ../../../root/sys/cpu7
|   `-- drivers
|-- disk
|   |-- devices
|   `-- drivers
`-- input
     |-- devices
     `-- drivers

17 directories
[root@elm3b79 devices]# tree -d root/sys/
root/sys/
|-- cpu0
|-- cpu1
|-- cpu2
|-- cpu3
|-- cpu4
|-- cpu5
|-- cpu6
|-- cpu7
|-- pic0
`-- rtc0

10 directories
*****************BEFORE****************************

And here is the output after my changes:
******************AFTER****************************
[root@elm3b79 devices]# tree -d bus/system/devices/
bus/system/devices/
|-- cpu0 -> ../../../root/sys/node0/sys/cpu0
|-- cpu1 -> ../../../root/sys/node0/sys/cpu1
|-- cpu2 -> ../../../root/sys/node0/sys/cpu2
|-- cpu3 -> ../../../root/sys/node0/sys/cpu3
|-- cpu4 -> ../../../root/sys/node1/sys/cpu4
|-- cpu5 -> ../../../root/sys/node1/sys/cpu5
|-- cpu6 -> ../../../root/sys/node1/sys/cpu6
|-- cpu7 -> ../../../root/sys/node1/sys/cpu7
|-- memblk0 -> ../../../root/sys/node0/sys/memblk0
|-- memblk1 -> ../../../root/sys/node1/sys/memblk1
|-- node0 -> ../../../root/sys/node0
|-- node1 -> ../../../root/sys/node1
|-- pic0 -> ../../../root/sys/pic0
`-- rtc0 -> ../../../root/sys/rtc0

14 directories
[root@elm3b79 devices]# tree -d class/
class/
|-- cpu
|   |-- devices
|   |   |-- 0 -> ../../../root/sys/node0/sys/cpu0
|   |   |-- 1 -> ../../../root/sys/node0/sys/cpu1
|   |   |-- 2 -> ../../../root/sys/node0/sys/cpu2
|   |   |-- 3 -> ../../../root/sys/node0/sys/cpu3
|   |   |-- 4 -> ../../../root/sys/node1/sys/cpu4
|   |   |-- 5 -> ../../../root/sys/node1/sys/cpu5
|   |   |-- 6 -> ../../../root/sys/node1/sys/cpu6
|   |   `-- 7 -> ../../../root/sys/node1/sys/cpu7
|   `-- drivers
|-- disk
|   |-- devices
|   `-- drivers
|-- input
|   |-- devices
|   `-- drivers
|-- memblk
|   |-- devices
|   |   |-- 0 -> ../../../root/sys/node0/sys/memblk0
|   |   `-- 1 -> ../../../root/sys/node1/sys/memblk1
|   `-- drivers
`-- node
     |-- devices
     |   |-- 0 -> ../../../root/sys/node0
     |   `-- 1 -> ../../../root/sys/node1
     `-- drivers

27 directories
[root@elm3b79 devices]# tree -d root/sys/
root/sys/
|-- node0
|   `-- sys
|       |-- cpu0
|       |-- cpu1
|       |-- cpu2
|       |-- cpu3
|       `-- memblk0
|-- node1
|   `-- sys
|       |-- cpu4
|       |-- cpu5
|       |-- cpu6
|       |-- cpu7
|       `-- memblk1
|-- pic0
`-- rtc0

16 directories
******************AFTER****************************

Basically, the patch just adds nodes and memblks to the topology and 
nests the cpus/memblks under the nodes.  I'd like to add more 
information to these directories (node-node distances, cpu speeds, 
memory block sizes/physical page ranges, etc, etc, etc), but this is 
just a first-pass.

Cheers!

-Matt

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Patrick Mochel]

Matt,

I have some comments about the structure of the code, but those will come 
in another email..

> [root@elm3b79 devices]# tree -d root/sys/
> root/sys/
> |-- node0
> |   `-- sys
> |       |-- cpu0
> |       |-- cpu1
> |       |-- cpu2
> |       |-- cpu3
> |       `-- memblk0
> |-- node1
> |   `-- sys
> |       |-- cpu4
> |       |-- cpu5
> |       |-- cpu6
> |       |-- cpu7
> |       `-- memblk1
> |-- pic0
> `-- rtc0

Shouldn't nodes (or, erm, boards) be added as children of the root? 
Aren't all types of devices present on the various boards (PCI, etc)?

	-pat

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Matthew Dobson]

Patrick Mochel wrote:
> Matt,
> 
> I have some comments about the structure of the code, but those will come 
> in another email..
Cool...  Any/all comments are definitely welcome...

>>[root@elm3b79 devices]# tree -d root/sys/
>>root/sys/
>>|-- node0
>>|   `-- sys
>>|       |-- cpu0
>>|       |-- cpu1
>>|       |-- cpu2
>>|       |-- cpu3
>>|       `-- memblk0
>>|-- node1
>>|   `-- sys
>>|       |-- cpu4
>>|       |-- cpu5
>>|       |-- cpu6
>>|       |-- cpu7
>>|       `-- memblk1
>>|-- pic0
>>`-- rtc0
> 
> 
> Shouldn't nodes (or, erm, boards) be added as children of the root? 
Um, yes!  I don't quite know how, though...  I call sys_register_root() 
for each of the nodes...  That call seems to parent them under the 
root/sys directory...  How can I change that?

> Aren't all types of devices present on the various boards (PCI, etc)?
Yes...  I have some patches that I'm working that will put PCI busses 
and devices into the topology infrastructure (both in-kernel & via 
driverfs).  Again, this is just a first pass of what I'd like to see... ;)

Cheers!

-Matt


> 
> 	-pat
> 
> 

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Patrick Mochel]

> > I have some comments about the structure of the code, but those will come 
> > in another email..
> Cool...  Any/all comments are definitely welcome...

You'll have to wait prolly 1 more day, though. Sorry..

> > Shouldn't nodes (or, erm, boards) be added as children of the root? 
> Um, yes!  I don't quite know how, though...  I call sys_register_root() 
> for each of the nodes...  That call seems to parent them under the 
> root/sys directory...  How can I change that?

Woops. That sounds like my booboo. I'll look into that.

> > Aren't all types of devices present on the various boards (PCI, etc)?
> Yes...  I have some patches that I'm working that will put PCI busses 
> and devices into the topology infrastructure (both in-kernel & via 
> driverfs).  Again, this is just a first pass of what I'd like to see... ;)

Sweet, that should be cool.

	-pat

Re: [rfc][patch] driverfs multi-node(board) patch [2/2]

[Patrick Mochel]

Ok, I'm finally getting back to you..

> 	Ok..  here are the real changes.  I'd really like to get some feedback on 
> what you (or anyone else) thinks of these proposed changes.  This sets 
> up a generic topology initialization routine which should discover all 
> online nodes (boards), CPUs, and Memory Blocks at boot time.  It also 
> makes the CPUs and memblks it discovers children of the appropriate nodes.

You didn't append the patch, which is annoying, but I'll deal..

The main problem I have is the code placement. I put the CPU stuff under 
arch/ because I anticpate wrapping the cpu structure with an arch-specific 
one, so you can ascertain arch-specific information via the generic 
structure. Moving it out of arch/ precludes that from happening (easily). 

Ditto for memblks, though I'm not really sure what other info you'd want 
in the structures. 

Ditto+ for nodes, or boards. Those are definitely arch-specific 
structures, and shouldn't be in drivers/base/. On top of that, I don't 
think their registration should all be munged together in one file. Maybe 
they should be, in their own play area (under arch/i386/mach-ccnuma/).


	-pat