From: shaohui.zheng@intel.com To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, haicheng.li@linux.intel.com, lethal@linux-sh.org, ak@linux.intel.com, shaohui.zheng@linux.intel.com, Haicheng Li <haicheng.li@intel.com>, Shaohui Zheng <shaohui.zheng@intel.com> Subject: [8/8,v3] NUMA Hotplug Emulator: documentation Date: Wed, 17 Nov 2010 10:08:07 +0800 [thread overview] Message-ID: <20101117021000.985643862@intel.com> (raw) In-Reply-To: 20101117020759.016741414@intel.com [-- Attachment #1: 008-hotplug-emulator-doc-x86_64-of-numa-hotplug-emulator.patch --] [-- Type: text/plain, Size: 4784 bytes --] From: Shaohui Zheng <shaohui.zheng@intel.com> add a text file Documentation/x86/x86_64/numa_hotplug_emulator.txt to explain the usage for the hotplug emulator. Signed-off-by: Haicheng Li <haicheng.li@intel.com> Signed-off-by: Shaohui Zheng <shaohui.zheng@intel.com> --- Index: linux-hpe4/Documentation/x86/x86_64/numa_hotplug_emulator.txt =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linux-hpe4/Documentation/x86/x86_64/numa_hotplug_emulator.txt 2010-11-17 09:01:10.342836513 +0800 @@ -0,0 +1,92 @@ +NUMA Hotplug Emulator for x86 +--------------------------------------------------- + +NUMA hotplug emulator is able to emulate NUMA Node Hotplug +thru a pure software way. It intends to help people easily debug +and test node/cpu/memory hotplug related stuff on a +none-numa-hotplug-support machine, even a UMA machine and virtual +environment. + +1) Node hotplug emulation: + +The emulator firstly hides RAM via E820 table, and then it can +fake offlined nodes with the hidden RAM. + +After system bootup, user is able to hotplug-add these offlined +nodes, which is just similar to a real hotplug hardware behavior. + +Using boot option "numa=hide=N*size" to fake offlined nodes: + - N is the number of hidden nodes + - size is the memory size (in MB) per hidden node. + +There is a sysfs entry "probe" under /sys/devices/system/node/ for user +to hotplug the fake offlined nodes: + + - to show all fake offlined nodes: + $ cat /sys/devices/system/node/probe + + - to hotadd a fake offlined node, e.g. nodeid is N: + $ echo N > /sys/devices/system/node/probe + +2) CPU hotplug emulation: + +The emulator reserve CPUs throu grub parameter, the reserved CPUs can be +hot-add/hot-remove in software method, it emulates the process of physical +cpu hotplug. + +When hotplug a CPU with emulator, we are using a logical CPU to emulate the CPU +socket hotplug process. For the CPU supported SMT, some logical CPUs are in the +same socket, but it may located in different NUMA node after we have emulator. +We put the logical CPU into a fake CPU socket, and assign it an unique +phys_proc_id. For the fake socket, we put one logical CPU in only. + + - to hide CPUs + - Using boot option "maxcpus=N" hide CPUs + N is the number of initialize CPUs + - Using boot option "cpu_hpe=on" to enable cpu hotplug emulation + when cpu_hpe is enabled, the rest CPUs will not be initialized + + - to hot-add CPU to node + $ echo nid > cpu/probe + + - to hot-remove CPU + $ echo nid > cpu/release + +3) Memory hotplug emulation: + +The emulator reserve memory before OS booting, the reserved memory region +is remove from e820 table, and they can be hot-added via the probe interface, +this interface was extend to support add memory to the specified node, It +maintains backwards compatibility. + +The difficulty of Memory Release is well-known, we have no plan for it until now. + + - reserve memory throu grub parameter + mem=1024m + + - add a memory section to node 3 + $ echo 0x40000000,3 > memory/probe + OR + $ echo 1024m,3 > memory/probe + OR + $ echo "physical_address=0x40000000 numa_node=3" > memory/probe + +4) Script for hotplug testing + +These scripts provides convenience when we hot-add memory/cpu in batch. + +- Online all memory sections: +for m in /sys/devices/system/memory/memory*; +do + echo online > $m/state; +done + +- CPU Online: +for c in /sys/devices/system/cpu/cpu*; +do + echo 1 > $c/online; +done + +- Haicheng Li <haicheng.li@intel.com> +- Shaohui Zheng <shaohui.zheng@intel.com> + Nov 2010 Index: linux-hpe4/Documentation/x86/x86_64/boot-options.txt =================================================================== --- linux-hpe4.orig/Documentation/x86/x86_64/boot-options.txt 2010-11-17 10:01:37.093461435 +0800 +++ linux-hpe4/Documentation/x86/x86_64/boot-options.txt 2010-11-17 10:03:10.881043878 +0800 @@ -173,6 +173,13 @@ numa=fake=<N> If given as an integer, fills all system RAM with N fake nodes interleaved over physical nodes. + numa=hide=N*size1[,size2,...] + Give an string seperated by comma, each sub string stands for a serie nodes. + system will reserve an area to create hide numa nodes for them. + + for example: numa=hide=2*512,256 + system will reserve (2*512 + 256) M for 3 hide nodes. 2 nodes with 512M memory, + and 1 node with 256 memory ACPI @@ -316,3 +323,8 @@ Do not use GB pages for kernel direct mappings. gbpages Use GB pages for kernel direct mappings. + cpu_hpe=on/off + Enable/disable cpu hotplug emulation with software method. when cpu_hpe=on, + sysfs provides probe/release interface to hot add/remove cpu dynamically. + this option is disabled in default. + -- Thanks & Regards, Shaohui
WARNING: multiple messages have this Message-ID (diff)
From: shaohui.zheng@intel.com To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, haicheng.li@linux.intel.com, lethal@linux-sh.org, ak@linux.intel.com, shaohui.zheng@linux.intel.com, Haicheng Li <haicheng.li@intel.com>, Shaohui Zheng <shaohui.zheng@intel.com> Subject: [8/8,v3] NUMA Hotplug Emulator: documentation Date: Wed, 17 Nov 2010 10:08:07 +0800 [thread overview] Message-ID: <20101117021000.985643862@intel.com> (raw) In-Reply-To: 20101117020759.016741414@intel.com [-- Attachment #1: 008-hotplug-emulator-doc-x86_64-of-numa-hotplug-emulator.patch --] [-- Type: text/plain, Size: 5080 bytes --] From: Shaohui Zheng <shaohui.zheng@intel.com> add a text file Documentation/x86/x86_64/numa_hotplug_emulator.txt to explain the usage for the hotplug emulator. Signed-off-by: Haicheng Li <haicheng.li@intel.com> Signed-off-by: Shaohui Zheng <shaohui.zheng@intel.com> --- Index: linux-hpe4/Documentation/x86/x86_64/numa_hotplug_emulator.txt =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linux-hpe4/Documentation/x86/x86_64/numa_hotplug_emulator.txt 2010-11-17 09:01:10.342836513 +0800 @@ -0,0 +1,92 @@ +NUMA Hotplug Emulator for x86 +--------------------------------------------------- + +NUMA hotplug emulator is able to emulate NUMA Node Hotplug +thru a pure software way. It intends to help people easily debug +and test node/cpu/memory hotplug related stuff on a +none-numa-hotplug-support machine, even a UMA machine and virtual +environment. + +1) Node hotplug emulation: + +The emulator firstly hides RAM via E820 table, and then it can +fake offlined nodes with the hidden RAM. + +After system bootup, user is able to hotplug-add these offlined +nodes, which is just similar to a real hotplug hardware behavior. + +Using boot option "numa=hide=N*size" to fake offlined nodes: + - N is the number of hidden nodes + - size is the memory size (in MB) per hidden node. + +There is a sysfs entry "probe" under /sys/devices/system/node/ for user +to hotplug the fake offlined nodes: + + - to show all fake offlined nodes: + $ cat /sys/devices/system/node/probe + + - to hotadd a fake offlined node, e.g. nodeid is N: + $ echo N > /sys/devices/system/node/probe + +2) CPU hotplug emulation: + +The emulator reserve CPUs throu grub parameter, the reserved CPUs can be +hot-add/hot-remove in software method, it emulates the process of physical +cpu hotplug. + +When hotplug a CPU with emulator, we are using a logical CPU to emulate the CPU +socket hotplug process. For the CPU supported SMT, some logical CPUs are in the +same socket, but it may located in different NUMA node after we have emulator. +We put the logical CPU into a fake CPU socket, and assign it an unique +phys_proc_id. For the fake socket, we put one logical CPU in only. + + - to hide CPUs + - Using boot option "maxcpus=N" hide CPUs + N is the number of initialize CPUs + - Using boot option "cpu_hpe=on" to enable cpu hotplug emulation + when cpu_hpe is enabled, the rest CPUs will not be initialized + + - to hot-add CPU to node + $ echo nid > cpu/probe + + - to hot-remove CPU + $ echo nid > cpu/release + +3) Memory hotplug emulation: + +The emulator reserve memory before OS booting, the reserved memory region +is remove from e820 table, and they can be hot-added via the probe interface, +this interface was extend to support add memory to the specified node, It +maintains backwards compatibility. + +The difficulty of Memory Release is well-known, we have no plan for it until now. + + - reserve memory throu grub parameter + mem=1024m + + - add a memory section to node 3 + $ echo 0x40000000,3 > memory/probe + OR + $ echo 1024m,3 > memory/probe + OR + $ echo "physical_address=0x40000000 numa_node=3" > memory/probe + +4) Script for hotplug testing + +These scripts provides convenience when we hot-add memory/cpu in batch. + +- Online all memory sections: +for m in /sys/devices/system/memory/memory*; +do + echo online > $m/state; +done + +- CPU Online: +for c in /sys/devices/system/cpu/cpu*; +do + echo 1 > $c/online; +done + +- Haicheng Li <haicheng.li@intel.com> +- Shaohui Zheng <shaohui.zheng@intel.com> + Nov 2010 Index: linux-hpe4/Documentation/x86/x86_64/boot-options.txt =================================================================== --- linux-hpe4.orig/Documentation/x86/x86_64/boot-options.txt 2010-11-17 10:01:37.093461435 +0800 +++ linux-hpe4/Documentation/x86/x86_64/boot-options.txt 2010-11-17 10:03:10.881043878 +0800 @@ -173,6 +173,13 @@ numa=fake=<N> If given as an integer, fills all system RAM with N fake nodes interleaved over physical nodes. + numa=hide=N*size1[,size2,...] + Give an string seperated by comma, each sub string stands for a serie nodes. + system will reserve an area to create hide numa nodes for them. + + for example: numa=hide=2*512,256 + system will reserve (2*512 + 256) M for 3 hide nodes. 2 nodes with 512M memory, + and 1 node with 256 memory ACPI @@ -316,3 +323,8 @@ Do not use GB pages for kernel direct mappings. gbpages Use GB pages for kernel direct mappings. + cpu_hpe=on/off + Enable/disable cpu hotplug emulation with software method. when cpu_hpe=on, + sysfs provides probe/release interface to hot add/remove cpu dynamically. + this option is disabled in default. + -- Thanks & Regards, Shaohui -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2010-11-17 4:45 UTC|newest] Thread overview: 139+ messages / expand[flat|nested] mbox.gz Atom feed top 2010-11-17 2:07 [0/8,v3] NUMA Hotplug Emulator - Introduction & Feedbacks shaohui.zheng 2010-11-17 2:07 ` shaohui.zheng 2010-11-17 2:08 ` [1/8,v3] NUMA Hotplug Emulator: add function to hide memory region via e820 table shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 8:16 ` David Rientjes 2010-11-17 8:16 ` David Rientjes 2010-11-18 9:20 ` Shaohui Zheng 2010-11-18 9:20 ` Shaohui Zheng 2010-11-18 21:16 ` David Rientjes 2010-11-18 21:16 ` David Rientjes 2010-11-19 0:12 ` Shaohui Zheng 2010-11-19 0:12 ` Shaohui Zheng 2010-11-21 0:45 ` David Rientjes 2010-11-21 0:45 ` David Rientjes 2010-11-21 14:00 ` Américo Wang 2010-11-21 14:00 ` Américo Wang 2010-11-21 21:33 ` David Rientjes 2010-11-21 21:33 ` David Rientjes 2010-11-17 2:08 ` [2/8,v3] NUMA Hotplug Emulator: infrastructure of NUMA hotplug emulation shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 8:16 ` David Rientjes 2010-11-17 8:16 ` David Rientjes 2010-11-17 7:51 ` Shaohui Zheng 2010-11-17 7:51 ` Shaohui Zheng 2010-11-17 21:10 ` David Rientjes 2010-11-17 21:10 ` David Rientjes 2010-11-18 4:14 ` Shaohui Zheng 2010-11-18 4:14 ` Shaohui Zheng 2010-11-18 6:27 ` Paul Mundt 2010-11-18 6:27 ` Paul Mundt 2010-11-18 5:27 ` Shaohui Zheng 2010-11-18 5:27 ` Shaohui Zheng 2010-11-18 21:24 ` David Rientjes 2010-11-18 21:24 ` David Rientjes 2010-11-19 0:32 ` Shaohui Zheng 2010-11-19 0:32 ` Shaohui Zheng 2010-11-21 0:48 ` David Rientjes 2010-11-21 0:48 ` David Rientjes 2010-11-21 2:28 ` [patch 1/2] x86: add numa=possible command line option David Rientjes 2010-11-21 2:28 ` David Rientjes 2010-11-21 2:28 ` [patch 2/2] mm: add node hotplug emulation David Rientjes 2010-11-21 2:28 ` David Rientjes 2010-11-21 17:34 ` Greg KH 2010-11-21 17:34 ` Greg KH 2010-11-21 21:48 ` David Rientjes 2010-11-21 21:48 ` David Rientjes 2010-11-21 23:08 ` [patch 2/2 v2] " David Rientjes 2010-11-21 23:08 ` David Rientjes 2010-11-22 0:56 ` Greg KH 2010-11-22 0:56 ` Greg KH 2010-11-28 1:52 ` David Rientjes 2010-11-28 1:52 ` David Rientjes 2010-11-28 5:17 ` Greg KH 2010-11-28 5:17 ` Greg KH 2010-11-30 0:04 ` David Rientjes 2010-11-30 0:04 ` David Rientjes 2010-11-21 14:26 ` [patch 1/2] x86: add numa=possible command line option Américo Wang 2010-11-21 14:26 ` Américo Wang 2010-11-21 21:46 ` David Rientjes 2010-11-21 21:46 ` David Rientjes 2010-11-22 15:43 ` Américo Wang 2010-11-22 15:43 ` Américo Wang 2010-11-21 15:14 ` [2/8,v3] NUMA Hotplug Emulator: infrastructure of NUMA hotplug emulation Li, Haicheng 2010-11-21 15:14 ` Li, Haicheng 2010-11-21 21:42 ` David Rientjes 2010-11-21 21:42 ` David Rientjes 2010-11-18 21:19 ` David Rientjes 2010-11-18 21:19 ` David Rientjes 2010-11-17 2:08 ` [3/8,v3] NUMA Hotplug Emulator: Userland interface to hotplug-add fake offlined nodes shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 8:16 ` David Rientjes 2010-11-17 8:16 ` David Rientjes 2010-11-17 2:08 ` [4/8,v3] NUMA Hotplug Emulator: Abstract cpu register functions shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 2:08 ` [5/8,v3] NUMA Hotplug Emulator: support cpu probe/release in x86 shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-21 14:45 ` Américo Wang 2010-11-21 14:45 ` Américo Wang 2010-11-22 0:01 ` Shaohui Zheng 2010-11-22 0:01 ` Shaohui Zheng 2010-11-22 15:51 ` Américo Wang 2010-11-22 15:51 ` Américo Wang 2010-11-22 23:29 ` Shaohui Zheng 2010-11-22 23:29 ` Shaohui Zheng 2010-11-17 2:08 ` [6/8,v3] NUMA Hotplug Emulator: Fake CPU socket with logical CPU on x86 shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 2:08 ` [7/8,v3] NUMA Hotplug Emulator: extend memory probe interface to support NUMA shaohui.zheng 2010-11-17 2:08 ` shaohui.zheng 2010-11-17 18:50 ` Dave Hansen 2010-11-17 18:50 ` Dave Hansen 2010-11-17 21:18 ` David Rientjes 2010-11-17 21:18 ` David Rientjes 2010-11-17 21:55 ` Dave Hansen 2010-11-17 21:55 ` Dave Hansen 2010-11-17 22:44 ` David Rientjes 2010-11-17 22:44 ` David Rientjes 2010-11-17 23:00 ` Dave Hansen 2010-11-17 23:00 ` Dave Hansen 2010-11-17 23:17 ` David Rientjes 2010-11-17 23:17 ` David Rientjes 2010-11-18 16:59 ` Aaron Durbin 2010-11-18 16:59 ` Aaron Durbin 2010-11-18 4:48 ` Shaohui Zheng 2010-11-18 4:48 ` Shaohui Zheng 2010-11-18 6:24 ` Paul Mundt 2010-11-18 6:24 ` Paul Mundt 2010-11-18 21:28 ` David Rientjes 2010-11-18 21:28 ` David Rientjes 2010-11-18 21:31 ` David Rientjes 2010-11-18 21:31 ` David Rientjes 2010-11-18 4:36 ` Shaohui Zheng 2010-11-18 4:36 ` Shaohui Zheng 2010-11-19 7:51 ` Shaohui Zheng 2010-11-19 16:36 ` Dave Hansen 2010-11-19 16:36 ` Dave Hansen 2010-11-17 2:08 ` shaohui.zheng [this message] 2010-11-17 2:08 ` [8/8,v3] NUMA Hotplug Emulator: documentation shaohui.zheng 2010-11-17 23:06 ` Randy Dunlap 2010-11-17 23:06 ` Randy Dunlap 2010-11-18 2:31 ` Shaohui Zheng 2010-11-18 2:31 ` Shaohui Zheng 2010-11-21 15:03 ` Américo Wang 2010-11-21 15:03 ` Américo Wang 2010-11-21 15:16 ` Li, Haicheng 2010-11-21 15:16 ` Li, Haicheng 2010-11-21 23:33 ` Shaohui Zheng 2010-11-21 23:33 ` Shaohui Zheng 2010-11-22 16:04 ` Américo Wang 2010-11-22 16:04 ` Américo Wang 2010-11-22 23:23 ` Shaohui Zheng 2010-11-22 23:23 ` Shaohui Zheng 2010-11-17 5:22 ` [0/8,v3] NUMA Hotplug Emulator - Introduction & Feedbacks Paul Mundt 2010-11-17 5:22 ` Paul Mundt 2010-11-19 5:54 ` Shaohui Zheng 2010-11-19 5:54 ` Shaohui Zheng 2010-11-17 9:26 ` Yinghai Lu 2010-11-17 9:26 ` Yinghai Lu 2010-11-18 2:03 ` Shaohui Zheng 2010-11-18 2:03 ` Shaohui Zheng
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20101117021000.985643862@intel.com \ --to=shaohui.zheng@intel.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=haicheng.li@intel.com \ --cc=haicheng.li@linux.intel.com \ --cc=lethal@linux-sh.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=shaohui.zheng@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.