[RFC PATCH 0/12] memory-hotplug : hot-remove physical memory

[RFC PATCH 0/12] memory-hotplug : hot-remove physical memory

[Yasuaki Ishimatsu]

This patch series aims to support physical memory hot-remove.

  [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory
  [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages
  [RFC PATCH 3/12] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove
  [RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs
  [RFC PATCH 5/12] memory hotplug : does not release memory region in PAGES_PER_SECTION chunks
  [RFC PATCH 6/12] memory-hotplug : remove_memory calls __remove_pages
  [RFC PATCH 7/12] memory-hotplug : check page type in get_page_bootmem
  [RFC PATCH 8/12] memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for
sparse-vmemmap
  [RFC PATCH 9/12] memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap
  [RFC PATCH 10/12] memory-hotplug : free memmap of sparse-vmemmap
  [RFC PATCH 11/12] memory-hotplug : add node_device_release
  [RFC PATCH 12/12] memory-hotplug : remove sysfs file of node

Even if you apply these patches, you cannot remove the physical memory
completely since these patches are still under development. I want you to
cooperate to improve the physical memory hot-remove. So please review these
patches and give your comment/idea.

The patches can free/remove following things:

  - acpi_memory_info                          : [RFC PATCH 3/12]
  - /sys/firmware/memmap/X/{end, start, type} : [RFC PATCH 4/12]
  - iomem_resource                            : [RFC PATCH 5/12]
  - mem_section and related sysfs files       : [RFC PATCH 6-10/12]
  - node and related sysfs files              : [RFC PATCH 11-12/12]

The patches cannot do following things yet:

  - page table of removed memory

If you find lack of function for physical memory hot-remove, please let me
know.

---

 arch/powerpc/platforms/pseries/hotplug-memory.c |   16 +-
 arch/x86/mm/init_64.c                           |  142 ++++++++++++++++++++++++
 drivers/acpi/acpi_memhotplug.c                  |   25 +++-
 drivers/base/memory.c                           |   24 +++-
 drivers/base/node.c                             |    7 +
 drivers/firmware/memmap.c                       |   71 ++++++++++++
 include/linux/firmware-map.h                    |    6 +
 include/linux/memory.h                          |    1
 include/linux/memory_hotplug.h                  |   17 --
 include/linux/mm.h                              |    5
 mm/memory_hotplug.c                             |  101 +++++++++++++----
 mm/sparse.c                                     |    5
 12 files changed, 374 insertions(+), 46 deletions(-)

[RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Yasuaki Ishimatsu]

remove_memory() does not remove memory but just offlines memory. The patch
changes name of it to offline_memory().

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 drivers/acpi/acpi_memhotplug.c |    2 +-
 drivers/base/memory.c          |    4 ++--
 include/linux/memory_hotplug.h |    2 +-
 mm/memory_hotplug.c            |    6 +++---
 4 files changed, 7 insertions(+), 7 deletions(-)

Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
===================================================================
--- linux-3.5-rc4.orig/drivers/acpi/acpi_memhotplug.c	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:48:38.263940481 +0900
@@ -318,7 +318,7 @@ static int acpi_memory_disable_device(st
 	 */
 	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
 		if (info->enabled) {
-			result = remove_memory(info->start_addr, info->length);
+			result = offline_memory(info->start_addr, info->length);
 			if (result)
 				return result;
 		}
Index: linux-3.5-rc4/drivers/base/memory.c
===================================================================
--- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:48:46.072842803 +0900
@@ -266,8 +266,8 @@ memory_block_action(unsigned long phys_i
 			break;
 		case MEM_OFFLINE:
 			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
-			ret = remove_memory(start_paddr,
-					    nr_pages << PAGE_SHIFT);
+			ret = offline_memory(start_paddr,
+					     nr_pages << PAGE_SHIFT);
 			break;
 		default:
 			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:46.072842803 +0900
@@ -990,7 +990,7 @@ out:
 	return ret;
 }

-int remove_memory(u64 start, u64 size)
+int offline_memory(u64 start, u64 size)
 {
 	unsigned long start_pfn, end_pfn;

@@ -999,9 +999,9 @@ int remove_memory(u64 start, u64 size)
 	return offline_pages(start_pfn, end_pfn, 120 * HZ);
 }
 #else
-int remove_memory(u64 start, u64 size)
+int offline_memory(u64 start, u64 size)
 {
 	return -EINVAL;
 }
 #endif /* CONFIG_MEMORY_HOTREMOVE */
-EXPORT_SYMBOL_GPL(remove_memory);
+EXPORT_SYMBOL_GPL(offline_memory);
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 13:48:38.264940468 +0900
@@ -233,7 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
-extern int remove_memory(u64 start, u64 size);
+extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
 extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms);

[RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

When offline_pages() is called to offlined memory, the function fails since
all memory has been offlined. In this case, the function should succeed.
The patch adds the check function into offline_pages().

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 drivers/base/memory.c  |   20 ++++++++++++++++++++
 include/linux/memory.h |    1 +
 mm/memory_hotplug.c    |    5 +++++
 3 files changed, 26 insertions(+)

Index: linux-3.5-rc4/drivers/base/memory.c
===================================================================
--- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
+++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
@@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
 }
 EXPORT_SYMBOL(unregister_memory_isolate_notifier);

+bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
+{
+	struct memory_block *mem;
+	struct mem_section *section;
+	unsigned long pfn, section_nr;
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		section = __nr_to_section(section_nr);
+		mem = find_memory_block(section);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+		return false;
+	}
+
+	return true;
+}
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */
Index: linux-3.5-rc4/include/linux/memory.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
@@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
 extern struct memory_block *find_memory_block_hinted(struct mem_section *,
 							struct memory_block *);
 extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
 #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
 enum mem_add_context { BOOT, HOTPLUG };
 #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
@@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned

 	lock_memory_hotplug();

+	if (memory_is_offline(start_pfn, end_pfn)) {
+		ret = 0;
+		goto out;
+	}
+
 	zone = page_zone(pfn_to_page(start_pfn));
 	node = zone_to_nid(zone);
 	nr_pages = end_pfn - start_pfn;

[RFC PATCH 3/12] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

[Yasuaki Ishimatsu]

acpi_memory_device_remove() has been prepared to remove physical memory.
But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
  - offline memory
  - remove physical memory (only return -EBUSY)
  - free acpi_memory_device

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 drivers/acpi/acpi_memhotplug.c |   23 ++++++++++++++++++++++-
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    8 ++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)

Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
===================================================================
--- linux-3.5-rc4.orig/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:28:16.722211802 +0900
+++ linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:38:01.085906251 +0900
@@ -452,12 +452,33 @@ static int acpi_memory_device_add(struct
 static int acpi_memory_device_remove(struct acpi_device *device, int type)
 {
 	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;

 	if (!device || !acpi_driver_data(device))
 		return -EINVAL;

 	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		result = offline_memory(info->start_addr, info->length);
+		if (result)
+			return result;
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
 	kfree(mem_device);

 	return 0;
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-26 13:28:16.773211163 +0900
+++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 13:37:30.545288962 +0900
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
 extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:34:22.425639879 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:30.096046767 +0900
@@ -659,6 +659,14 @@ out:
 }
 EXPORT_SYMBOL_GPL(add_memory);

+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

[RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs

[Yasuaki Ishimatsu]

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type}
sysfs files are created. But there is no code to remove these files. The patch
implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
       memory which is allocated by bootmem.

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 drivers/firmware/memmap.c    |   71 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/firmware-map.h |    6 +++
 mm/memory_hotplug.c          |    6 +++
 3 files changed, 82 insertions(+), 1 deletion(-)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:37:30.546288901 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:44:37.069955820 +0900
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);

 int remove_memory(int nid, u64 start, u64 size)
 {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size, "System RAM");
+	unlock_memory_hotplug();
+	return 0;

 }
 EXPORT_SYMBOL_GPL(remove_memory);
Index: linux-3.5-rc4/include/linux/firmware-map.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/firmware-map.h	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/include/linux/firmware-map.h	2012-06-26 13:44:37.070955807 +0900
@@ -25,6 +25,7 @@

 int firmware_map_add_early(u64 start, u64 end, const char *type);
 int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);

 #else /* CONFIG_FIRMWARE_MEMMAP */

@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
 	return 0;
 }

+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FIRMWARE_MEMMAP */

 #endif /* _LINUX_FIRMWARE_MAP_H */
Index: linux-3.5-rc4/drivers/firmware/memmap.c
===================================================================
--- linux-3.5-rc4.orig/drivers/firmware/memmap.c	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/drivers/firmware/memmap.c	2012-06-26 13:47:17.606948898 +0900
@@ -123,6 +123,16 @@ static int firmware_map_add_entry(u64 st
 	return 0;
 }

+/**
+ * firmware_map_remove_entry() - Does the real work to remove a firmware
+ * memmap entry.
+ * @entry: removed entry.
+ **/
+static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
+{
+	list_del(&entry->list);
+}
+
 /*
  * Add memmap entry on sysfs
  */
@@ -144,6 +154,31 @@ static int add_sysfs_fw_map_entry(struct
 	return 0;
 }

+/*
+ * Remove memmap entry on sysfs
+ */
+static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
+{
+	kobject_del(&entry->kobj);
+}
+
+/*
+ * Search memmap entry
+ */
+
+struct firmware_map_entry * __meminit
+find_firmware_map_entry(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	list_for_each_entry(entry, &map_entries, list)
+		if ((entry->start == start) && (entry->end == end) &&
+		    (!strcmp(entry->type, type)))
+			return entry;
+
+	return NULL;
+}
+
 /**
  * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
  * memory hotplug.
@@ -196,6 +231,42 @@ int __init firmware_map_add_early(u64 st
 	return firmware_map_add_entry(start, end, type, entry);
 }

+void release_firmware_map_entry(struct firmware_map_entry *entry)
+{
+	/*
+	 * FIXME : There is no idea.
+	 *         How to free the entry which allocated bootmem?
+	 */
+}
+
+/**
+ * firmware_map_remove() - remove a firmware mapping entry
+ * @start: Start of the memory range.
+ * @end:   End of the memory range (inclusive).
+ * @type:  Type of the memory range.
+ *
+ * removes a firmware mapping entry.
+ *
+ * Returns 0 on success, or -EINVAL if no entry.
+ **/
+int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	entry = find_firmware_map_entry(start, end, type);
+	if (!entry)
+		return -EINVAL;
+
+	/* remove the memmap entry */
+	remove_sysfs_fw_map_entry(entry);
+
+	firmware_map_remove_entry(entry);
+
+	release_firmware_map_entry(entry);
+
+	return 0;
+}
+
 /*
  * Sysfs functions -------------------------------------------------------------
  */

[RFC PATCH 5/12] memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks

[Yasuaki Ishimatsu]

Since applying a patch(de7f0cba96786c), release_mem_region() has been changed
as called in PAGES_PER_SECTION chunks because register_memory_resource() is
called in PAGES_PER_SECTION chunks by add_memory(). But it seems firmware
dependency. If CRS are written in the PAGES_PER_SECTION chunks in ACPI DSDT
Table, register_memory_resource() is called in PAGES_PER_SECTION chunks.
But if CRS are written in the DIMM unit in ACPI DSDT Table,
register_memory_resource() is called in DIMM unit. So release_mem_region()
should not be called in PAGES_PER_SECTION chunks. The patch fixes it.

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   13 +++++++++----
 mm/memory_hotplug.c                             |    4 ++--
 2 files changed, 11 insertions(+), 6 deletions(-)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:50:57.075205100 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:55:36.152716100 +0900
@@ -358,11 +358,11 @@ int __remove_pages(struct zone *zone, un
 	BUG_ON(phys_start_pfn & ~PAGE_SECTION_MASK);
 	BUG_ON(nr_pages % PAGES_PER_SECTION);

+	release_mem_region(phys_start_pfn << PAGE_SHIFT,  nr_pages * PAGE_SIZE);
+
 	sections_to_remove = nr_pages / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
-		release_mem_region(pfn << PAGE_SHIFT,
-				   PAGES_PER_SECTION << PAGE_SHIFT);
 		ret = __remove_section(zone, __pfn_to_section(pfn));
 		if (ret)
 			break;
Index: linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c
===================================================================
--- linux-3.5-rc4.orig/arch/powerpc/platforms/pseries/hotplug-memory.c	2012-06-26 13:50:57.075205100
+0900
+++ linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c	2012-06-26 13:55:36.153716088 +0900
@@ -77,7 +77,8 @@ static int pseries_remove_memblock(unsig
 {
 	unsigned long start, start_pfn;
 	struct zone *zone;
-	int ret;
+	int i, ret;
+	int sections_to_remove;

 	start_pfn = base >> PAGE_SHIFT;

@@ -97,9 +98,13 @@ static int pseries_remove_memblock(unsig
 	 * to sysfs "state" file and we can't remove sysfs entries
 	 * while writing to it. So we have to defer it to here.
 	 */
-	ret = __remove_pages(zone, start_pfn, memblock_size >> PAGE_SHIFT);
-	if (ret)
-		return ret;
+	sections_to_remove = (memblock_size >> PAGE_SHIFT) / PAGES_PER_SECTION;
+	for (i = 0; i < sections_to_remove; i++) {
+		unsigned long pfn = start_pfn + i * PAGES_PER_SECTION;
+		ret = __remove_pages(zone, start_pfn,  PAGES_PER_SECTION);
+		if (ret)
+			return ret;
+	}

 	/*
 	 * Update memory regions for memory remove

[RFC PATCH 6/12] memory-hotplug : remove_memory calls __remove_pages

[Yasuaki Ishimatsu]

The patch adds __remove_pages() to remove_memory(). Then the range of
phys_start_pfn argument and nr_pages argument in __remove_pagse() may
have different zone. So zone argument is removed from __remove_pages()
and __remove_pages() caluculates zone in each section.

When CONFIG_SPARSEMEM_VMEMMAP is defined, there is no way to remove a memmap.
So __remove_section only calls unregister_memory_section().

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 arch/powerpc/platforms/pseries/hotplug-memory.c |    5 +----
 include/linux/memory_hotplug.h                  |    3 +--
 mm/memory_hotplug.c                             |   20 +++++++++++++-------
 3 files changed, 15 insertions(+), 13 deletions(-)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-27 09:07:49.124709164 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-27 09:08:30.612190446 +0900
@@ -275,11 +275,14 @@ static int __meminit __add_section(int n
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 static int __remove_section(struct zone *zone, struct mem_section *ms)
 {
-	/*
-	 * XXX: Freeing memmap with vmemmap is not implement yet.
-	 *      This should be removed later.
-	 */
-	return -EBUSY;
+	int ret;
+
+	if (!valid_section(ms))
+		return ret;
+
+	ret = unregister_memory_section(ms);
+
+	return ret;
 }
 #else
 static int __remove_section(struct zone *zone, struct mem_section *ms)
@@ -346,11 +349,11 @@ EXPORT_SYMBOL_GPL(__add_pages);
  * sure that pages are marked reserved and zones are adjust properly by
  * calling offline_pages().
  */
-int __remove_pages(struct zone *zone, unsigned long phys_start_pfn,
-		 unsigned long nr_pages)
+int __remove_pages(unsigned long phys_start_pfn, unsigned long nr_pages)
 {
 	unsigned long i, ret = 0;
 	int sections_to_remove;
+	struct zone *zone;

 	/*
 	 * We can only remove entire sections
@@ -363,6 +366,7 @@ int __remove_pages(struct zone *zone, un
 	sections_to_remove = nr_pages / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
+		zone = page_zone(pfn_to_page(pfn));
 		ret = __remove_section(zone, __pfn_to_section(pfn));
 		if (ret)
 			break;
@@ -664,6 +668,8 @@ int remove_memory(int nid, u64 start, u6
 	lock_memory_hotplug();
 	/* remove memmap entry */
 	firmware_map_remove(start, start + size, "System RAM");
+
+	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
 	unlock_memory_hotplug();
 	return 0;

Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-27 09:06:35.616628141 +0900
+++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-27 09:08:30.612190446 +0900
@@ -89,8 +89,7 @@ extern bool is_pageblock_removable_noloc
 /* reasonably generic interface to expand the physical pages in a zone  */
 extern int __add_pages(int nid, struct zone *zone, unsigned long start_pfn,
 	unsigned long nr_pages);
-extern int __remove_pages(struct zone *zone, unsigned long start_pfn,
-	unsigned long nr_pages);
+extern int __remove_pages(unsigned long start_pfn, unsigned long nr_pages);

 #ifdef CONFIG_NUMA
 extern int memory_add_physaddr_to_nid(u64 start);
Index: linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c
===================================================================
--- linux-3.5-rc4.orig/arch/powerpc/platforms/pseries/hotplug-memory.c	2012-06-27 09:08:25.737248748
+0900
+++ linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c	2012-06-27 09:08:30.613190434 +0900
@@ -76,7 +76,6 @@ unsigned long memory_block_size_bytes(vo
 static int pseries_remove_memblock(unsigned long base, unsigned int memblock_size)
 {
 	unsigned long start, start_pfn;
-	struct zone *zone;
 	int i, ret;
 	int sections_to_remove;

@@ -87,8 +86,6 @@ static int pseries_remove_memblock(unsig
 		return 0;
 	}

-	zone = page_zone(pfn_to_page(start_pfn));
-
 	/*
 	 * Remove section mappings and sysfs entries for the
 	 * section of the memory we are removing.
@@ -101,7 +98,7 @@ static int pseries_remove_memblock(unsig
 	sections_to_remove = (memblock_size >> PAGE_SHIFT) / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = start_pfn + i * PAGES_PER_SECTION;
-		ret = __remove_pages(zone, start_pfn,  PAGES_PER_SECTION);
+		ret = __remove_pages(start_pfn,  PAGES_PER_SECTION);
 		if (ret)
 			return ret;
 	}

[RFC PATCH 7/12] memory-hotplug : check page type in get_page_bootmem

[Yasuaki Ishimatsu]

There is a possibility that get_page_bootmem() is called to the same page many
times. So when get_page_bootmem is called to the same page, the function only
increments page->_count.

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:56:34.371988312 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 14:03:37.564697621 +0900
@@ -95,10 +95,17 @@ static void release_memory_resource(stru
 static void get_page_bootmem(unsigned long info,  struct page *page,
 			     unsigned long type)
 {
-	page->lru.next = (struct list_head *) type;
-	SetPagePrivate(page);
-	set_page_private(page, info);
-	atomic_inc(&page->_count);
+	unsigned long page_type;
+
+	page_type = (unsigned long) page->lru.next;
+	if (type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE ||
+	    type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE){
+		page->lru.next = (struct list_head *) type;
+		SetPagePrivate(page);
+		set_page_private(page, info);
+		atomic_inc(&page->_count);
+	} else
+		atomic_inc(&page->_count);
 }

 /* reference to __meminit __free_pages_bootmem is valid

[RFC PATCH 8/12] memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for sparse-vmemmap

[Yasuaki Ishimatsu]

For implementing register_page_bootmem_info_node of sparse-vmemmap,
register_page_bootmem_info_node and put_page_bootmem are moved to
memory_hotplug.c

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyan <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 include/linux/memory_hotplug.h |    9 ---------
 mm/memory_hotplug.c            |    8 ++++++--
 2 files changed, 6 insertions(+), 11 deletions(-)

Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-26 13:56:34.372988299 +0900
+++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 14:05:16.603459518 +0900
@@ -160,17 +160,8 @@ static inline void arch_refresh_nodedata
 #endif /* CONFIG_NUMA */
 #endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */

-#ifdef CONFIG_SPARSEMEM_VMEMMAP
-static inline void register_page_bootmem_info_node(struct pglist_data *pgdat)
-{
-}
-static inline void put_page_bootmem(struct page *page)
-{
-}
-#else
 extern void register_page_bootmem_info_node(struct pglist_data *pgdat);
 extern void put_page_bootmem(struct page *page);
-#endif

 /*
  * Lock for memory hotplug guarantees 1) all callbacks for memory hotplug
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 14:03:37.564697621 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 14:05:16.604459506 +0900
@@ -91,7 +91,6 @@ static void release_memory_resource(stru
 }

 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
-#ifndef CONFIG_SPARSEMEM_VMEMMAP
 static void get_page_bootmem(unsigned long info,  struct page *page,
 			     unsigned long type)
 {
@@ -127,6 +126,7 @@ void __ref put_page_bootmem(struct page

 }

+#ifndef CONFIG_SPARSEMEM_VMEMMAP
 static void register_page_bootmem_info_section(unsigned long start_pfn)
 {
 	unsigned long *usemap, mapsize, section_nr, i;
@@ -163,6 +163,11 @@ static void register_page_bootmem_info_s
 		get_page_bootmem(section_nr, page, MIX_SECTION_INFO);

 }
+#else
+static inline void register_page_bootmem_info_section(unsigned long start_pfn)
+{
+}
+#endif

 void register_page_bootmem_info_node(struct pglist_data *pgdat)
 {
@@ -198,7 +203,6 @@ void register_page_bootmem_info_node(str
 		register_page_bootmem_info_section(pfn);

 }
-#endif /* !CONFIG_SPARSEMEM_VMEMMAP */

 static void grow_zone_span(struct zone *zone, unsigned long start_pfn,
 			   unsigned long end_pfn)

[RFC PATCH 9/12] memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap

[Yasuaki Ishimatsu]

For removing memmap region of sparse-vmemmap which is allocated bootmem,
memmap region of sparse-vmemmap needs to be registered by get_page_bootmem().
So the patch searches pages of virtual mapping and registers the pages by
get_page_bootmem().

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 arch/x86/mm/init_64.c          |   53 +++++++++++++++++++++++++++++++++++++++++
 include/linux/memory_hotplug.h |    2 +
 include/linux/mm.h             |    3 +-
 mm/memory_hotplug.c            |   23 +++++++++++++++--
 4 files changed, 77 insertions(+), 4 deletions(-)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 14:05:16.604459506 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 14:27:19.850916817 +0900
@@ -91,8 +91,8 @@ static void release_memory_resource(stru
 }

 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
-static void get_page_bootmem(unsigned long info,  struct page *page,
-			     unsigned long type)
+void get_page_bootmem(unsigned long info,  struct page *page,
+		      unsigned long type)
 {
 	unsigned long page_type;

@@ -164,8 +164,25 @@ static void register_page_bootmem_info_s

 }
 #else
-static inline void register_page_bootmem_info_section(unsigned long start_pfn)
+static void register_page_bootmem_info_section(unsigned long start_pfn)
 {
+	unsigned long mapsize, section_nr;
+	struct mem_section *ms;
+	struct page *page, *memmap;
+
+	if (!pfn_valid(start_pfn))
+		return;
+
+	section_nr = pfn_to_section_nr(start_pfn);
+	ms = __nr_to_section(section_nr);
+
+	memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
+
+	page = virt_to_page(memmap);
+	mapsize = sizeof(struct page) * PAGES_PER_SECTION;
+	mapsize = PAGE_ALIGN(mapsize) >> PAGE_SHIFT;
+
+	register_page_bootmem_memmap(section_nr, memmap, PAGES_PER_SECTION);
 }
 #endif

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/mm.h	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/include/linux/mm.h	2012-06-26 14:26:17.481696320 +0900
@@ -1586,7 +1586,8 @@ int vmemmap_populate_basepages(struct pa
 						unsigned long pages, int node);
 int vmemmap_populate(struct page *start_page, unsigned long pages, int node);
 void vmemmap_populate_print_last(void);
-
+void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
+				  unsigned long size);

 enum mf_flags {
 	MF_COUNT_INCREASED = 1 << 0,
Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================
--- linux-3.5-rc4.orig/arch/x86/mm/init_64.c	2012-06-25 04:53:04.000000000 +0900
+++ linux-3.5-rc4/arch/x86/mm/init_64.c	2012-06-26 14:26:17.481696320 +0900
@@ -978,6 +978,59 @@ vmemmap_populate(struct page *start_page
 	return 0;
 }

+void __meminit
+register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
+			     unsigned long size)
+{
+	unsigned long addr = (unsigned long)start_page;
+	unsigned long end = (unsigned long)(start_page + size);
+	unsigned long next;
+	pgd_t *pgd;
+	pud_t *pud;
+	pmd_t *pmd;
+
+	for (; addr < end; addr = next) {
+		pte_t *pte = NULL;
+
+		pgd = pgd_offset_k(addr);
+		if (pgd_none(*pgd)) {
+			next = (addr + PAGE_SIZE) & PAGE_MASK;
+			continue;
+		}
+		get_page_bootmem(section_nr, pgd_page(*pgd), MIX_SECTION_INFO);
+
+		pud = pud_offset(pgd, addr);
+		if (pud_none(*pud)) {
+			next = (addr + PAGE_SIZE) & PAGE_MASK;
+			continue;
+		}
+		get_page_bootmem(section_nr, pud_page(*pud), MIX_SECTION_INFO);
+
+		if (!cpu_has_pse) {
+			next = (addr + PAGE_SIZE) & PAGE_MASK;
+			pmd = pmd_offset(pud, addr);
+			if (pmd_none(*pmd))
+				continue;
+			get_page_bootmem(section_nr, pmd_page(*pmd),
+					 MIX_SECTION_INFO);
+
+			pte = pte_offset_kernel(pmd, addr);
+			if (pte_none(*pte))
+				continue;
+			get_page_bootmem(section_nr, pte_page(*pte),
+					 SECTION_INFO);
+		} else {
+			next = pmd_addr_end(addr, end);
+
+			pmd = pmd_offset(pud, addr);
+			if (pmd_none(*pmd))
+				continue;
+			get_page_bootmem(section_nr, pmd_page(*pmd),
+					 SECTION_INFO);
+		}
+	}
+}
+
 void __meminit vmemmap_populate_print_last(void)
 {
 	if (p_start) {
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-26 14:05:16.603459518 +0900
+++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 14:11:14.175989381 +0900
@@ -162,6 +162,8 @@ static inline void arch_refresh_nodedata

 extern void register_page_bootmem_info_node(struct pglist_data *pgdat);
 extern void put_page_bootmem(struct page *page);
+extern void get_page_bootmem(unsigned long ingo, struct page *page,
+			     unsigned long type);

 /*
  * Lock for memory hotplug guarantees 1) all callbacks for memory hotplug

[RFC PATCH 10/12] memory-hotplug : free memmap of sparse-vmemmap

[Yasuaki Ishimatsu]

I don't think that all pages of virtual mapping in removed memory can be
freed, since page which type is MIX_SECTION_INFO is difficult to free.
So, the patch only frees page which type is SECTION_INFO at first.

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 arch/x86/mm/init_64.c |   89 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h    |    2 +
 mm/memory_hotplug.c   |    5 ++
 mm/sparse.c           |    5 +-
 4 files changed, 99 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================
--- linux-3.5-rc4.orig/include/linux/mm.h	2012-06-27 09:11:13.790150442 +0900
+++ linux-3.5-rc4/include/linux/mm.h	2012-06-27 09:11:16.433117400 +0900
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
 void vmemmap_populate_print_last(void);
 void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
 				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);

 enum mf_flags {
 	MF_COUNT_INCREASED = 1 << 0,
Index: linux-3.5-rc4/mm/sparse.c
===================================================================
--- linux-3.5-rc4.orig/mm/sparse.c	2012-06-27 09:06:35.317631878 +0900
+++ linux-3.5-rc4/mm/sparse.c	2012-06-27 09:11:16.434117388 +0900
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
 	/* This will make the necessary allocations eventually. */
 	return sparse_mem_map_populate(pnum, nid);
 }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
 {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);
 }
 static void free_map_bootmem(struct page *page, unsigned long nr_pages)
 {
+	vmemmap_free_bootmem(page, nr_pages);
 }
 #else
 static struct page *__kmalloc_section_memmap(unsigned long nr_pages)
Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================
--- linux-3.5-rc4.orig/arch/x86/mm/init_64.c	2012-06-27 09:11:13.791150430 +0900
+++ linux-3.5-rc4/arch/x86/mm/init_64.c	2012-06-27 09:11:59.254581998 +0900
@@ -978,6 +978,95 @@ vmemmap_populate(struct page *start_page
 	return 0;
 }

+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page *page)
+{
+	pgd_t *pgd;
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	page = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return PAGE_SIZE;
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return PAGE_SIZE;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		page = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		page = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		next = find_and_clear_pte_page(addr, end, page);
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page))
+			vfree(page);
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page,
+				   get_order(sizeof(struct page) *  order));
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		next = find_and_clear_pte_page(addr, end, page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
 void __meminit
 register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
 			     unsigned long size)
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-27 09:11:13.789150454 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-27 09:11:16.436117363 +0900
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 static int __remove_section(struct zone *zone, struct mem_section *ms)
 {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
 	int ret;

 	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone

 	ret = unregister_memory_section(ms);

+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
 	return ret;
 }
 #else

[RFC PATCH 11/12] memory-hotplug : add node_device_release

[Yasuaki Ishimatsu]

When calling unregister_node(), the function shows following message at
device_release().

Device 'node2' does not have a release() function, it is broken and must be fixed.

So the patch implements node_device_release()

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

Index: linux-3.5-rc1/drivers/base/node.c
===================================================================
--- linux-3.5-rc1.orig/drivers/base/node.c	2012-06-14 09:09:53.000000000 +0900
+++ linux-3.5-rc1/drivers/base/node.c	2012-06-25 18:40:45.810261969 +0900
@@ -252,6 +252,12 @@ static inline void hugetlb_register_node
 static inline void hugetlb_unregister_node(struct node *node) {}
 #endif

+static void node_device_release(struct device *dev)
+{
+	struct node *node_dev = to_node(dev);
+
+	memset(node_dev, 0, sizeof(struct node));
+}

 /*
  * register_node - Setup a sysfs device for a node.
@@ -265,6 +271,7 @@ int register_node(struct node *node, int

 	node->dev.id = num;
 	node->dev.bus = &node_subsys;
+	node->dev.release = node_device_release;
 	error = device_register(&node->dev);

 	if (!error){

[RFC PATCH 12/12] memory-hotplug : remove sysfs file of node

[Yasuaki Ishimatsu]

The patch adds node_set_offline() and unregister_one_node() to remove_memory()
for removing sysfs file of node.

CC: Len Brown <len.brown@intel.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
CC: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

---
 mm/memory_hotplug.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
--- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 14:32:03.630368866 +0900
+++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 14:39:58.090437374 +0900
@@ -702,6 +702,11 @@ int remove_memory(int nid, u64 start, u6
 	/* remove memmap entry */
 	firmware_map_remove(start, start + size, "System RAM");

+	if (!node_present_pages(nid)) {
+		node_set_offline(nid);
+		unregister_one_node(nid);
+	}
+
 	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
 	unlock_memory_hotplug();
 	return 0;

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[David Rientjes]

On Wed, 27 Jun 2012, Yasuaki Ishimatsu wrote:

> remove_memory() does not remove memory but just offlines memory. The patch
> changes name of it to offline_memory().
> 

The kernel is never going to physically remove the memory itself, so I 
don't see the big problem with calling it remove_memory().  If you're 
going to change it to offline_memory(), which is just as good but not 
better, then I'd suggest changing add_memory() to online_memory() for 
completeness.

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[David Rientjes]

On Wed, 27 Jun 2012, Yasuaki Ishimatsu wrote:

> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
> 
>  	lock_memory_hotplug();
> 
> +	if (memory_is_offline(start_pfn, end_pfn)) {
> +		ret = 0;
> +		goto out;
> +	}
> +
>  	zone = page_zone(pfn_to_page(start_pfn));
>  	node = zone_to_nid(zone);
>  	nr_pages = end_pfn - start_pfn;

Are there additional prerequisites for this patch?  Otherwise it changes 
the return value of offline_memory() which will now call 
acpi_memory_powerdown_device() in the acpi memhotplug case when disabling.  
Is that a problem?

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Wen Congyang]

At 06/27/2012 01:42 PM, Yasuaki Ishimatsu Wrote:
> remove_memory() does not remove memory but just offlines memory. The patch
> changes name of it to offline_memory().

There are 3 functions in the kernel:
1. add_memory()
2. online_pages()
3. remove_memory()

So I think offline_pages() is better than offline_memory().

Thanks
Wen Congyang
> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  drivers/acpi/acpi_memhotplug.c |    2 +-
>  drivers/base/memory.c          |    4 ++--
>  include/linux/memory_hotplug.h |    2 +-
>  mm/memory_hotplug.c            |    6 +++---
>  4 files changed, 7 insertions(+), 7 deletions(-)
> 
> Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/acpi/acpi_memhotplug.c	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:48:38.263940481 +0900
> @@ -318,7 +318,7 @@ static int acpi_memory_disable_device(st
>  	 */
>  	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
>  		if (info->enabled) {
> -			result = remove_memory(info->start_addr, info->length);
> +			result = offline_memory(info->start_addr, info->length);
>  			if (result)
>  				return result;
>  		}
> Index: linux-3.5-rc4/drivers/base/memory.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:48:46.072842803 +0900
> @@ -266,8 +266,8 @@ memory_block_action(unsigned long phys_i
>  			break;
>  		case MEM_OFFLINE:
>  			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
> -			ret = remove_memory(start_paddr,
> -					    nr_pages << PAGE_SHIFT);
> +			ret = offline_memory(start_paddr,
> +					     nr_pages << PAGE_SHIFT);
>  			break;
>  		default:
>  			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:46.072842803 +0900
> @@ -990,7 +990,7 @@ out:
>  	return ret;
>  }
> 
> -int remove_memory(u64 start, u64 size)
> +int offline_memory(u64 start, u64 size)
>  {
>  	unsigned long start_pfn, end_pfn;
> 
> @@ -999,9 +999,9 @@ int remove_memory(u64 start, u64 size)
>  	return offline_pages(start_pfn, end_pfn, 120 * HZ);
>  }
>  #else
> -int remove_memory(u64 start, u64 size)
> +int offline_memory(u64 start, u64 size)
>  {
>  	return -EINVAL;
>  }
>  #endif /* CONFIG_MEMORY_HOTREMOVE */
> -EXPORT_SYMBOL_GPL(remove_memory);
> +EXPORT_SYMBOL_GPL(offline_memory);
> Index: linux-3.5-rc4/include/linux/memory_hotplug.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 13:48:38.264940468 +0900
> @@ -233,7 +233,7 @@ static inline int is_mem_section_removab
>  extern int mem_online_node(int nid);
>  extern int add_memory(int nid, u64 start, u64 size);
>  extern int arch_add_memory(int nid, u64 start, u64 size);
> -extern int remove_memory(u64 start, u64 size);
> +extern int offline_memory(u64 start, u64 size);
>  extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
>  								int nr_pages);
>  extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms);
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Wen Congyang]

At 06/27/2012 03:14 PM, Wen Congyang Wrote:
> At 06/27/2012 01:42 PM, Yasuaki Ishimatsu Wrote:
>> remove_memory() does not remove memory but just offlines memory. The patch
>> changes name of it to offline_memory().
> 
> There are 3 functions in the kernel:
> 1. add_memory()
> 2. online_pages()
> 3. remove_memory()
> 
> So I think offline_pages() is better than offline_memory().

There is already a function named offline_pages(). So we
should call offline_pages() instead of remove_memory() in
memory_block_action(), and there is no need to rename
remove_memory().

Thanks
Wen Congyang

> 
> Thanks
> Wen Congyang
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>  drivers/acpi/acpi_memhotplug.c |    2 +-
>>  drivers/base/memory.c          |    4 ++--
>>  include/linux/memory_hotplug.h |    2 +-
>>  mm/memory_hotplug.c            |    6 +++---
>>  4 files changed, 7 insertions(+), 7 deletions(-)
>>
>> Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/acpi/acpi_memhotplug.c	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:48:38.263940481 +0900
>> @@ -318,7 +318,7 @@ static int acpi_memory_disable_device(st
>>  	 */
>>  	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
>>  		if (info->enabled) {
>> -			result = remove_memory(info->start_addr, info->length);
>> +			result = offline_memory(info->start_addr, info->length);
>>  			if (result)
>>  				return result;
>>  		}
>> Index: linux-3.5-rc4/drivers/base/memory.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:48:46.072842803 +0900
>> @@ -266,8 +266,8 @@ memory_block_action(unsigned long phys_i
>>  			break;
>>  		case MEM_OFFLINE:
>>  			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
>> -			ret = remove_memory(start_paddr,
>> -					    nr_pages << PAGE_SHIFT);
>> +			ret = offline_memory(start_paddr,
>> +					     nr_pages << PAGE_SHIFT);
>>  			break;
>>  		default:
>>  			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:46.072842803 +0900
>> @@ -990,7 +990,7 @@ out:
>>  	return ret;
>>  }
>>
>> -int remove_memory(u64 start, u64 size)
>> +int offline_memory(u64 start, u64 size)
>>  {
>>  	unsigned long start_pfn, end_pfn;
>>
>> @@ -999,9 +999,9 @@ int remove_memory(u64 start, u64 size)
>>  	return offline_pages(start_pfn, end_pfn, 120 * HZ);
>>  }
>>  #else
>> -int remove_memory(u64 start, u64 size)
>> +int offline_memory(u64 start, u64 size)
>>  {
>>  	return -EINVAL;
>>  }
>>  #endif /* CONFIG_MEMORY_HOTREMOVE */
>> -EXPORT_SYMBOL_GPL(remove_memory);
>> +EXPORT_SYMBOL_GPL(offline_memory);
>> Index: linux-3.5-rc4/include/linux/memory_hotplug.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 13:48:38.264940468 +0900
>> @@ -233,7 +233,7 @@ static inline int is_mem_section_removab
>>  extern int mem_online_node(int nid);
>>  extern int add_memory(int nid, u64 start, u64 size);
>>  extern int arch_add_memory(int nid, u64 start, u64 size);
>> -extern int remove_memory(u64 start, u64 size);
>> +extern int offline_memory(u64 start, u64 size);
>>  extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
>>  								int nr_pages);
>>  extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms);
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Wen Congyang]

At 06/27/2012 01:44 PM, Yasuaki Ishimatsu Wrote:
> When offline_pages() is called to offlined memory, the function fails since
> all memory has been offlined. In this case, the function should succeed.
> The patch adds the check function into offline_pages().

You miss such case: some pages are online, while some pages are offline.
offline_pages() will fail too in such case.

Thanks
Wen Congyang

> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  drivers/base/memory.c  |   20 ++++++++++++++++++++
>  include/linux/memory.h |    1 +
>  mm/memory_hotplug.c    |    5 +++++
>  3 files changed, 26 insertions(+)
> 
> Index: linux-3.5-rc4/drivers/base/memory.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>  }
>  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
> 
> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
> +{
> +	struct memory_block *mem;
> +	struct mem_section *section;
> +	unsigned long pfn, section_nr;
> +
> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
> +		section_nr = pfn_to_section_nr(pfn);
> +		section = __nr_to_section(section_nr);
> +		mem = find_memory_block(section);
> +		if (!mem)
> +			continue;
> +		if (mem->state == MEM_OFFLINE)
> +			continue;
> +		return false;
> +	}
> +
> +	return true;
> +}
> +
>  /*
>   * register_memory - Setup a sysfs device for a memory block
>   */
> Index: linux-3.5-rc4/include/linux/memory.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>  							struct memory_block *);
>  extern struct memory_block *find_memory_block(struct mem_section *);
> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>  enum mem_add_context { BOOT, HOTPLUG };
>  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
> 
>  	lock_memory_hotplug();
> 
> +	if (memory_is_offline(start_pfn, end_pfn)) {
> +		ret = 0;
> +		goto out;
> +	}
> +
>  	zone = page_zone(pfn_to_page(start_pfn));
>  	node = zone_to_nid(zone);
>  	nr_pages = end_pfn - start_pfn;
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Yasuaki Ishimatsu]

Hi David and Wen,

Thank you for reviewing my patch.

2012/06/27 17:47, Wen Congyang wrote:
> At 06/27/2012 03:14 PM, Wen Congyang Wrote:
>> At 06/27/2012 01:42 PM, Yasuaki Ishimatsu Wrote:
>>> remove_memory() does not remove memory but just offlines memory. The patch
>>> changes name of it to offline_memory().
>>
>> There are 3 functions in the kernel:
>> 1. add_memory()
>> 2. online_pages()
>> 3. remove_memory()
>>
>> So I think offline_pages() is better than offline_memory().
> 
> There is already a function named offline_pages(). So we
> should call offline_pages() instead of remove_memory() in
> memory_block_action(), and there is no need to rename
> remove_memory().

As Wen says, Linux has 4 functions for memory hotplug already.
In my recognition, these functions are prepared for following purpose.

1. add_memory     : add physical memory
2. online_pages   : online logical memory
3. remove_memory  : offline logical memory
4. offline_pages  : offline logical memory

add_memory() is used for adding physical memory. I think remove_memory()
would rather be used for removing physical memory than be used for removing
logical memory. So I renamed remove_memory() to offline_memory().
How do you think?

Regards,
Yasuaki Ishimatsu

> 
> Thanks
> Wen Congyang
> 
>>
>> Thanks
>> Wen Congyang
>>>
>>> CC: Len Brown <len.brown@intel.com>
>>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>>> CC: Paul Mackerras <paulus@samba.org>
>>> CC: Christoph Lameter <cl@linux.com>
>>> Cc: Minchan Kim <minchan.kim@gmail.com>
>>> CC: Andrew Morton <akpm@linux-foundation.org>
>>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>>> CC: Wen Congyang <wency@cn.fujitsu.com>
>>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>>
>>> ---
>>>   drivers/acpi/acpi_memhotplug.c |    2 +-
>>>   drivers/base/memory.c          |    4 ++--
>>>   include/linux/memory_hotplug.h |    2 +-
>>>   mm/memory_hotplug.c            |    6 +++---
>>>   4 files changed, 7 insertions(+), 7 deletions(-)
>>>
>>> Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/drivers/acpi/acpi_memhotplug.c	2012-06-25 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c	2012-06-26 13:48:38.263940481 +0900
>>> @@ -318,7 +318,7 @@ static int acpi_memory_disable_device(st
>>>   	 */
>>>   	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
>>>   		if (info->enabled) {
>>> -			result = remove_memory(info->start_addr, info->length);
>>> +			result = offline_memory(info->start_addr, info->length);
>>>   			if (result)
>>>   				return result;
>>>   		}
>>> Index: linux-3.5-rc4/drivers/base/memory.c
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-25 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:48:46.072842803 +0900
>>> @@ -266,8 +266,8 @@ memory_block_action(unsigned long phys_i
>>>   			break;
>>>   		case MEM_OFFLINE:
>>>   			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
>>> -			ret = remove_memory(start_paddr,
>>> -					    nr_pages << PAGE_SHIFT);
>>> +			ret = offline_memory(start_paddr,
>>> +					     nr_pages << PAGE_SHIFT);
>>>   			break;
>>>   		default:
>>>   			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
>>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-25 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:46.072842803 +0900
>>> @@ -990,7 +990,7 @@ out:
>>>   	return ret;
>>>   }
>>>
>>> -int remove_memory(u64 start, u64 size)
>>> +int offline_memory(u64 start, u64 size)
>>>   {
>>>   	unsigned long start_pfn, end_pfn;
>>>
>>> @@ -999,9 +999,9 @@ int remove_memory(u64 start, u64 size)
>>>   	return offline_pages(start_pfn, end_pfn, 120 * HZ);
>>>   }
>>>   #else
>>> -int remove_memory(u64 start, u64 size)
>>> +int offline_memory(u64 start, u64 size)
>>>   {
>>>   	return -EINVAL;
>>>   }
>>>   #endif /* CONFIG_MEMORY_HOTREMOVE */
>>> -EXPORT_SYMBOL_GPL(remove_memory);
>>> +EXPORT_SYMBOL_GPL(offline_memory);
>>> Index: linux-3.5-rc4/include/linux/memory_hotplug.h
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/include/linux/memory_hotplug.h	2012-06-25 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/include/linux/memory_hotplug.h	2012-06-26 13:48:38.264940468 +0900
>>> @@ -233,7 +233,7 @@ static inline int is_mem_section_removab
>>>   extern int mem_online_node(int nid);
>>>   extern int add_memory(int nid, u64 start, u64 size);
>>>   extern int arch_add_memory(int nid, u64 start, u64 size);
>>> -extern int remove_memory(u64 start, u64 size);
>>> +extern int offline_memory(u64 start, u64 size);
>>>   extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
>>>   								int nr_pages);
>>>   extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms);
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Wen Congyang]

At 06/28/2012 11:01 AM, Yasuaki Ishimatsu Wrote:
> Hi David and Wen,
> 
> Thank you for reviewing my patch.
> 
> 2012/06/27 17:47, Wen Congyang wrote:
>> At 06/27/2012 03:14 PM, Wen Congyang Wrote:
>>> At 06/27/2012 01:42 PM, Yasuaki Ishimatsu Wrote:
>>>> remove_memory() does not remove memory but just offlines memory. The patch
>>>> changes name of it to offline_memory().
>>>
>>> There are 3 functions in the kernel:
>>> 1. add_memory()
>>> 2. online_pages()
>>> 3. remove_memory()
>>>
>>> So I think offline_pages() is better than offline_memory().
>>
>> There is already a function named offline_pages(). So we
>> should call offline_pages() instead of remove_memory() in
>> memory_block_action(), and there is no need to rename
>> remove_memory().
> 
> As Wen says, Linux has 4 functions for memory hotplug already.
> In my recognition, these functions are prepared for following purpose.
> 
> 1. add_memory     : add physical memory
> 2. online_pages   : online logical memory
> 3. remove_memory  : offline logical memory
> 4. offline_pages  : offline logical memory
> 
> add_memory() is used for adding physical memory. I think remove_memory()
> would rather be used for removing physical memory than be used for removing
> logical memory. So I renamed remove_memory() to offline_memory().
> How do you think?

Hmm, remove_memory() will revert all things we do in add_memory(), so I think
there is no need to rename it. If we rename it to offline_memory(), we should
also rename add_memory() to online_memory().

Thanks
Wen Congyang

> 
> Regards,
> Yasuaki Ishimatsu
> 
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>> Thanks
>>> Wen Congyang
>>>>

Re: [RFC PATCH 1/12] memory-hotplug : rename remove_memory to offline_memory

[Yasuaki Ishimatsu]

Hi Wen,

2012/06/28 12:25, Wen Congyang wrote:
> At 06/28/2012 11:01 AM, Yasuaki Ishimatsu Wrote:
>> Hi David and Wen,
>>
>> Thank you for reviewing my patch.
>>
>> 2012/06/27 17:47, Wen Congyang wrote:
>>> At 06/27/2012 03:14 PM, Wen Congyang Wrote:
>>>> At 06/27/2012 01:42 PM, Yasuaki Ishimatsu Wrote:
>>>>> remove_memory() does not remove memory but just offlines memory. The patch
>>>>> changes name of it to offline_memory().
>>>>
>>>> There are 3 functions in the kernel:
>>>> 1. add_memory()
>>>> 2. online_pages()
>>>> 3. remove_memory()
>>>>
>>>> So I think offline_pages() is better than offline_memory().
>>>
>>> There is already a function named offline_pages(). So we
>>> should call offline_pages() instead of remove_memory() in
>>> memory_block_action(), and there is no need to rename
>>> remove_memory().
>>
>> As Wen says, Linux has 4 functions for memory hotplug already.
>> In my recognition, these functions are prepared for following purpose.
>>
>> 1. add_memory     : add physical memory
>> 2. online_pages   : online logical memory
>> 3. remove_memory  : offline logical memory
>> 4. offline_pages  : offline logical memory
>>
>> add_memory() is used for adding physical memory. I think remove_memory()
>> would rather be used for removing physical memory than be used for removing
>> logical memory. So I renamed remove_memory() to offline_memory().
>> How do you think?
> 
> Hmm, remove_memory() will revert all things we do in add_memory(), so I think

I think so too.

add_memory() prepares to use physical memory. It prepares some structures
(pgdat, page table, node, etc) for using the physical memory at the system.
But it does not online the meomory. For onlining the memory, we use
online_pages().

So I think that remove_memory() should remove these structures which are
prepared by add_memory() not offline memory. But current remove_memory() code
only calls offline_pages() and offlines memory.

The patch series recreates remove_memory() for removing these structures
after [RFC PATCH 3/12]. The reason to change the name of remove_memory() is a
preparation to recreate it.

Thanks,
Yasuaki Ishimatsu

> there is no need to rename it. If we rename it to offline_memory(), we should
> also rename add_memory() to online_memory().
> 
> Thanks
> Wen Congyang
> 
>>
>> Regards,
>> Yasuaki Ishimatsu
>>
>>>
>>> Thanks
>>> Wen Congyang
>>>
>>>>
>>>> Thanks
>>>> Wen Congyang
>>>>>

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi Wen,

2012/06/27 17:49, Wen Congyang wrote:
> At 06/27/2012 01:44 PM, Yasuaki Ishimatsu Wrote:
>> When offline_pages() is called to offlined memory, the function fails since
>> all memory has been offlined. In this case, the function should succeed.
>> The patch adds the check function into offline_pages().
> 
> You miss such case: some pages are online, while some pages are offline.
> offline_pages() will fail too in such case.

You are right. But current code fails, when the function is called to offline
memory. In this case, the function should succeed. So the patch confirms
whether the memory was offlined or not. And if memory has already been
offlined, offline_pages return 0.

Thanks,
Yasuaki Ishimatsu

> 
> Thanks
> Wen Congyang
> 
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   drivers/base/memory.c  |   20 ++++++++++++++++++++
>>   include/linux/memory.h |    1 +
>>   mm/memory_hotplug.c    |    5 +++++
>>   3 files changed, 26 insertions(+)
>>
>> Index: linux-3.5-rc4/drivers/base/memory.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
>> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
>> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>>   }
>>   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
>>
>> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
>> +{
>> +	struct memory_block *mem;
>> +	struct mem_section *section;
>> +	unsigned long pfn, section_nr;
>> +
>> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
>> +		section_nr = pfn_to_section_nr(pfn);
>> +		section = __nr_to_section(section_nr);
>> +		mem = find_memory_block(section);
>> +		if (!mem)
>> +			continue;
>> +		if (mem->state == MEM_OFFLINE)
>> +			continue;
>> +		return false;
>> +	}
>> +
>> +	return true;
>> +}
>> +
>>   /*
>>    * register_memory - Setup a sysfs device for a memory block
>>    */
>> Index: linux-3.5-rc4/include/linux/memory.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
>> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>>   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>>   							struct memory_block *);
>>   extern struct memory_block *find_memory_block(struct mem_section *);
>> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>>   #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>>   enum mem_add_context { BOOT, HOTPLUG };
>>   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
>> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>>
>>   	lock_memory_hotplug();
>>
>> +	if (memory_is_offline(start_pfn, end_pfn)) {
>> +		ret = 0;
>> +		goto out;
>> +	}
>> +
>>   	zone = page_zone(pfn_to_page(start_pfn));
>>   	node = zone_to_nid(zone);
>>   	nr_pages = end_pfn - start_pfn;
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[KOSAKI Motohiro]

On Wed, Jun 27, 2012 at 1:44 AM, Yasuaki Ishimatsu
<isimatu.yasuaki@jp.fujitsu.com> wrote:
> When offline_pages() is called to offlined memory, the function fails sin=
ce
> all memory has been offlined. In this case, the function should succeed.
> The patch adds the check function into offline_pages().

I don't understand your point. I think following misoperation should
fail. Otherwise
administrator have no way to know their fault.

$ echo offline > memoryN/state
$ echo offline > memoryN/state

In general, we don't like to ignore an error except the standard require it=
.

>
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>
> ---
> =A0drivers/base/memory.c =A0| =A0 20 ++++++++++++++++++++
> =A0include/linux/memory.h | =A0 =A01 +
> =A0mm/memory_hotplug.c =A0 =A0| =A0 =A05 +++++
> =A03 files changed, 26 insertions(+)
>
> Index: linux-3.5-rc4/drivers/base/memory.c
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- linux-3.5-rc4.orig/drivers/base/memory.c =A0 =A02012-06-26 13:28:16.7=
26211752 +0900
> +++ linux-3.5-rc4/drivers/base/memory.c 2012-06-26 13:34:22.423639904 +09=
00
> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
> =A0}
> =A0EXPORT_SYMBOL(unregister_memory_isolate_notifier);
>
> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)

I dislike this function name. 'memory' is too vague to me.


> +{
> + =A0 =A0 =A0 struct memory_block *mem;
> + =A0 =A0 =A0 struct mem_section *section;
> + =A0 =A0 =A0 unsigned long pfn, section_nr;
> +
> + =A0 =A0 =A0 for (pfn =3D start_pfn; pfn < end_pfn; pfn +=3D PAGES_PER_S=
ECTION) {
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 section_nr =3D pfn_to_section_nr(pfn);
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 section =3D __nr_to_section(section_nr);
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 mem =3D find_memory_block(section);

This seems to have strong sparse dependency.
Hm, I wonder why memory-hotplug.c can enable when X86_64_ACPI_NUMA.

# eventually, we can have this option just 'select SPARSEMEM'
config MEMORY_HOTPLUG
	bool "Allow for memory hot-add"
	depends on SPARSEMEM || X86_64_ACPI_NUMA


> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (!mem)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (mem->state =3D=3D MEM_OFFLINE)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 return false;
> + =A0 =A0 =A0 }
> +
> + =A0 =A0 =A0 return true;
> +}
> +
> =A0/*
> =A0* register_memory - Setup a sysfs device for a memory block
> =A0*/
> Index: linux-3.5-rc4/include/linux/memory.h
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- linux-3.5-rc4.orig/include/linux/memory.h =A0 2012-06-25 04:53:04.000=
000000 +0900
> +++ linux-3.5-rc4/include/linux/memory.h =A0 =A0 =A0 =A02012-06-26 13:34:=
22.424639891 +0900
> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
> =A0extern struct memory_block *find_memory_block_hinted(struct mem_sectio=
n *,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct memory_block *);
> =A0extern struct memory_block *find_memory_block(struct mem_section *);
> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end=
_pfn);
> =A0#define CONFIG_MEM_BLOCK_SIZE =A0(PAGES_PER_SECTION<<PAGE_SHIFT)
> =A0enum mem_add_context { BOOT, HOTPLUG };
> =A0#endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c =A0 =A0 =A02012-06-26 13:28:16=
.743211538 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c =A0 2012-06-26 13:48:38.264940468 +=
0900
> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>
> =A0 =A0 =A0 =A0lock_memory_hotplug();
>
> + =A0 =A0 =A0 if (memory_is_offline(start_pfn, end_pfn)) {
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret =3D 0;
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
> + =A0 =A0 =A0 }
> +
> =A0 =A0 =A0 =A0zone =3D page_zone(pfn_to_page(start_pfn));
> =A0 =A0 =A0 =A0node =3D zone_to_nid(zone);
> =A0 =A0 =A0 =A0nr_pages =3D end_pfn - start_pfn;
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. =A0For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=3Dmailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[KOSAKI Motohiro]

On Thu, Jun 28, 2012 at 1:06 AM, Yasuaki Ishimatsu
<isimatu.yasuaki@jp.fujitsu.com> wrote:
> Hi Wen,
>
> 2012/06/27 17:49, Wen Congyang wrote:
>> At 06/27/2012 01:44 PM, Yasuaki Ishimatsu Wrote:
>>> When offline_pages() is called to offlined memory, the function fails since
>>> all memory has been offlined. In this case, the function should succeed.
>>> The patch adds the check function into offline_pages().
>>
>> You miss such case: some pages are online, while some pages are offline.
>> offline_pages() will fail too in such case.
>
> You are right. But current code fails, when the function is called to offline
> memory. In this case, the function should succeed. So the patch confirms
> whether the memory was offlined or not. And if memory has already been
> offlined, offline_pages return 0.

Can you please explain why the caller can't check it? I hope to avoid
an ignorance
as far as we can.

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi Kosaki-san,

2012/06/28 14:27, KOSAKI Motohiro wrote:
> On Thu, Jun 28, 2012 at 1:06 AM, Yasuaki Ishimatsu
> <isimatu.yasuaki@jp.fujitsu.com> wrote:
>> Hi Wen,
>>
>> 2012/06/27 17:49, Wen Congyang wrote:
>>> At 06/27/2012 01:44 PM, Yasuaki Ishimatsu Wrote:
>>>> When offline_pages() is called to offlined memory, the function fails since
>>>> all memory has been offlined. In this case, the function should succeed.
>>>> The patch adds the check function into offline_pages().
>>>
>>> You miss such case: some pages are online, while some pages are offline.
>>> offline_pages() will fail too in such case.
>>
>> You are right. But current code fails, when the function is called to offline
>> memory. In this case, the function should succeed. So the patch confirms
>> whether the memory was offlined or not. And if memory has already been
>> offlined, offline_pages return 0.
>
> Can you please explain why the caller can't check it? I hope to avoid
> an ignorance
> as far as we can.

Of course, caller side can check it. But there is a possibility that
offline_pages() is called by many functions. So I do not think that it
is good that all functions which call offline_pages() check it.

Thanks,
Yasuaki Ishimatsu

Re: [RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs

[Wen Congyang]

At 06/27/2012 01:47 PM, Yasuaki Ishimatsu Wrote:
> When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type}
> sysfs files are created. But there is no code to remove these files. The patch
> implements the function to remove them.
> 
> Note : The code does not free firmware_map_entry since there is no way to free
>        memory which is allocated by bootmem.
> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  drivers/firmware/memmap.c    |   71 +++++++++++++++++++++++++++++++++++++++++++
>  include/linux/firmware-map.h |    6 +++
>  mm/memory_hotplug.c          |    6 +++
>  3 files changed, 82 insertions(+), 1 deletion(-)
> 
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:37:30.546288901 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:44:37.069955820 +0900
> @@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
> 
>  int remove_memory(int nid, u64 start, u64 size)
>  {
> -	return -EBUSY;
> +	lock_memory_hotplug();
> +	/* remove memmap entry */
> +	firmware_map_remove(start, start + size, "System RAM");
> +	unlock_memory_hotplug();
> +	return 0;
> 
>  }
>  EXPORT_SYMBOL_GPL(remove_memory);
> Index: linux-3.5-rc4/include/linux/firmware-map.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/firmware-map.h	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/include/linux/firmware-map.h	2012-06-26 13:44:37.070955807 +0900
> @@ -25,6 +25,7 @@
> 
>  int firmware_map_add_early(u64 start, u64 end, const char *type);
>  int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
> +int firmware_map_remove(u64 start, u64 end, const char *type);
> 
>  #else /* CONFIG_FIRMWARE_MEMMAP */
> 
> @@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
>  	return 0;
>  }
> 
> +static inline int firmware_map_remove(u64 start, u64 end, const char *type)
> +{
> +	return 0;
> +}
> +
>  #endif /* CONFIG_FIRMWARE_MEMMAP */
> 
>  #endif /* _LINUX_FIRMWARE_MAP_H */
> Index: linux-3.5-rc4/drivers/firmware/memmap.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/firmware/memmap.c	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/drivers/firmware/memmap.c	2012-06-26 13:47:17.606948898 +0900
> @@ -123,6 +123,16 @@ static int firmware_map_add_entry(u64 st
>  	return 0;
>  }
> 
> +/**
> + * firmware_map_remove_entry() - Does the real work to remove a firmware
> + * memmap entry.
> + * @entry: removed entry.
> + **/
> +static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
> +{
> +	list_del(&entry->list);
> +}
> +
>  /*
>   * Add memmap entry on sysfs
>   */
> @@ -144,6 +154,31 @@ static int add_sysfs_fw_map_entry(struct
>  	return 0;
>  }
> 
> +/*
> + * Remove memmap entry on sysfs
> + */
> +static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
> +{
> +	kobject_del(&entry->kobj);
> +}
> +
> +/*
> + * Search memmap entry
> + */
> +
> +struct firmware_map_entry * __meminit
> +find_firmware_map_entry(u64 start, u64 end, const char *type)
> +{
> +	struct firmware_map_entry *entry;
> +
> +	list_for_each_entry(entry, &map_entries, list)
> +		if ((entry->start == start) && (entry->end == end) &&
> +		    (!strcmp(entry->type, type)))
> +			return entry;
> +
> +	return NULL;
> +}
> +
>  /**
>   * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
>   * memory hotplug.
> @@ -196,6 +231,42 @@ int __init firmware_map_add_early(u64 st
>  	return firmware_map_add_entry(start, end, type, entry);
>  }
> 
> +void release_firmware_map_entry(struct firmware_map_entry *entry)
> +{
> +	/*
> +	 * FIXME : There is no idea.
> +	 *         How to free the entry which allocated bootmem?
> +	 */
> +}
> +
> +/**
> + * firmware_map_remove() - remove a firmware mapping entry
> + * @start: Start of the memory range.
> + * @end:   End of the memory range (inclusive).
> + * @type:  Type of the memory range.
> + *
> + * removes a firmware mapping entry.
> + *
> + * Returns 0 on success, or -EINVAL if no entry.
> + **/
> +int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
> +{
> +	struct firmware_map_entry *entry;
> +
> +	entry = find_firmware_map_entry(start, end, type);

Hmm, we cannot find the entry easily, because the end can be:
1. start + size
2. start + size - 1

So, We should fix this bug first.

> +	if (!entry)
> +		return -EINVAL;
> +
> +	/* remove the memmap entry */
> +	remove_sysfs_fw_map_entry(entry);
> +
> +	firmware_map_remove_entry(entry);
> +
> +	release_firmware_map_entry(entry);

I guess you want to free the memory in the above function. But I think it is
not a good idea to free it here. We should free it when the entry->kobj's kref
is decreased to 0.

Thanks
Wen Congyang

> +
> +	return 0;
> +}
> +
>  /*
>   * Sysfs functions -------------------------------------------------------------
>   */
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi Kosaki-san,

2012/06/28 14:26, KOSAKI Motohiro wrote:
> On Wed, Jun 27, 2012 at 1:44 AM, Yasuaki Ishimatsu
> <isimatu.yasuaki@jp.fujitsu.com> wrote:
>> When offline_pages() is called to offlined memory, the function fails since
>> all memory has been offlined. In this case, the function should succeed.
>> The patch adds the check function into offline_pages().
>
> I don't understand your point. I think following misoperation should
> fail. Otherwise
> administrator have no way to know their fault.
>
> $ echo offline > memoryN/state
> $ echo offline > memoryN/state
>
> In general, we don't like to ignore an error except the standard require it.

I understood the intention of previous mail (why the caller can't check it? ).
I'll move memory_is_offline() to caller side.

>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   drivers/base/memory.c  |   20 ++++++++++++++++++++
>>   include/linux/memory.h |    1 +
>>   mm/memory_hotplug.c    |    5 +++++
>>   3 files changed, 26 insertions(+)
>>
>> Index: linux-3.5-rc4/drivers/base/memory.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/base/memory.c    2012-06-26 13:28:16.726211752 +0900
>> +++ linux-3.5-rc4/drivers/base/memory.c 2012-06-26 13:34:22.423639904 +0900
>> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>>   }
>>   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
>>
>> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
>
> I dislike this function name. 'memory' is too vague to me.

O.K.
I retry to change the name of the function.

>
>
>> +{
>> +       struct memory_block *mem;
>> +       struct mem_section *section;
>> +       unsigned long pfn, section_nr;
>> +
>> +       for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
>> +               section_nr = pfn_to_section_nr(pfn);
>> +               section = __nr_to_section(section_nr);
>> +               mem = find_memory_block(section);
>
> This seems to have strong sparse dependency.

Thanks.
I will consider other CONFIG_.

Thanks.
Yasuaki Ishimatsu

> Hm, I wonder why memory-hotplug.c can enable when X86_64_ACPI_NUMA.
>
> # eventually, we can have this option just 'select SPARSEMEM'
> config MEMORY_HOTPLUG
> 	bool "Allow for memory hot-add"
> 	depends on SPARSEMEM || X86_64_ACPI_NUMA
>
>
>> +               if (!mem)
>> +                       continue;
>> +               if (mem->state == MEM_OFFLINE)
>> +                       continue;
>> +               return false;
>> +       }
>> +
>> +       return true;
>> +}
>> +
>>   /*
>>   * register_memory - Setup a sysfs device for a memory block
>>   */
>> Index: linux-3.5-rc4/include/linux/memory.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/memory.h   2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/memory.h        2012-06-26 13:34:22.424639891 +0900
>> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>>   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>>                                                         struct memory_block *);
>>   extern struct memory_block *find_memory_block(struct mem_section *);
>> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>>   #define CONFIG_MEM_BLOCK_SIZE  (PAGES_PER_SECTION<<PAGE_SHIFT)
>>   enum mem_add_context { BOOT, HOTPLUG };
>>   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c      2012-06-26 13:28:16.743211538 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c   2012-06-26 13:48:38.264940468 +0900
>> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>>
>>         lock_memory_hotplug();
>>
>> +       if (memory_is_offline(start_pfn, end_pfn)) {
>> +               ret = 0;
>> +               goto out;
>> +       }
>> +
>>         zone = page_zone(pfn_to_page(start_pfn));
>>         node = zone_to_nid(zone);
>>         nr_pages = end_pfn - start_pfn;
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi David,

2012/06/27 15:16, David Rientjes wrote:
> On Wed, 27 Jun 2012, Yasuaki Ishimatsu wrote:
>
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
>> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>>
>>   	lock_memory_hotplug();
>>
>> +	if (memory_is_offline(start_pfn, end_pfn)) {
>> +		ret = 0;
>> +		goto out;
>> +	}
>> +
>>   	zone = page_zone(pfn_to_page(start_pfn));
>>   	node = zone_to_nid(zone);
>>   	nr_pages = end_pfn - start_pfn;
>
> Are there additional prerequisites for this patch?  Otherwise it changes
> the return value of offline_memory() which will now call
> acpi_memory_powerdown_device() in the acpi memhotplug case when disabling.
> Is that a problem?

I have understood there is a person who expects "offline_pages()" to fail
in this case by kosaki's comment. So I'll move memory_is_offline to caller
side.

Thanks,
Yasuaki Ishimatsu

>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

Re: [RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs

[Yasuaki Ishimatsu]

Hi Wen,

2012/06/28 15:32, Wen Congyang wrote:
> At 06/27/2012 01:47 PM, Yasuaki Ishimatsu Wrote:
>> When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type}
>> sysfs files are created. But there is no code to remove these files. The patch
>> implements the function to remove them.
>>
>> Note : The code does not free firmware_map_entry since there is no way to free
>>         memory which is allocated by bootmem.
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   drivers/firmware/memmap.c    |   71 +++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/firmware-map.h |    6 +++
>>   mm/memory_hotplug.c          |    6 +++
>>   3 files changed, 82 insertions(+), 1 deletion(-)
>>
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:37:30.546288901 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:44:37.069955820 +0900
>> @@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
>>
>>   int remove_memory(int nid, u64 start, u64 size)
>>   {
>> -	return -EBUSY;
>> +	lock_memory_hotplug();
>> +	/* remove memmap entry */
>> +	firmware_map_remove(start, start + size, "System RAM");
>> +	unlock_memory_hotplug();
>> +	return 0;
>>
>>   }
>>   EXPORT_SYMBOL_GPL(remove_memory);
>> Index: linux-3.5-rc4/include/linux/firmware-map.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/firmware-map.h	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/firmware-map.h	2012-06-26 13:44:37.070955807 +0900
>> @@ -25,6 +25,7 @@
>>
>>   int firmware_map_add_early(u64 start, u64 end, const char *type);
>>   int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
>> +int firmware_map_remove(u64 start, u64 end, const char *type);
>>
>>   #else /* CONFIG_FIRMWARE_MEMMAP */
>>
>> @@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
>>   	return 0;
>>   }
>>
>> +static inline int firmware_map_remove(u64 start, u64 end, const char *type)
>> +{
>> +	return 0;
>> +}
>> +
>>   #endif /* CONFIG_FIRMWARE_MEMMAP */
>>
>>   #endif /* _LINUX_FIRMWARE_MAP_H */
>> Index: linux-3.5-rc4/drivers/firmware/memmap.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/firmware/memmap.c	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/drivers/firmware/memmap.c	2012-06-26 13:47:17.606948898 +0900
>> @@ -123,6 +123,16 @@ static int firmware_map_add_entry(u64 st
>>   	return 0;
>>   }
>>
>> +/**
>> + * firmware_map_remove_entry() - Does the real work to remove a firmware
>> + * memmap entry.
>> + * @entry: removed entry.
>> + **/
>> +static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
>> +{
>> +	list_del(&entry->list);
>> +}
>> +
>>   /*
>>    * Add memmap entry on sysfs
>>    */
>> @@ -144,6 +154,31 @@ static int add_sysfs_fw_map_entry(struct
>>   	return 0;
>>   }
>>
>> +/*
>> + * Remove memmap entry on sysfs
>> + */
>> +static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
>> +{
>> +	kobject_del(&entry->kobj);
>> +}
>> +
>> +/*
>> + * Search memmap entry
>> + */
>> +
>> +struct firmware_map_entry * __meminit
>> +find_firmware_map_entry(u64 start, u64 end, const char *type)
>> +{
>> +	struct firmware_map_entry *entry;
>> +
>> +	list_for_each_entry(entry, &map_entries, list)
>> +		if ((entry->start == start) && (entry->end == end) &&
>> +		    (!strcmp(entry->type, type)))
>> +			return entry;
>> +
>> +	return NULL;
>> +}
>> +
>>   /**
>>    * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
>>    * memory hotplug.
>> @@ -196,6 +231,42 @@ int __init firmware_map_add_early(u64 st
>>   	return firmware_map_add_entry(start, end, type, entry);
>>   }
>>
>> +void release_firmware_map_entry(struct firmware_map_entry *entry)
>> +{
>> +	/*
>> +	 * FIXME : There is no idea.
>> +	 *         How to free the entry which allocated bootmem?
>> +	 */
>> +}
>> +
>> +/**
>> + * firmware_map_remove() - remove a firmware mapping entry
>> + * @start: Start of the memory range.
>> + * @end:   End of the memory range (inclusive).
>> + * @type:  Type of the memory range.
>> + *
>> + * removes a firmware mapping entry.
>> + *
>> + * Returns 0 on success, or -EINVAL if no entry.
>> + **/
>> +int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
>> +{
>> +	struct firmware_map_entry *entry;
>> +
>> +	entry = find_firmware_map_entry(start, end, type);
>
> Hmm, we cannot find the entry easily, because the end can be:
> 1. start + size
> 2. start + size - 1
>
> So, We should fix this bug first.

This is not a bug.

start and size arguments of firmware_map_remove() include
acpi_memory_info->{start_addr, length}. And when creating a firmware_map_entry,
the entry is created by acpi_memory_info->{start_addr, length}. So I don't
think that we need care your comment.

>
>> +	if (!entry)
>> +		return -EINVAL;
>> +
>> +	/* remove the memmap entry */
>> +	remove_sysfs_fw_map_entry(entry);
>> +
>> +	firmware_map_remove_entry(entry);
>> +
>> +	release_firmware_map_entry(entry);
>
> I guess you want to free the memory in the above function. But I think it is
> not a good idea to free it here. We should free it when the entry->kobj's kref
> is decreased to 0.

Thanks.
I'll update your comment at next version.

Thanks,
Yasuaki Ishimatsu

>
> Thanks
> Wen Congyang
>
>> +
>> +	return 0;
>> +}
>> +
>>   /*
>>    * Sysfs functions -------------------------------------------------------------
>>    */
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

Re: [RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs

[Wen Congyang]

At 06/28/2012 04:07 PM, Yasuaki Ishimatsu Wrote:
> Hi Wen,
> 
> 2012/06/28 15:32, Wen Congyang wrote:
>> At 06/27/2012 01:47 PM, Yasuaki Ishimatsu Wrote:
>>> When (hot)adding memory into system, /sys/firmware/memmap/X/{end,
>>> start, type}
>>> sysfs files are created. But there is no code to remove these files.
>>> The patch
>>> implements the function to remove them.
>>>
>>> Note : The code does not free firmware_map_entry since there is no
>>> way to free
>>>         memory which is allocated by bootmem.
>>>
>>> CC: Len Brown <len.brown@intel.com>
>>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>>> CC: Paul Mackerras <paulus@samba.org>
>>> CC: Christoph Lameter <cl@linux.com>
>>> Cc: Minchan Kim <minchan.kim@gmail.com>
>>> CC: Andrew Morton <akpm@linux-foundation.org>
>>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>>> CC: Wen Congyang <wency@cn.fujitsu.com>
>>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>>
>>> ---
>>>   drivers/firmware/memmap.c    |   71
>>> +++++++++++++++++++++++++++++++++++++++++++
>>>   include/linux/firmware-map.h |    6 +++
>>>   mm/memory_hotplug.c          |    6 +++
>>>   3 files changed, 82 insertions(+), 1 deletion(-)
>>>
>>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c    2012-06-26
>>> 13:37:30.546288901 +0900
>>> +++ linux-3.5-rc4/mm/memory_hotplug.c    2012-06-26
>>> 13:44:37.069955820 +0900
>>> @@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
>>>
>>>   int remove_memory(int nid, u64 start, u64 size)
>>>   {
>>> -    return -EBUSY;
>>> +    lock_memory_hotplug();
>>> +    /* remove memmap entry */
>>> +    firmware_map_remove(start, start + size, "System RAM");
>>> +    unlock_memory_hotplug();
>>> +    return 0;
>>>
>>>   }
>>>   EXPORT_SYMBOL_GPL(remove_memory);
>>> Index: linux-3.5-rc4/include/linux/firmware-map.h
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/include/linux/firmware-map.h    2012-06-25
>>> 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/include/linux/firmware-map.h    2012-06-26
>>> 13:44:37.070955807 +0900
>>> @@ -25,6 +25,7 @@
>>>
>>>   int firmware_map_add_early(u64 start, u64 end, const char *type);
>>>   int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
>>> +int firmware_map_remove(u64 start, u64 end, const char *type);
>>>
>>>   #else /* CONFIG_FIRMWARE_MEMMAP */
>>>
>>> @@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
>>>       return 0;
>>>   }
>>>
>>> +static inline int firmware_map_remove(u64 start, u64 end, const char
>>> *type)
>>> +{
>>> +    return 0;
>>> +}
>>> +
>>>   #endif /* CONFIG_FIRMWARE_MEMMAP */
>>>
>>>   #endif /* _LINUX_FIRMWARE_MAP_H */
>>> Index: linux-3.5-rc4/drivers/firmware/memmap.c
>>> ===================================================================
>>> --- linux-3.5-rc4.orig/drivers/firmware/memmap.c    2012-06-25
>>> 04:53:04.000000000 +0900
>>> +++ linux-3.5-rc4/drivers/firmware/memmap.c    2012-06-26
>>> 13:47:17.606948898 +0900
>>> @@ -123,6 +123,16 @@ static int firmware_map_add_entry(u64 st
>>>       return 0;
>>>   }
>>>
>>> +/**
>>> + * firmware_map_remove_entry() - Does the real work to remove a
>>> firmware
>>> + * memmap entry.
>>> + * @entry: removed entry.
>>> + **/
>>> +static inline void firmware_map_remove_entry(struct
>>> firmware_map_entry *entry)
>>> +{
>>> +    list_del(&entry->list);
>>> +}
>>> +
>>>   /*
>>>    * Add memmap entry on sysfs
>>>    */
>>> @@ -144,6 +154,31 @@ static int add_sysfs_fw_map_entry(struct
>>>       return 0;
>>>   }
>>>
>>> +/*
>>> + * Remove memmap entry on sysfs
>>> + */
>>> +static inline void remove_sysfs_fw_map_entry(struct
>>> firmware_map_entry *entry)
>>> +{
>>> +    kobject_del(&entry->kobj);
>>> +}
>>> +
>>> +/*
>>> + * Search memmap entry
>>> + */
>>> +
>>> +struct firmware_map_entry * __meminit
>>> +find_firmware_map_entry(u64 start, u64 end, const char *type)
>>> +{
>>> +    struct firmware_map_entry *entry;
>>> +
>>> +    list_for_each_entry(entry, &map_entries, list)
>>> +        if ((entry->start == start) && (entry->end == end) &&
>>> +            (!strcmp(entry->type, type)))
>>> +            return entry;
>>> +
>>> +    return NULL;
>>> +}
>>> +
>>>   /**
>>>    * firmware_map_add_hotplug() - Adds a firmware mapping entry when
>>> we do
>>>    * memory hotplug.
>>> @@ -196,6 +231,42 @@ int __init firmware_map_add_early(u64 st
>>>       return firmware_map_add_entry(start, end, type, entry);
>>>   }
>>>
>>> +void release_firmware_map_entry(struct firmware_map_entry *entry)
>>> +{
>>> +    /*
>>> +     * FIXME : There is no idea.
>>> +     *         How to free the entry which allocated bootmem?
>>> +     */
>>> +}
>>> +
>>> +/**
>>> + * firmware_map_remove() - remove a firmware mapping entry
>>> + * @start: Start of the memory range.
>>> + * @end:   End of the memory range (inclusive).
>>> + * @type:  Type of the memory range.
>>> + *
>>> + * removes a firmware mapping entry.
>>> + *
>>> + * Returns 0 on success, or -EINVAL if no entry.
>>> + **/
>>> +int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
>>> +{
>>> +    struct firmware_map_entry *entry;
>>> +
>>> +    entry = find_firmware_map_entry(start, end, type);
>>
>> Hmm, we cannot find the entry easily, because the end can be:
>> 1. start + size
>> 2. start + size - 1
>>
>> So, We should fix this bug first.
> 
> This is not a bug.
> 
> start and size arguments of firmware_map_remove() include
> acpi_memory_info->{start_addr, length}. And when creating a
> firmware_map_entry,
> the entry is created by acpi_memory_info->{start_addr, length}. So I don't
> think that we need care your comment.

If the memory device is hotpluged before the os starts, and the memory
map is included in e820 map, the entry will be created by firmware_map_add_early().

The function firmware_map_add_early() is called by e820_reserve_resources():
=====================
	for (i = 0; i < e820_saved.nr_map; i++) {
		struct e820entry *entry = &e820_saved.map[i];
		firmware_map_add_early(entry->addr,
			entry->addr + entry->size - 1,
			e820_type_to_string(entry->type));
	}
=====================

Note: the end is addr + size - 1, not addr + size.

In such case, you cannot find the entry.

Thanks
Wen Congyang

> 
>>
>>> +    if (!entry)
>>> +        return -EINVAL;
>>> +
>>> +    /* remove the memmap entry */
>>> +    remove_sysfs_fw_map_entry(entry);
>>> +
>>> +    firmware_map_remove_entry(entry);
>>> +
>>> +    release_firmware_map_entry(entry);
>>
>> I guess you want to free the memory in the above function. But I think
>> it is
>> not a good idea to free it here. We should free it when the
>> entry->kobj's kref
>> is decreased to 0.
> 
> Thanks.
> I'll update your comment at next version.
> 
> Thanks,
> Yasuaki Ishimatsu
> 
>>
>> Thanks
>> Wen Congyang
>>
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>   /*
>>>    * Sysfs functions
>>> -------------------------------------------------------------
>>>    */
>>>
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-kernel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>
>>
>> -- 
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
> 
> 
> 
> 

Re: [RFC PATCH 4/12] memory-hotplug : remove /sys/firmware/memmap/X sysfs

[Yasuaki Ishimatsu]

Hi Wen,

2012/06/28 17:38, Wen Congyang wrote:
> At 06/28/2012 04:07 PM, Yasuaki Ishimatsu Wrote:
>> Hi Wen,
>>
>> 2012/06/28 15:32, Wen Congyang wrote:
>>> At 06/27/2012 01:47 PM, Yasuaki Ishimatsu Wrote:
>>>> When (hot)adding memory into system, /sys/firmware/memmap/X/{end,
>>>> start, type}
>>>> sysfs files are created. But there is no code to remove these files.
>>>> The patch
>>>> implements the function to remove them.
>>>>
>>>> Note : The code does not free firmware_map_entry since there is no
>>>> way to free
>>>>          memory which is allocated by bootmem.
>>>>
>>>> CC: Len Brown <len.brown@intel.com>
>>>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>>>> CC: Paul Mackerras <paulus@samba.org>
>>>> CC: Christoph Lameter <cl@linux.com>
>>>> Cc: Minchan Kim <minchan.kim@gmail.com>
>>>> CC: Andrew Morton <akpm@linux-foundation.org>
>>>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>>>> CC: Wen Congyang <wency@cn.fujitsu.com>
>>>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>>>
>>>> ---
>>>>    drivers/firmware/memmap.c    |   71
>>>> +++++++++++++++++++++++++++++++++++++++++++
>>>>    include/linux/firmware-map.h |    6 +++
>>>>    mm/memory_hotplug.c          |    6 +++
>>>>    3 files changed, 82 insertions(+), 1 deletion(-)
>>>>
>>>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>>>> ===================================================================
>>>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c    2012-06-26
>>>> 13:37:30.546288901 +0900
>>>> +++ linux-3.5-rc4/mm/memory_hotplug.c    2012-06-26
>>>> 13:44:37.069955820 +0900
>>>> @@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
>>>>
>>>>    int remove_memory(int nid, u64 start, u64 size)
>>>>    {
>>>> -    return -EBUSY;
>>>> +    lock_memory_hotplug();
>>>> +    /* remove memmap entry */
>>>> +    firmware_map_remove(start, start + size, "System RAM");
>>>> +    unlock_memory_hotplug();
>>>> +    return 0;
>>>>
>>>>    }
>>>>    EXPORT_SYMBOL_GPL(remove_memory);
>>>> Index: linux-3.5-rc4/include/linux/firmware-map.h
>>>> ===================================================================
>>>> --- linux-3.5-rc4.orig/include/linux/firmware-map.h    2012-06-25
>>>> 04:53:04.000000000 +0900
>>>> +++ linux-3.5-rc4/include/linux/firmware-map.h    2012-06-26
>>>> 13:44:37.070955807 +0900
>>>> @@ -25,6 +25,7 @@
>>>>
>>>>    int firmware_map_add_early(u64 start, u64 end, const char *type);
>>>>    int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
>>>> +int firmware_map_remove(u64 start, u64 end, const char *type);
>>>>
>>>>    #else /* CONFIG_FIRMWARE_MEMMAP */
>>>>
>>>> @@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
>>>>        return 0;
>>>>    }
>>>>
>>>> +static inline int firmware_map_remove(u64 start, u64 end, const char
>>>> *type)
>>>> +{
>>>> +    return 0;
>>>> +}
>>>> +
>>>>    #endif /* CONFIG_FIRMWARE_MEMMAP */
>>>>
>>>>    #endif /* _LINUX_FIRMWARE_MAP_H */
>>>> Index: linux-3.5-rc4/drivers/firmware/memmap.c
>>>> ===================================================================
>>>> --- linux-3.5-rc4.orig/drivers/firmware/memmap.c    2012-06-25
>>>> 04:53:04.000000000 +0900
>>>> +++ linux-3.5-rc4/drivers/firmware/memmap.c    2012-06-26
>>>> 13:47:17.606948898 +0900
>>>> @@ -123,6 +123,16 @@ static int firmware_map_add_entry(u64 st
>>>>        return 0;
>>>>    }
>>>>
>>>> +/**
>>>> + * firmware_map_remove_entry() - Does the real work to remove a
>>>> firmware
>>>> + * memmap entry.
>>>> + * @entry: removed entry.
>>>> + **/
>>>> +static inline void firmware_map_remove_entry(struct
>>>> firmware_map_entry *entry)
>>>> +{
>>>> +    list_del(&entry->list);
>>>> +}
>>>> +
>>>>    /*
>>>>     * Add memmap entry on sysfs
>>>>     */
>>>> @@ -144,6 +154,31 @@ static int add_sysfs_fw_map_entry(struct
>>>>        return 0;
>>>>    }
>>>>
>>>> +/*
>>>> + * Remove memmap entry on sysfs
>>>> + */
>>>> +static inline void remove_sysfs_fw_map_entry(struct
>>>> firmware_map_entry *entry)
>>>> +{
>>>> +    kobject_del(&entry->kobj);
>>>> +}
>>>> +
>>>> +/*
>>>> + * Search memmap entry
>>>> + */
>>>> +
>>>> +struct firmware_map_entry * __meminit
>>>> +find_firmware_map_entry(u64 start, u64 end, const char *type)
>>>> +{
>>>> +    struct firmware_map_entry *entry;
>>>> +
>>>> +    list_for_each_entry(entry, &map_entries, list)
>>>> +        if ((entry->start == start) && (entry->end == end) &&
>>>> +            (!strcmp(entry->type, type)))
>>>> +            return entry;
>>>> +
>>>> +    return NULL;
>>>> +}
>>>> +
>>>>    /**
>>>>     * firmware_map_add_hotplug() - Adds a firmware mapping entry when
>>>> we do
>>>>     * memory hotplug.
>>>> @@ -196,6 +231,42 @@ int __init firmware_map_add_early(u64 st
>>>>        return firmware_map_add_entry(start, end, type, entry);
>>>>    }
>>>>
>>>> +void release_firmware_map_entry(struct firmware_map_entry *entry)
>>>> +{
>>>> +    /*
>>>> +     * FIXME : There is no idea.
>>>> +     *         How to free the entry which allocated bootmem?
>>>> +     */
>>>> +}
>>>> +
>>>> +/**
>>>> + * firmware_map_remove() - remove a firmware mapping entry
>>>> + * @start: Start of the memory range.
>>>> + * @end:   End of the memory range (inclusive).
>>>> + * @type:  Type of the memory range.
>>>> + *
>>>> + * removes a firmware mapping entry.
>>>> + *
>>>> + * Returns 0 on success, or -EINVAL if no entry.
>>>> + **/
>>>> +int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
>>>> +{
>>>> +    struct firmware_map_entry *entry;
>>>> +
>>>> +    entry = find_firmware_map_entry(start, end, type);
>>>
>>> Hmm, we cannot find the entry easily, because the end can be:
>>> 1. start + size
>>> 2. start + size - 1
>>>
>>> So, We should fix this bug first.
>>
>> This is not a bug.
>>
>> start and size arguments of firmware_map_remove() include
>> acpi_memory_info->{start_addr, length}. And when creating a
>> firmware_map_entry,
>> the entry is created by acpi_memory_info->{start_addr, length}. So I don't
>> think that we need care your comment.
>
> If the memory device is hotpluged before the os starts, and the memory
> map is included in e820 map, the entry will be created by firmware_map_add_early().
>
> The function firmware_map_add_early() is called by e820_reserve_resources():
> =====================
> 	for (i = 0; i < e820_saved.nr_map; i++) {
> 		struct e820entry *entry = &e820_saved.map[i];
> 		firmware_map_add_early(entry->addr,
> 			entry->addr + entry->size - 1,
> 			e820_type_to_string(entry->type));
> 	}
> =====================
>
> Note: the end is addr + size - 1, not addr + size.
>
> In such case, you cannot find the entry.

Thank you for your explanation. I understood it.
end argument of firmware_map_add_hotplug() has always "addr + size",
not "addr + size - 1". I think that changing argument of
firmware_map_add_early() is high risk since the function has been used
since early times. So, I will unify to "addr + size - 1".

Thanks,
Yasuaki Ishimatsu

> Thanks
> Wen Congyang
>
>>
>>>
>>>> +    if (!entry)
>>>> +        return -EINVAL;
>>>> +
>>>> +    /* remove the memmap entry */
>>>> +    remove_sysfs_fw_map_entry(entry);
>>>> +
>>>> +    firmware_map_remove_entry(entry);
>>>> +
>>>> +    release_firmware_map_entry(entry);
>>>
>>> I guess you want to free the memory in the above function. But I think
>>> it is
>>> not a good idea to free it here. We should free it when the
>>> entry->kobj's kref
>>> is decreased to 0.
>>
>> Thanks.
>> I'll update your comment at next version.
>>
>> Thanks,
>> Yasuaki Ishimatsu
>>
>>>
>>> Thanks
>>> Wen Congyang
>>>
>>>> +
>>>> +    return 0;
>>>> +}
>>>> +
>>>>    /*
>>>>     * Sysfs functions
>>>> -------------------------------------------------------------
>>>>     */
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-kernel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>>
>>>
>>> --
>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>>> the body to majordomo@kvack.org.  For more info on Linux MM,
>>> see: http://www.linux-mm.org/ .
>>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>>
>>
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Jiang Liu]

On 06/27/2012 01:44 PM, Yasuaki Ishimatsu wrote:
> When offline_pages() is called to offlined memory, the function fails since
> all memory has been offlined. In this case, the function should succeed.
> The patch adds the check function into offline_pages().
> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  drivers/base/memory.c  |   20 ++++++++++++++++++++
>  include/linux/memory.h |    1 +
>  mm/memory_hotplug.c    |    5 +++++
>  3 files changed, 26 insertions(+)
> 
> Index: linux-3.5-rc4/drivers/base/memory.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>  }
>  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
> 
> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
> +{
> +	struct memory_block *mem;
> +	struct mem_section *section;
> +	unsigned long pfn, section_nr;
> +
> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
> +		section_nr = pfn_to_section_nr(pfn);
> +		section = __nr_to_section(section_nr);
Is it possible for __nr_to_section return NULL here?

> +		mem = find_memory_block(section);
> +		if (!mem)
> +			continue;
> +		if (mem->state == MEM_OFFLINE)
> +			continue;
> +		return false;
> +	}
> +
> +	return true;
> +}
Need a put_dev(&mem->dev) for the last memory block device handled before return.

> +
>  /*
>   * register_memory - Setup a sysfs device for a memory block
>   */
> Index: linux-3.5-rc4/include/linux/memory.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>  							struct memory_block *);
>  extern struct memory_block *find_memory_block(struct mem_section *);
> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>  enum mem_add_context { BOOT, HOTPLUG };
>  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
> 
>  	lock_memory_hotplug();
> 
> +	if (memory_is_offline(start_pfn, end_pfn)) {
> +		ret = 0;
> +		goto out;
> +	}
> +
>  	zone = page_zone(pfn_to_page(start_pfn));
>  	node = zone_to_nid(zone);
>  	nr_pages = end_pfn - start_pfn;
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Jiang Liu]

On 06/27/2012 01:44 PM, Yasuaki Ishimatsu wrote:
> When offline_pages() is called to offlined memory, the function fails since
> all memory has been offlined. In this case, the function should succeed.
> The patch adds the check function into offline_pages().
> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  drivers/base/memory.c  |   20 ++++++++++++++++++++
>  include/linux/memory.h |    1 +
>  mm/memory_hotplug.c    |    5 +++++
>  3 files changed, 26 insertions(+)
> 
> Index: linux-3.5-rc4/drivers/base/memory.c
> ===================================================================
> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>  }
>  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
> 
> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
> +{
> +	struct memory_block *mem;
> +	struct mem_section *section;
> +	unsigned long pfn, section_nr;
> +
> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
> +		section_nr = pfn_to_section_nr(pfn);
> +		section = __nr_to_section(section_nr);
> +		mem = find_memory_block(section);
Seems find_memory_block_hinted() is more efficient than find_memory_block() here.

> +		if (!mem)
> +			continue;
> +		if (mem->state == MEM_OFFLINE)
> +			continue;
> +		return false;
> +	}
> +
> +	return true;
> +}
> +
>  /*
>   * register_memory - Setup a sysfs device for a memory block
>   */
> Index: linux-3.5-rc4/include/linux/memory.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>  							struct memory_block *);
>  extern struct memory_block *find_memory_block(struct mem_section *);
> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>  enum mem_add_context { BOOT, HOTPLUG };
>  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
> 
>  	lock_memory_hotplug();
> 
> +	if (memory_is_offline(start_pfn, end_pfn)) {
> +		ret = 0;
> +		goto out;
> +	}
> +
>  	zone = page_zone(pfn_to_page(start_pfn));
>  	node = zone_to_nid(zone);
>  	nr_pages = end_pfn - start_pfn;
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC PATCH 10/12] memory-hotplug : free memmap of sparse-vmemmap

[Jiang Liu]

On 06/27/2012 01:56 PM, Yasuaki Ishimatsu wrote:
> I don't think that all pages of virtual mapping in removed memory can be
> freed, since page which type is MIX_SECTION_INFO is difficult to free.
> So, the patch only frees page which type is SECTION_INFO at first.
> 
> CC: Len Brown <len.brown@intel.com>
> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> CC: Paul Mackerras <paulus@samba.org>
> CC: Christoph Lameter <cl@linux.com>
> Cc: Minchan Kim <minchan.kim@gmail.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> CC: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 
> ---
>  arch/x86/mm/init_64.c |   89 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/mm.h    |    2 +
>  mm/memory_hotplug.c   |    5 ++
>  mm/sparse.c           |    5 +-
>  4 files changed, 99 insertions(+), 2 deletions(-)
> 
> Index: linux-3.5-rc4/include/linux/mm.h
> ===================================================================
> --- linux-3.5-rc4.orig/include/linux/mm.h	2012-06-27 09:11:13.790150442 +0900
> +++ linux-3.5-rc4/include/linux/mm.h	2012-06-27 09:11:16.433117400 +0900
> @@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
>  void vmemmap_populate_print_last(void);
>  void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
>  				  unsigned long size);
> +void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
> +void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
> 
>  enum mf_flags {
>  	MF_COUNT_INCREASED = 1 << 0,
> Index: linux-3.5-rc4/mm/sparse.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/sparse.c	2012-06-27 09:06:35.317631878 +0900
> +++ linux-3.5-rc4/mm/sparse.c	2012-06-27 09:11:16.434117388 +0900
> @@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
>  	/* This will make the necessary allocations eventually. */
>  	return sparse_mem_map_populate(pnum, nid);
>  }
> -static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
> +static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
>  {
> -	return; /* XXX: Not implemented yet */
> +	vmemmap_kfree(page, nr_pages);
>  }
>  static void free_map_bootmem(struct page *page, unsigned long nr_pages)
>  {
> +	vmemmap_free_bootmem(page, nr_pages);
>  }
>  #else
>  static struct page *__kmalloc_section_memmap(unsigned long nr_pages)
> Index: linux-3.5-rc4/arch/x86/mm/init_64.c
> ===================================================================
> --- linux-3.5-rc4.orig/arch/x86/mm/init_64.c	2012-06-27 09:11:13.791150430 +0900
> +++ linux-3.5-rc4/arch/x86/mm/init_64.c	2012-06-27 09:11:59.254581998 +0900
> @@ -978,6 +978,95 @@ vmemmap_populate(struct page *start_page
>  	return 0;
>  }
> 
> +unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
> +				      struct page *page)
I think the third parameter should be "struct page **pp" instead of "struct page *page".
And "page = pte_page(*pte)" should be "*pp = pte_page(*pte)".
Otherwise the found page pointer can't be returned to the caller and vmemmap_kfree()
just sees random value in variable "page".

> +{
> +	pgd_t *pgd;
> +	pud_t *pud;
> +	pmd_t *pmd;
> +	pte_t *pte;
> +	unsigned long next;
> +
> +	page = NULL;
> +
> +	pgd = pgd_offset_k(addr);
> +	if (pgd_none(*pgd))
> +		return PAGE_SIZE;
> +
> +	pud = pud_offset(pgd, addr);
> +	if (pud_none(*pud))
> +		return PAGE_SIZE;
> +
> +	if (!cpu_has_pse) {
> +		next = (addr + PAGE_SIZE) & PAGE_MASK;
> +		pmd = pmd_offset(pud, addr);
> +		if (pmd_none(*pmd))
> +			return next;
> +
> +		pte = pte_offset_kernel(pmd, addr);
> +		if (pte_none(*pte))
> +			return next;
> +
> +		page = pte_page(*pte);
> +		pte_clear(&init_mm, addr, pte);
> +	} else {
> +		next = pmd_addr_end(addr, end);
> +
> +		pmd = pmd_offset(pud, addr);
> +		if (pmd_none(*pmd))
> +			return next;
> +
> +		page = pmd_page(*pmd);
> +		pmd_clear(pmd);
> +	}
> +
> +	return next;
> +}
> +
> +void __meminit
> +vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
> +{
> +	unsigned long addr = (unsigned long)memmap;
> +	unsigned long end = (unsigned long)(memmap + nr_pages);
> +	unsigned long next;
> +	unsigned int order;
> +	struct page *page;
> +
> +	for (; addr < end; addr = next) {
> +		next = find_and_clear_pte_page(addr, end, page);
> +		if (!page)
> +			continue;
> +
> +		if (is_vmalloc_addr(page))
> +			vfree(page);
> +		else {
> +			order = next - addr;
> +			free_pages((unsigned long)page,
> +				   get_order(sizeof(struct page) *  order));
> +		}
> +	}
> +}
> +
> +void __meminit
> +vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
> +{
> +	unsigned long addr = (unsigned long)memmap;
> +	unsigned long end = (unsigned long)(memmap + nr_pages);
> +	unsigned long next;
> +	struct page *page;
> +	unsigned long magic;
> +
> +	for (; addr < end; addr = next) {
> +		next = find_and_clear_pte_page(addr, end, page);
> +		if (!page)
> +			continue;
> +
> +		magic = (unsigned long) page->lru.next;
> +		if (magic == SECTION_INFO)
> +			put_page_bootmem(page);
> +	}
> +}
> +
>  void __meminit
>  register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
>  			     unsigned long size)
> Index: linux-3.5-rc4/mm/memory_hotplug.c
> ===================================================================
> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-27 09:11:13.789150454 +0900
> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-27 09:11:16.436117363 +0900
> @@ -303,6 +303,8 @@ static int __meminit __add_section(int n
>  #ifdef CONFIG_SPARSEMEM_VMEMMAP
>  static int __remove_section(struct zone *zone, struct mem_section *ms)
>  {
> +	unsigned long flags;
> +	struct pglist_data *pgdat = zone->zone_pgdat;
>  	int ret;
> 
>  	if (!valid_section(ms))
> @@ -310,6 +312,9 @@ static int __remove_section(struct zone
> 
>  	ret = unregister_memory_section(ms);
> 
> +	pgdat_resize_lock(pgdat, &flags);
> +	sparse_remove_one_section(zone, ms);
> +	pgdat_resize_unlock(pgdat, &flags);
>  	return ret;
>  }
>  #else
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi Jiang,

Thank you for your feedback.

2012/07/01 0:46, Jiang Liu wrote:
> On 06/27/2012 01:44 PM, Yasuaki Ishimatsu wrote:
>> When offline_pages() is called to offlined memory, the function fails since
>> all memory has been offlined. In this case, the function should succeed.
>> The patch adds the check function into offline_pages().
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   drivers/base/memory.c  |   20 ++++++++++++++++++++
>>   include/linux/memory.h |    1 +
>>   mm/memory_hotplug.c    |    5 +++++
>>   3 files changed, 26 insertions(+)
>>
>> Index: linux-3.5-rc4/drivers/base/memory.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
>> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
>> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>>   }
>>   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
>>
>> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
>> +{
>> +	struct memory_block *mem;
>> +	struct mem_section *section;
>> +	unsigned long pfn, section_nr;
>> +
>> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
>> +		section_nr = pfn_to_section_nr(pfn);
>> +		section = __nr_to_section(section_nr);
> Is it possible for __nr_to_section return NULL here?

Yes. I'll add NULL check.

> 
>> +		mem = find_memory_block(section);
>> +		if (!mem)
>> +			continue;
>> +		if (mem->state == MEM_OFFLINE)
>> +			continue;
>> +		return false;
>> +	}
>> +
>> +	return true;
>> +}
> Need a put_dev(&mem->dev) for the last memory block device handled before return.

Thanks.
I think kobject_put(&mem->dev.kobj) should be handled for all memory block
devices found by find_memory_block(). I'll update it.

Thanks,
Yasuaki Ishimatsu

> 
>> +
>>   /*
>>    * register_memory - Setup a sysfs device for a memory block
>>    */
>> Index: linux-3.5-rc4/include/linux/memory.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
>> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>>   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>>   							struct memory_block *);
>>   extern struct memory_block *find_memory_block(struct mem_section *);
>> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>>   #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>>   enum mem_add_context { BOOT, HOTPLUG };
>>   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
>> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>>
>>   	lock_memory_hotplug();
>>
>> +	if (memory_is_offline(start_pfn, end_pfn)) {
>> +		ret = 0;
>> +		goto out;
>> +	}
>> +
>>   	zone = page_zone(pfn_to_page(start_pfn));
>>   	node = zone_to_nid(zone);
>>   	nr_pages = end_pfn - start_pfn;
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

Re: [RFC PATCH 2/12] memory-hogplug : check memory offline in offline_pages

[Yasuaki Ishimatsu]

Hi Jiang,

2012/07/01 0:51, Jiang Liu wrote:
> On 06/27/2012 01:44 PM, Yasuaki Ishimatsu wrote:
>> When offline_pages() is called to offlined memory, the function fails since
>> all memory has been offlined. In this case, the function should succeed.
>> The patch adds the check function into offline_pages().
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   drivers/base/memory.c  |   20 ++++++++++++++++++++
>>   include/linux/memory.h |    1 +
>>   mm/memory_hotplug.c    |    5 +++++
>>   3 files changed, 26 insertions(+)
>>
>> Index: linux-3.5-rc4/drivers/base/memory.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/drivers/base/memory.c	2012-06-26 13:28:16.726211752 +0900
>> +++ linux-3.5-rc4/drivers/base/memory.c	2012-06-26 13:34:22.423639904 +0900
>> @@ -70,6 +70,26 @@ void unregister_memory_isolate_notifier(
>>   }
>>   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
>>
>> +bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn)
>> +{
>> +	struct memory_block *mem;
>> +	struct mem_section *section;
>> +	unsigned long pfn, section_nr;
>> +
>> +	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
>> +		section_nr = pfn_to_section_nr(pfn);
>> +		section = __nr_to_section(section_nr);
>> +		mem = find_memory_block(section);
> Seems find_memory_block_hinted() is more efficient than find_memory_block() here.

Thanks. I'll update it.

Thanks,
Yasuaki Ishimatsu

> 
>> +		if (!mem)
>> +			continue;
>> +		if (mem->state == MEM_OFFLINE)
>> +			continue;
>> +		return false;
>> +	}
>> +
>> +	return true;
>> +}
>> +
>>   /*
>>    * register_memory - Setup a sysfs device for a memory block
>>    */
>> Index: linux-3.5-rc4/include/linux/memory.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/memory.h	2012-06-25 04:53:04.000000000 +0900
>> +++ linux-3.5-rc4/include/linux/memory.h	2012-06-26 13:34:22.424639891 +0900
>> @@ -120,6 +120,7 @@ extern int memory_isolate_notify(unsigne
>>   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
>>   							struct memory_block *);
>>   extern struct memory_block *find_memory_block(struct mem_section *);
>> +extern bool memory_is_offline(unsigned long start_pfn, unsigned long end_pfn);
>>   #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
>>   enum mem_add_context { BOOT, HOTPLUG };
>>   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-26 13:28:16.743211538 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-26 13:48:38.264940468 +0900
>> @@ -887,6 +887,11 @@ static int __ref offline_pages(unsigned
>>
>>   	lock_memory_hotplug();
>>
>> +	if (memory_is_offline(start_pfn, end_pfn)) {
>> +		ret = 0;
>> +		goto out;
>> +	}
>> +
>>   	zone = page_zone(pfn_to_page(start_pfn));
>>   	node = zone_to_nid(zone);
>>   	nr_pages = end_pfn - start_pfn;
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

Re: [RFC PATCH 10/12] memory-hotplug : free memmap of sparse-vmemmap

[Yasuaki Ishimatsu]

Hi Jiang,

2012/07/01 0:58, Jiang Liu wrote:
> On 06/27/2012 01:56 PM, Yasuaki Ishimatsu wrote:
>> I don't think that all pages of virtual mapping in removed memory can be
>> freed, since page which type is MIX_SECTION_INFO is difficult to free.
>> So, the patch only frees page which type is SECTION_INFO at first.
>>
>> CC: Len Brown <len.brown@intel.com>
>> CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>> CC: Paul Mackerras <paulus@samba.org>
>> CC: Christoph Lameter <cl@linux.com>
>> Cc: Minchan Kim <minchan.kim@gmail.com>
>> CC: Andrew Morton <akpm@linux-foundation.org>
>> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>> CC: Wen Congyang <wency@cn.fujitsu.com>
>> Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>>
>> ---
>>   arch/x86/mm/init_64.c |   89 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/mm.h    |    2 +
>>   mm/memory_hotplug.c   |    5 ++
>>   mm/sparse.c           |    5 +-
>>   4 files changed, 99 insertions(+), 2 deletions(-)
>>
>> Index: linux-3.5-rc4/include/linux/mm.h
>> ===================================================================
>> --- linux-3.5-rc4.orig/include/linux/mm.h	2012-06-27 09:11:13.790150442 +0900
>> +++ linux-3.5-rc4/include/linux/mm.h	2012-06-27 09:11:16.433117400 +0900
>> @@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
>>   void vmemmap_populate_print_last(void);
>>   void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
>>   				  unsigned long size);
>> +void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
>> +void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
>>
>>   enum mf_flags {
>>   	MF_COUNT_INCREASED = 1 << 0,
>> Index: linux-3.5-rc4/mm/sparse.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/sparse.c	2012-06-27 09:06:35.317631878 +0900
>> +++ linux-3.5-rc4/mm/sparse.c	2012-06-27 09:11:16.434117388 +0900
>> @@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
>>   	/* This will make the necessary allocations eventually. */
>>   	return sparse_mem_map_populate(pnum, nid);
>>   }
>> -static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
>> +static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
>>   {
>> -	return; /* XXX: Not implemented yet */
>> +	vmemmap_kfree(page, nr_pages);
>>   }
>>   static void free_map_bootmem(struct page *page, unsigned long nr_pages)
>>   {
>> +	vmemmap_free_bootmem(page, nr_pages);
>>   }
>>   #else
>>   static struct page *__kmalloc_section_memmap(unsigned long nr_pages)
>> Index: linux-3.5-rc4/arch/x86/mm/init_64.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/arch/x86/mm/init_64.c	2012-06-27 09:11:13.791150430 +0900
>> +++ linux-3.5-rc4/arch/x86/mm/init_64.c	2012-06-27 09:11:59.254581998 +0900
>> @@ -978,6 +978,95 @@ vmemmap_populate(struct page *start_page
>>   	return 0;
>>   }
>>
>> +unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
>> +				      struct page *page)
> I think the third parameter should be "struct page **pp" instead of "struct page *page".
> And "page = pte_page(*pte)" should be "*pp = pte_page(*pte)".
> Otherwise the found page pointer can't be returned to the caller and vmemmap_kfree()
> just sees random value in variable "page".

Oh, you are right. I'll update it.

Thanks,
Yasuaki Ishimatsu

>> +{
>> +	pgd_t *pgd;
>> +	pud_t *pud;
>> +	pmd_t *pmd;
>> +	pte_t *pte;
>> +	unsigned long next;
>> +
>> +	page = NULL;
>> +
>> +	pgd = pgd_offset_k(addr);
>> +	if (pgd_none(*pgd))
>> +		return PAGE_SIZE;
>> +
>> +	pud = pud_offset(pgd, addr);
>> +	if (pud_none(*pud))
>> +		return PAGE_SIZE;
>> +
>> +	if (!cpu_has_pse) {
>> +		next = (addr + PAGE_SIZE) & PAGE_MASK;
>> +		pmd = pmd_offset(pud, addr);
>> +		if (pmd_none(*pmd))
>> +			return next;
>> +
>> +		pte = pte_offset_kernel(pmd, addr);
>> +		if (pte_none(*pte))
>> +			return next;
>> +
>> +		page = pte_page(*pte);
>> +		pte_clear(&init_mm, addr, pte);
>> +	} else {
>> +		next = pmd_addr_end(addr, end);
>> +
>> +		pmd = pmd_offset(pud, addr);
>> +		if (pmd_none(*pmd))
>> +			return next;
>> +
>> +		page = pmd_page(*pmd);
>> +		pmd_clear(pmd);
>> +	}
>> +
>> +	return next;
>> +}
>> +
>> +void __meminit
>> +vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
>> +{
>> +	unsigned long addr = (unsigned long)memmap;
>> +	unsigned long end = (unsigned long)(memmap + nr_pages);
>> +	unsigned long next;
>> +	unsigned int order;
>> +	struct page *page;
>> +
>> +	for (; addr < end; addr = next) {
>> +		next = find_and_clear_pte_page(addr, end, page);
>> +		if (!page)
>> +			continue;
>> +
>> +		if (is_vmalloc_addr(page))
>> +			vfree(page);
>> +		else {
>> +			order = next - addr;
>> +			free_pages((unsigned long)page,
>> +				   get_order(sizeof(struct page) *  order));
>> +		}
>> +	}
>> +}
>> +
>> +void __meminit
>> +vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
>> +{
>> +	unsigned long addr = (unsigned long)memmap;
>> +	unsigned long end = (unsigned long)(memmap + nr_pages);
>> +	unsigned long next;
>> +	struct page *page;
>> +	unsigned long magic;
>> +
>> +	for (; addr < end; addr = next) {
>> +		next = find_and_clear_pte_page(addr, end, page);
>> +		if (!page)
>> +			continue;
>> +
>> +		magic = (unsigned long) page->lru.next;
>> +		if (magic == SECTION_INFO)
>> +			put_page_bootmem(page);
>> +	}
>> +}
>> +
>>   void __meminit
>>   register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
>>   			     unsigned long size)
>> Index: linux-3.5-rc4/mm/memory_hotplug.c
>> ===================================================================
>> --- linux-3.5-rc4.orig/mm/memory_hotplug.c	2012-06-27 09:11:13.789150454 +0900
>> +++ linux-3.5-rc4/mm/memory_hotplug.c	2012-06-27 09:11:16.436117363 +0900
>> @@ -303,6 +303,8 @@ static int __meminit __add_section(int n
>>   #ifdef CONFIG_SPARSEMEM_VMEMMAP
>>   static int __remove_section(struct zone *zone, struct mem_section *ms)
>>   {
>> +	unsigned long flags;
>> +	struct pglist_data *pgdat = zone->zone_pgdat;
>>   	int ret;
>>
>>   	if (!valid_section(ms))
>> @@ -310,6 +312,9 @@ static int __remove_section(struct zone
>>
>>   	ret = unregister_memory_section(ms);
>>
>> +	pgdat_resize_lock(pgdat, &flags);
>> +	sparse_remove_one_section(zone, ms);
>> +	pgdat_resize_unlock(pgdat, &flags);
>>   	return ret;
>>   }
>>   #else
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>