linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Cody P Schafer <cody@linux.vnet.ibm.com>
To: Linux MM <linux-mm@kvack.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Cody P Schafer <cody@linux.vnet.ibm.com>,
	Simon Jeons <simon.jeons@gmail.com>
Subject: [RFC PATCH v3 22/31] page_alloc: transplant pages that are being flushed from the per-cpu lists
Date: Thu,  2 May 2013 17:00:54 -0700	[thread overview]
Message-ID: <1367539263-19999-23-git-send-email-cody@linux.vnet.ibm.com> (raw)
In-Reply-To: <1367539263-19999-1-git-send-email-cody@linux.vnet.ibm.com>

In free_pcppages_bulk(), check if a page needs to be moved to a new
node/zone & then perform the transplant (in a slightly defered manner).

Signed-off-by: Cody P Schafer <cody@linux.vnet.ibm.com>
---
 mm/page_alloc.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 38a2161..879ab9d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -644,13 +644,14 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 	int migratetype = 0;
 	int batch_free = 0;
 	int to_free = count;
+	struct page *pos, *page;
+	LIST_HEAD(need_move);
 
 	spin_lock(&zone->lock);
 	zone->all_unreclaimable = 0;
 	zone->pages_scanned = 0;
 
 	while (to_free) {
-		struct page *page;
 		struct list_head *list;
 
 		/*
@@ -673,11 +674,23 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 
 		do {
 			int mt;	/* migratetype of the to-be-freed page */
+			int dest_nid;
 
 			page = list_entry(list->prev, struct page, lru);
 			/* must delete as __free_one_page list manipulates */
 			list_del(&page->lru);
 			mt = get_freepage_migratetype(page);
+
+			dest_nid = dnuma_page_needs_move(page);
+			if (dest_nid != NUMA_NO_NODE) {
+				dnuma_prior_free_to_new_zone(page, 0,
+						nid_zone(dest_nid,
+							page_zonenum(page)),
+						dest_nid);
+				list_add(&page->lru, &need_move);
+				continue;
+			}
+
 			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
 			__free_one_page(page, zone, 0, mt);
 			trace_mm_page_pcpu_drain(page, 0, mt);
@@ -689,6 +702,27 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 		} while (--to_free && --batch_free && !list_empty(list));
 	}
 	spin_unlock(&zone->lock);
+
+	list_for_each_entry_safe(page, pos, &need_move, lru) {
+		struct zone *dest_zone = page_zone(page);
+		int mt;
+
+		spin_lock(&dest_zone->lock);
+
+		VM_BUG_ON(dest_zone != page_zone(page));
+		pr_devel("freeing pcp page %pK with changed node\n", page);
+		list_del(&page->lru);
+		mt = get_freepage_migratetype(page);
+		__free_one_page(page, dest_zone, 0, mt);
+		trace_mm_page_pcpu_drain(page, 0, mt);
+
+		/* XXX: fold into "post_free_to_new_zone()" ? */
+		if (is_migrate_cma(mt))
+			__mod_zone_page_state(dest_zone, NR_FREE_CMA_PAGES, 1);
+		dnuma_post_free_to_new_zone(0);
+
+		spin_unlock(&dest_zone->lock);
+	}
 }
 
 static void free_one_page(struct zone *zone, struct page *page, int order,
-- 
1.8.2.2


  parent reply	other threads:[~2013-05-03  0:02 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-05-03  0:00 [RFC PATCH v3 00/31] Dynamic NUMA: Runtime NUMA memory layout reconfiguration Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 01/31] rbtree: add postorder iteration functions Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 02/31] rbtree: add rbtree_postorder_for_each_entry_safe() helper Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 03/31] mm/memory_hotplug: factor out zone+pgdat growth Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 04/31] memory_hotplug: export ensure_zone_is_initialized() in mm/internal.h Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 05/31] mm/memory_hotplug: use {pgdat,zone}_is_empty() when resizing zones & pgdats Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 06/31] mm: add nid_zone() helper Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 07/31] mm: Add Dynamic NUMA Kconfig Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 08/31] page_alloc: add return_pages_to_zone() when DYNAMIC_NUMA is enabled Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 09/31] page_alloc: in move_freepages(), skip pages instead of VM_BUG on node differences Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 10/31] page_alloc: when dynamic numa is enabled, don't check that all pages in a block belong to the same zone Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 11/31] page-flags dnuma: reserve a pageflag for determining if a page needs a node lookup Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 12/31] memory_hotplug: factor out locks in mem_online_cpu() Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 13/31] mm: add memlayout & dnuma to track pfn->nid & transplant pages between nodes Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 14/31] mm: memlayout+dnuma: add debugfs interface Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 15/31] drivers/base/memory.c: alphabetize headers Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 16/31] drivers/base/node,memory: rename function to match interface Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 17/31] drivers/base/node: rename unregister_mem_blk_under_nodes() to be more acurate Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 18/31] drivers/base/node: add unregister_mem_block_under_nodes() Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 19/31] mm: memory,memlayout: add refresh_memory_blocks() for Dynamic NUMA Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 20/31] page_alloc: use dnuma to transplant newly freed pages in __free_pages_ok() Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 21/31] page_alloc: use dnuma to transplant newly freed pages in free_hot_cold_page() Cody P Schafer
2013-05-03  0:00 ` Cody P Schafer [this message]
2013-05-03  0:00 ` [RFC PATCH v3 23/31] x86: memlayout: add a arch specific inital memlayout setter Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 24/31] init/main: call memlayout_global_init() in start_kernel() Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 25/31] dnuma: memlayout: add memory_add_physaddr_to_nid() for memory_hotplug Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 26/31] x86/mm/numa: when dnuma is enabled, use memlayout to handle memory hotplug's physaddr_to_nid Cody P Schafer
2013-05-03  0:00 ` [RFC PATCH v3 27/31] mm/memory_hotplug: VM_BUG if nid is too large Cody P Schafer
2013-05-03  0:01 ` [RFC PATCH v3 28/31] mm/page_alloc: in page_outside_zone_boundaries(), avoid premature decisions Cody P Schafer
2013-05-03  0:01 ` [RFC PATCH v3 29/31] mm/page_alloc: make pr_err() in page_outside_zone_boundaries() more useful Cody P Schafer
2013-05-03  0:01 ` [RFC PATCH v3 30/31] mm/page_alloc: use manage_pages instead of present pages when calculating default_zonelist_order() Cody P Schafer
2013-05-03  0:01 ` [RFC PATCH v3 31/31] mm: add a early_param "extra_nr_node_ids" to increase nr_node_ids above the minimum by a percentage Cody P Schafer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1367539263-19999-23-git-send-email-cody@linux.vnet.ibm.com \
    --to=cody@linux.vnet.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=simon.jeons@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).