From: Cody P Schafer <cody@linux.vnet.ibm.com>
To: Linux MM <linux-mm@kvack.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
Cody P Schafer <cody@linux.vnet.ibm.com>,
Simon Jeons <simon.jeons@gmail.com>
Subject: [RFC PATCH v3 22/31] page_alloc: transplant pages that are being flushed from the per-cpu lists
Date: Thu, 2 May 2013 17:00:54 -0700 [thread overview]
Message-ID: <1367539263-19999-23-git-send-email-cody@linux.vnet.ibm.com> (raw)
In-Reply-To: <1367539263-19999-1-git-send-email-cody@linux.vnet.ibm.com>
In free_pcppages_bulk(), check if a page needs to be moved to a new
node/zone & then perform the transplant (in a slightly defered manner).
Signed-off-by: Cody P Schafer <cody@linux.vnet.ibm.com>
---
mm/page_alloc.c | 36 +++++++++++++++++++++++++++++++++++-
1 file changed, 35 insertions(+), 1 deletion(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 38a2161..879ab9d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -644,13 +644,14 @@ static void free_pcppages_bulk(struct zone *zone, int count,
int migratetype = 0;
int batch_free = 0;
int to_free = count;
+ struct page *pos, *page;
+ LIST_HEAD(need_move);
spin_lock(&zone->lock);
zone->all_unreclaimable = 0;
zone->pages_scanned = 0;
while (to_free) {
- struct page *page;
struct list_head *list;
/*
@@ -673,11 +674,23 @@ static void free_pcppages_bulk(struct zone *zone, int count,
do {
int mt; /* migratetype of the to-be-freed page */
+ int dest_nid;
page = list_entry(list->prev, struct page, lru);
/* must delete as __free_one_page list manipulates */
list_del(&page->lru);
mt = get_freepage_migratetype(page);
+
+ dest_nid = dnuma_page_needs_move(page);
+ if (dest_nid != NUMA_NO_NODE) {
+ dnuma_prior_free_to_new_zone(page, 0,
+ nid_zone(dest_nid,
+ page_zonenum(page)),
+ dest_nid);
+ list_add(&page->lru, &need_move);
+ continue;
+ }
+
/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
__free_one_page(page, zone, 0, mt);
trace_mm_page_pcpu_drain(page, 0, mt);
@@ -689,6 +702,27 @@ static void free_pcppages_bulk(struct zone *zone, int count,
} while (--to_free && --batch_free && !list_empty(list));
}
spin_unlock(&zone->lock);
+
+ list_for_each_entry_safe(page, pos, &need_move, lru) {
+ struct zone *dest_zone = page_zone(page);
+ int mt;
+
+ spin_lock(&dest_zone->lock);
+
+ VM_BUG_ON(dest_zone != page_zone(page));
+ pr_devel("freeing pcp page %pK with changed node\n", page);
+ list_del(&page->lru);
+ mt = get_freepage_migratetype(page);
+ __free_one_page(page, dest_zone, 0, mt);
+ trace_mm_page_pcpu_drain(page, 0, mt);
+
+ /* XXX: fold into "post_free_to_new_zone()" ? */
+ if (is_migrate_cma(mt))
+ __mod_zone_page_state(dest_zone, NR_FREE_CMA_PAGES, 1);
+ dnuma_post_free_to_new_zone(0);
+
+ spin_unlock(&dest_zone->lock);
+ }
}
static void free_one_page(struct zone *zone, struct page *page, int order,
--
1.8.2.2
next prev parent reply other threads:[~2013-05-03 0:02 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-03 0:00 [RFC PATCH v3 00/31] Dynamic NUMA: Runtime NUMA memory layout reconfiguration Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 01/31] rbtree: add postorder iteration functions Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 02/31] rbtree: add rbtree_postorder_for_each_entry_safe() helper Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 03/31] mm/memory_hotplug: factor out zone+pgdat growth Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 04/31] memory_hotplug: export ensure_zone_is_initialized() in mm/internal.h Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 05/31] mm/memory_hotplug: use {pgdat,zone}_is_empty() when resizing zones & pgdats Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 06/31] mm: add nid_zone() helper Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 07/31] mm: Add Dynamic NUMA Kconfig Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 08/31] page_alloc: add return_pages_to_zone() when DYNAMIC_NUMA is enabled Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 09/31] page_alloc: in move_freepages(), skip pages instead of VM_BUG on node differences Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 10/31] page_alloc: when dynamic numa is enabled, don't check that all pages in a block belong to the same zone Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 11/31] page-flags dnuma: reserve a pageflag for determining if a page needs a node lookup Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 12/31] memory_hotplug: factor out locks in mem_online_cpu() Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 13/31] mm: add memlayout & dnuma to track pfn->nid & transplant pages between nodes Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 14/31] mm: memlayout+dnuma: add debugfs interface Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 15/31] drivers/base/memory.c: alphabetize headers Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 16/31] drivers/base/node,memory: rename function to match interface Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 17/31] drivers/base/node: rename unregister_mem_blk_under_nodes() to be more acurate Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 18/31] drivers/base/node: add unregister_mem_block_under_nodes() Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 19/31] mm: memory,memlayout: add refresh_memory_blocks() for Dynamic NUMA Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 20/31] page_alloc: use dnuma to transplant newly freed pages in __free_pages_ok() Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 21/31] page_alloc: use dnuma to transplant newly freed pages in free_hot_cold_page() Cody P Schafer
2013-05-03 0:00 ` Cody P Schafer [this message]
2013-05-03 0:00 ` [RFC PATCH v3 23/31] x86: memlayout: add a arch specific inital memlayout setter Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 24/31] init/main: call memlayout_global_init() in start_kernel() Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 25/31] dnuma: memlayout: add memory_add_physaddr_to_nid() for memory_hotplug Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 26/31] x86/mm/numa: when dnuma is enabled, use memlayout to handle memory hotplug's physaddr_to_nid Cody P Schafer
2013-05-03 0:00 ` [RFC PATCH v3 27/31] mm/memory_hotplug: VM_BUG if nid is too large Cody P Schafer
2013-05-03 0:01 ` [RFC PATCH v3 28/31] mm/page_alloc: in page_outside_zone_boundaries(), avoid premature decisions Cody P Schafer
2013-05-03 0:01 ` [RFC PATCH v3 29/31] mm/page_alloc: make pr_err() in page_outside_zone_boundaries() more useful Cody P Schafer
2013-05-03 0:01 ` [RFC PATCH v3 30/31] mm/page_alloc: use manage_pages instead of present pages when calculating default_zonelist_order() Cody P Schafer
2013-05-03 0:01 ` [RFC PATCH v3 31/31] mm: add a early_param "extra_nr_node_ids" to increase nr_node_ids above the minimum by a percentage Cody P Schafer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1367539263-19999-23-git-send-email-cody@linux.vnet.ibm.com \
--to=cody@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=simon.jeons@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).