* [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
For stable kernels first three patches should be enough.
changes since v1:
mm/balloon_compaction: ignore anonymous pages
* no changes
mm/balloon_compaction: keep ballooned pages away from normal migration path
* fix compilation without CONFIG_BALLOON_COMPACTION
mm/balloon_compaction: isolate balloon pages without lru_lock
* no changes
mm: introduce common page state for ballooned memory
* move __Set/ClearPageBalloon into linux/mm.h
* remove inc/dec_zone_page_state from __Set/ClearPageBalloon
mm/balloon_compaction: use common page ballooning
* call inc/dec_zone_page_state from balloon_page_insert/delete
mm/balloon_compaction: general cleanup
* fix compilation without CONFIG_MIGRATION
* fix compilation without CONFIG_BALLOON_COMPACTION
---
Konstantin Khlebnikov (6):
mm/balloon_compaction: ignore anonymous pages
mm/balloon_compaction: keep ballooned pages away from normal migration path
mm/balloon_compaction: isolate balloon pages without lru_lock
mm: introduce common page state for ballooned memory
mm/balloon_compaction: use common page ballooning
mm/balloon_compaction: general cleanup
Documentation/filesystems/proc.txt | 2
drivers/base/node.c | 16 +-
drivers/virtio/Kconfig | 1
drivers/virtio/virtio_balloon.c | 77 +++--------
fs/proc/meminfo.c | 6 +
fs/proc/page.c | 3
include/linux/balloon_compaction.h | 223 ++++++--------------------------
include/linux/migrate.h | 11 --
include/linux/mm.h | 20 +++
include/linux/mmzone.h | 3
include/linux/pagemap.h | 18 ---
include/uapi/linux/kernel-page-flags.h | 1
mm/Kconfig | 7 +
mm/Makefile | 3
mm/balloon_compaction.c | 219 ++++++++++---------------------
mm/compaction.c | 9 +
mm/migrate.c | 29 +---
mm/vmscan.c | 2
mm/vmstat.c | 8 +
tools/vm/page-types.c | 1
20 files changed, 210 insertions(+), 449 deletions(-)
--
Signature
^ permalink raw reply [flat|nested] 63+ messages in thread
* [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
For stable kernels first three patches should be enough.
changes since v1:
mm/balloon_compaction: ignore anonymous pages
* no changes
mm/balloon_compaction: keep ballooned pages away from normal migration path
* fix compilation without CONFIG_BALLOON_COMPACTION
mm/balloon_compaction: isolate balloon pages without lru_lock
* no changes
mm: introduce common page state for ballooned memory
* move __Set/ClearPageBalloon into linux/mm.h
* remove inc/dec_zone_page_state from __Set/ClearPageBalloon
mm/balloon_compaction: use common page ballooning
* call inc/dec_zone_page_state from balloon_page_insert/delete
mm/balloon_compaction: general cleanup
* fix compilation without CONFIG_MIGRATION
* fix compilation without CONFIG_BALLOON_COMPACTION
---
Konstantin Khlebnikov (6):
mm/balloon_compaction: ignore anonymous pages
mm/balloon_compaction: keep ballooned pages away from normal migration path
mm/balloon_compaction: isolate balloon pages without lru_lock
mm: introduce common page state for ballooned memory
mm/balloon_compaction: use common page ballooning
mm/balloon_compaction: general cleanup
Documentation/filesystems/proc.txt | 2
drivers/base/node.c | 16 +-
drivers/virtio/Kconfig | 1
drivers/virtio/virtio_balloon.c | 77 +++--------
fs/proc/meminfo.c | 6 +
fs/proc/page.c | 3
include/linux/balloon_compaction.h | 223 ++++++--------------------------
include/linux/migrate.h | 11 --
include/linux/mm.h | 20 +++
include/linux/mmzone.h | 3
include/linux/pagemap.h | 18 ---
include/uapi/linux/kernel-page-flags.h | 1
mm/Kconfig | 7 +
mm/Makefile | 3
mm/balloon_compaction.c | 219 ++++++++++---------------------
mm/compaction.c | 9 +
mm/migrate.c | 29 +---
mm/vmscan.c | 2
mm/vmstat.c | 8 +
tools/vm/page-types.c | 1
20 files changed, 210 insertions(+), 449 deletions(-)
--
Signature
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Sasha Levin reported KASAN splash inside isolate_migratepages_range().
Problem is in function __is_movable_balloon_page() which tests AS_BALLOON_MAP
in page->mapping->flags. This function has no protection against anonymous
pages. As result it tried to check address space flags in inside anon-vma.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Reported-by: Sasha Levin <sasha.levin@oracle.com>
Link: http://lkml.kernel.org/p/53E6CEAA.9020105@oracle.com
Cc: stable <stable@vger.kernel.org> # v3.8
---
include/linux/balloon_compaction.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 089743a..53d482e 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)
static inline bool __is_movable_balloon_page(struct page *page)
{
struct address_space *mapping = page->mapping;
- return mapping_balloon(mapping);
+ return !PageAnon(page) && mapping_balloon(mapping);
}
/*
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Sasha Levin reported KASAN splash inside isolate_migratepages_range().
Problem is in function __is_movable_balloon_page() which tests AS_BALLOON_MAP
in page->mapping->flags. This function has no protection against anonymous
pages. As result it tried to check address space flags in inside anon-vma.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Reported-by: Sasha Levin <sasha.levin@oracle.com>
Link: http://lkml.kernel.org/p/53E6CEAA.9020105@oracle.com
Cc: stable <stable@vger.kernel.org> # v3.8
---
include/linux/balloon_compaction.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 089743a..53d482e 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)
static inline bool __is_movable_balloon_page(struct page *page)
{
struct address_space *mapping = page->mapping;
- return mapping_balloon(mapping);
+ return !PageAnon(page) && mapping_balloon(mapping);
}
/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Proper testing shows yet another problem in balloon migration: it works only
once for each page. balloon_page_movable() check page flags and page_count.
In __unmap_and_move page is locked, reference counter is elevated, so
balloon_page_movable() _always_ fails here. As result in __unmap_and_move()
migration goes to the normal migration path.
Balloon ->migratepage() is so special, it returns MIGRATEPAGE_BALLOON_SUCCESS
instead of MIGRATEPAGE_SUCCESS. After that in move_to_new_page() successfully
migrated page got NULL into its mapping pointer and loses connectivity with
balloon and ability for further migration.
It's safe to use __is_movable_balloon_page here: page is isolated and pinned.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Cc: stable <stable@vger.kernel.org> # v3.8
---
include/linux/balloon_compaction.h | 5 +++++
mm/migrate.c | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 53d482e..284fc1d 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)
list_del(&page->lru);
}
+static inline bool __is_movable_balloon_page(struct page *page)
+{
+ return false;
+}
+
static inline bool balloon_page_movable(struct page *page)
{
return false;
diff --git a/mm/migrate.c b/mm/migrate.c
index 905b1aa..57c94f9 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(balloon_page_movable(page))) {
+ if (unlikely(__is_movable_balloon_page(page))) {
/*
* A ballooned page does not need any special attention from
* physical to virtual reverse mapping procedures.
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Proper testing shows yet another problem in balloon migration: it works only
once for each page. balloon_page_movable() check page flags and page_count.
In __unmap_and_move page is locked, reference counter is elevated, so
balloon_page_movable() _always_ fails here. As result in __unmap_and_move()
migration goes to the normal migration path.
Balloon ->migratepage() is so special, it returns MIGRATEPAGE_BALLOON_SUCCESS
instead of MIGRATEPAGE_SUCCESS. After that in move_to_new_page() successfully
migrated page got NULL into its mapping pointer and loses connectivity with
balloon and ability for further migration.
It's safe to use __is_movable_balloon_page here: page is isolated and pinned.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Cc: stable <stable@vger.kernel.org> # v3.8
---
include/linux/balloon_compaction.h | 5 +++++
mm/migrate.c | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 53d482e..284fc1d 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)
list_del(&page->lru);
}
+static inline bool __is_movable_balloon_page(struct page *page)
+{
+ return false;
+}
+
static inline bool balloon_page_movable(struct page *page)
{
return false;
diff --git a/mm/migrate.c b/mm/migrate.c
index 905b1aa..57c94f9 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(balloon_page_movable(page))) {
+ if (unlikely(__is_movable_balloon_page(page))) {
/*
* A ballooned page does not need any special attention from
* physical to virtual reverse mapping procedures.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
LRU-lock isn't required for balloon page isolation. This check makes migration
of some ballooned pages mostly impossible because isolate_migratepages_range()
drops LRU lock periodically.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Cc: stable <stable@vger.kernel.org> # v3.8
---
mm/compaction.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 73466e1..ad58f73 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
*/
if (!PageLRU(page)) {
if (unlikely(balloon_page_movable(page))) {
- if (locked && balloon_page_isolate(page)) {
+ if (balloon_page_isolate(page)) {
/* Successfully isolated */
goto isolate_success;
}
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
LRU-lock isn't required for balloon page isolation. This check makes migration
of some ballooned pages mostly impossible because isolate_migratepages_range()
drops LRU lock periodically.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Cc: stable <stable@vger.kernel.org> # v3.8
---
mm/compaction.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 73466e1..ad58f73 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
*/
if (!PageLRU(page)) {
if (unlikely(balloon_page_movable(page))) {
- if (locked && balloon_page_isolate(page)) {
+ if (balloon_page_isolate(page)) {
/* Successfully isolated */
goto isolate_success;
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
state of page->_mapcount counter. There is no conflict because ballooned pages
cannot be mapped and cannot be in buddy allocator.
Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
userspace via /proc/kpageflags as KPF_BALLOON.
All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
ballooning driver which wants use this feature.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
Documentation/filesystems/proc.txt | 2 ++
drivers/base/node.c | 16 ++++++++++------
fs/proc/meminfo.c | 6 ++++++
fs/proc/page.c | 3 +++
include/linux/mm.h | 20 ++++++++++++++++++++
include/linux/mmzone.h | 3 +++
include/uapi/linux/kernel-page-flags.h | 1 +
mm/Kconfig | 5 +++++
mm/vmstat.c | 8 +++++++-
tools/vm/page-types.c | 1 +
10 files changed, 58 insertions(+), 7 deletions(-)
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index eb8a10e..154a345 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -796,6 +796,7 @@ VmallocTotal: 112216 kB
VmallocUsed: 428 kB
VmallocChunk: 111088 kB
AnonHugePages: 49152 kB
+BalloonPages: 0 kB
MemTotal: Total usable ram (i.e. physical ram minus a few reserved
bits and the kernel binary code)
@@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new
Writeback: Memory which is actively being written back to the disk
AnonPages: Non-file backed pages mapped into userspace page tables
AnonHugePages: Non-file backed huge pages mapped into userspace page tables
+BalloonPages: Memory which was ballooned, not included into MemTotal
Mapped: files which have been mmaped, such as libraries
Slab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as caches
diff --git a/drivers/base/node.c b/drivers/base/node.c
index c6d3ae0..59e565c 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"Node %d AnonHugePages: %8lu kB\n"
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ "Node %d BalloonPages: %8lu kB\n"
+#endif
,
nid, K(node_page_state(nid, NR_FILE_DIRTY)),
nid, K(node_page_state(nid, NR_WRITEBACK)),
@@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev,
nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) +
node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)),
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))
- , nid,
- K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) *
- HPAGE_PMD_NR));
-#else
- nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)));
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+ ,nid, K(node_page_state(nid,
+ NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)
+#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ ,nid, K(node_page_state(nid, NR_BALLOON_PAGES))
#endif
+ );
n += hugetlb_report_node_meminfo(nid, buf + n);
return n;
}
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index aa1eee0..f897fbf 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"AnonHugePages: %8lu kB\n"
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ "BalloonPages: %8lu kB\n"
+#endif
,
K(i.totalram),
K(i.freeram),
@@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) *
HPAGE_PMD_NR)
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ ,K(global_page_state(NR_BALLOON_PAGES))
+#endif
);
hugetlb_report_meminfo(m);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index e647c55..1e3187d 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)
if (PageBuddy(page))
u |= 1 << KPF_BUDDY;
+ if (PageBalloon(page))
+ u |= 1 << KPF_BALLOON;
+
u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index da79328..a9a27ea 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)
atomic_set(&page->_mapcount, -1);
}
+#define PAGE_BALLOON_MAPCOUNT_VALUE (-256)
+
+static inline int PageBalloon(struct page *page)
+{
+ return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
+ atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
+}
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
+ atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(&page->_mapcount, -1);
+}
+
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 318df70..d88fd01 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -157,6 +157,9 @@ enum zone_stat_item {
WORKINGSET_NODERECLAIM,
NR_ANON_TRANSPARENT_HUGEPAGES,
NR_FREE_CMA_PAGES,
+#ifdef CONFIG_MEMORY_BALLOON
+ NR_BALLOON_PAGES,
+#endif
NR_VM_ZONE_STAT_ITEMS };
/*
diff --git a/include/uapi/linux/kernel-page-flags.h b/include/uapi/linux/kernel-page-flags.h
index 5116a0e..2f96d23 100644
--- a/include/uapi/linux/kernel-page-flags.h
+++ b/include/uapi/linux/kernel-page-flags.h
@@ -31,6 +31,7 @@
#define KPF_KSM 21
#define KPF_THP 22
+#define KPF_BALLOON 23
#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index 886db21..72e0db0 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK
boolean
#
+# support for memory ballooning
+config MEMORY_BALLOON
+ boolean
+
+#
# support for memory balloon compaction
config BALLOON_COMPACTION
bool "Allow for balloon memory compaction/migration"
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 84c688d..ff7bd29 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
TEXT_FOR_HIGHMEM(xx) xx "_movable",
const char * const vmstat_text[] = {
- /* Zoned VM counters */
+ /* enum zone_stat_item countes */
"nr_free_pages",
"nr_alloc_batch",
"nr_inactive_anon",
@@ -794,10 +794,16 @@ const char * const vmstat_text[] = {
"workingset_nodereclaim",
"nr_anon_transparent_hugepages",
"nr_free_cma",
+#ifdef CONFIG_MEMORY_BALLOON
+ "nr_balloon_pages",
+#endif
+
+ /* enum writeback_stat_item counters */
"nr_dirty_threshold",
"nr_dirty_background_threshold",
#ifdef CONFIG_VM_EVENT_COUNTERS
+ /* enum vm_event_item counters */
"pgpgin",
"pgpgout",
"pswpin",
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index c4d6d2e..264fbc2 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -132,6 +132,7 @@ static const char * const page_flag_names[] = {
[KPF_NOPAGE] = "n:nopage",
[KPF_KSM] = "x:ksm",
[KPF_THP] = "t:thp",
+ [KPF_BALLOON] = "o:balloon",
[KPF_RESERVED] = "r:reserved",
[KPF_MLOCKED] = "m:mlocked",
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
state of page->_mapcount counter. There is no conflict because ballooned pages
cannot be mapped and cannot be in buddy allocator.
Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
userspace via /proc/kpageflags as KPF_BALLOON.
All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
ballooning driver which wants use this feature.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
Documentation/filesystems/proc.txt | 2 ++
drivers/base/node.c | 16 ++++++++++------
fs/proc/meminfo.c | 6 ++++++
fs/proc/page.c | 3 +++
include/linux/mm.h | 20 ++++++++++++++++++++
include/linux/mmzone.h | 3 +++
include/uapi/linux/kernel-page-flags.h | 1 +
mm/Kconfig | 5 +++++
mm/vmstat.c | 8 +++++++-
tools/vm/page-types.c | 1 +
10 files changed, 58 insertions(+), 7 deletions(-)
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index eb8a10e..154a345 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -796,6 +796,7 @@ VmallocTotal: 112216 kB
VmallocUsed: 428 kB
VmallocChunk: 111088 kB
AnonHugePages: 49152 kB
+BalloonPages: 0 kB
MemTotal: Total usable ram (i.e. physical ram minus a few reserved
bits and the kernel binary code)
@@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new
Writeback: Memory which is actively being written back to the disk
AnonPages: Non-file backed pages mapped into userspace page tables
AnonHugePages: Non-file backed huge pages mapped into userspace page tables
+BalloonPages: Memory which was ballooned, not included into MemTotal
Mapped: files which have been mmaped, such as libraries
Slab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as caches
diff --git a/drivers/base/node.c b/drivers/base/node.c
index c6d3ae0..59e565c 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"Node %d AnonHugePages: %8lu kB\n"
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ "Node %d BalloonPages: %8lu kB\n"
+#endif
,
nid, K(node_page_state(nid, NR_FILE_DIRTY)),
nid, K(node_page_state(nid, NR_WRITEBACK)),
@@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev,
nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) +
node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)),
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))
- , nid,
- K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) *
- HPAGE_PMD_NR));
-#else
- nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)));
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+ ,nid, K(node_page_state(nid,
+ NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)
+#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ ,nid, K(node_page_state(nid, NR_BALLOON_PAGES))
#endif
+ );
n += hugetlb_report_node_meminfo(nid, buf + n);
return n;
}
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index aa1eee0..f897fbf 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"AnonHugePages: %8lu kB\n"
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ "BalloonPages: %8lu kB\n"
+#endif
,
K(i.totalram),
K(i.freeram),
@@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) *
HPAGE_PMD_NR)
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ ,K(global_page_state(NR_BALLOON_PAGES))
+#endif
);
hugetlb_report_meminfo(m);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index e647c55..1e3187d 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)
if (PageBuddy(page))
u |= 1 << KPF_BUDDY;
+ if (PageBalloon(page))
+ u |= 1 << KPF_BALLOON;
+
u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index da79328..a9a27ea 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)
atomic_set(&page->_mapcount, -1);
}
+#define PAGE_BALLOON_MAPCOUNT_VALUE (-256)
+
+static inline int PageBalloon(struct page *page)
+{
+ return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
+ atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
+}
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
+ atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(&page->_mapcount, -1);
+}
+
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 318df70..d88fd01 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -157,6 +157,9 @@ enum zone_stat_item {
WORKINGSET_NODERECLAIM,
NR_ANON_TRANSPARENT_HUGEPAGES,
NR_FREE_CMA_PAGES,
+#ifdef CONFIG_MEMORY_BALLOON
+ NR_BALLOON_PAGES,
+#endif
NR_VM_ZONE_STAT_ITEMS };
/*
diff --git a/include/uapi/linux/kernel-page-flags.h b/include/uapi/linux/kernel-page-flags.h
index 5116a0e..2f96d23 100644
--- a/include/uapi/linux/kernel-page-flags.h
+++ b/include/uapi/linux/kernel-page-flags.h
@@ -31,6 +31,7 @@
#define KPF_KSM 21
#define KPF_THP 22
+#define KPF_BALLOON 23
#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index 886db21..72e0db0 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK
boolean
#
+# support for memory ballooning
+config MEMORY_BALLOON
+ boolean
+
+#
# support for memory balloon compaction
config BALLOON_COMPACTION
bool "Allow for balloon memory compaction/migration"
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 84c688d..ff7bd29 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
TEXT_FOR_HIGHMEM(xx) xx "_movable",
const char * const vmstat_text[] = {
- /* Zoned VM counters */
+ /* enum zone_stat_item countes */
"nr_free_pages",
"nr_alloc_batch",
"nr_inactive_anon",
@@ -794,10 +794,16 @@ const char * const vmstat_text[] = {
"workingset_nodereclaim",
"nr_anon_transparent_hugepages",
"nr_free_cma",
+#ifdef CONFIG_MEMORY_BALLOON
+ "nr_balloon_pages",
+#endif
+
+ /* enum writeback_stat_item counters */
"nr_dirty_threshold",
"nr_dirty_background_threshold",
#ifdef CONFIG_VM_EVENT_COUNTERS
+ /* enum vm_event_item counters */
"pgpgin",
"pgpgout",
"pswpin",
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index c4d6d2e..264fbc2 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -132,6 +132,7 @@ static const char * const page_flag_names[] = {
[KPF_NOPAGE] = "n:nopage",
[KPF_KSM] = "x:ksm",
[KPF_THP] = "t:thp",
+ [KPF_BALLOON] = "o:balloon",
[KPF_RESERVED] = "r:reserved",
[KPF_MLOCKED] = "m:mlocked",
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
with PageBalloon which is stored directly in the struct page.
All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
drivers/virtio/Kconfig | 1
include/linux/balloon_compaction.h | 135 ++++++------------------------------
mm/Kconfig | 2 -
mm/Makefile | 3 +
mm/balloon_compaction.c | 7 +-
mm/compaction.c | 9 +-
mm/migrate.c | 6 +-
mm/vmscan.c | 2 -
8 files changed, 39 insertions(+), 126 deletions(-)
diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index c6683f2..00b2286 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -25,6 +25,7 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
+ select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 284fc1d..09f8c5a 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -46,6 +46,8 @@
#include <linux/gfp.h>
#include <linux/err.h>
+#ifdef CONFIG_MEMORY_BALLOON
+
/*
* Balloon device information descriptor.
* This struct is used to allow the common balloon compaction interface
@@ -93,91 +95,6 @@ static inline void balloon_page_free(struct page *page)
__free_page(page);
}
-#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
-extern struct address_space
-*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops);
-
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- kfree(balloon_mapping);
-}
-
-/*
- * page_flags_cleared - helper to perform balloon @page ->flags tests.
- *
- * As balloon pages are obtained from buddy and we do not play with page->flags
- * at driver level (exception made when we get the page lock for compaction),
- * we can safely identify a ballooned page by checking if the
- * PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also
- * helps us skip ballooned pages that are locked for compaction or release, thus
- * mitigating their racy check at balloon_page_movable()
- */
-static inline bool page_flags_cleared(struct page *page)
-{
- return !(page->flags & PAGE_FLAGS_CHECK_AT_PREP);
-}
-
-/*
- * __is_movable_balloon_page - helper to perform @page mapping->flags tests
- */
-static inline bool __is_movable_balloon_page(struct page *page)
-{
- struct address_space *mapping = page->mapping;
- return !PageAnon(page) && mapping_balloon(mapping);
-}
-
-/*
- * balloon_page_movable - test page->mapping->flags to identify balloon pages
- * that can be moved by compaction/migration.
- *
- * This function is used at core compaction's page isolation scheme, therefore
- * most pages exposed to it are not enlisted as balloon pages and so, to avoid
- * undesired side effects like racing against __free_pages(), we cannot afford
- * holding the page locked while testing page->mapping->flags here.
- *
- * As we might return false positives in the case of a balloon page being just
- * released under us, the page->mapping->flags need to be re-tested later,
- * under the proper page lock, at the functions that will be coping with the
- * balloon page case.
- */
-static inline bool balloon_page_movable(struct page *page)
-{
- /*
- * Before dereferencing and testing mapping->flags, let's make sure
- * this is not a page that uses ->mapping in a different way
- */
- if (page_flags_cleared(page) && !page_mapped(page) &&
- page_count(page) == 1)
- return __is_movable_balloon_page(page);
-
- return false;
-}
-
-/*
- * isolated_balloon_page - identify an isolated balloon page on private
- * compaction/migration page lists.
- *
- * After a compaction thread isolates a balloon page for migration, it raises
- * the page refcount to prevent concurrent compaction threads from re-isolating
- * the same page. For that reason putback_movable_pages(), or other routines
- * that need to identify isolated balloon pages on private pagelists, cannot
- * rely on balloon_page_movable() to accomplish the task.
- */
-static inline bool isolated_balloon_page(struct page *page)
-{
- /* Already isolated balloon pages, by default, have a raised refcount */
- if (page_flags_cleared(page) && !page_mapped(page) &&
- page_count(page) >= 2)
- return __is_movable_balloon_page(page);
-
- return false;
-}
-
/*
* balloon_page_insert - insert a page into the balloon's page list and make
* the page->mapping assignment accordingly.
@@ -192,6 +109,8 @@ static inline void balloon_page_insert(struct page *page,
struct address_space *mapping,
struct list_head *head)
{
+ __SetPageBalloon(page);
+ inc_zone_page_state(page, NR_BALLOON_PAGES);
page->mapping = mapping;
list_add(&page->lru, head);
}
@@ -206,10 +125,29 @@ static inline void balloon_page_insert(struct page *page,
*/
static inline void balloon_page_delete(struct page *page)
{
+ __ClearPageBalloon(page);
+ dec_zone_page_state(page, NR_BALLOON_PAGES);
page->mapping = NULL;
list_del(&page->lru);
}
+#endif /* CONFIG_MEMORY_BALLOON */
+
+#ifdef CONFIG_BALLOON_COMPACTION
+
+extern bool balloon_page_isolate(struct page *page);
+extern void balloon_page_putback(struct page *page);
+extern int balloon_page_migrate(struct page *newpage,
+ struct page *page, enum migrate_mode mode);
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
+
/*
* balloon_page_device - get the b_dev_info descriptor for the balloon device
* that enqueues the given page.
@@ -246,33 +184,6 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
return;
}
-static inline void balloon_page_insert(struct page *page,
- struct address_space *mapping,
- struct list_head *head)
-{
- list_add(&page->lru, head);
-}
-
-static inline void balloon_page_delete(struct page *page)
-{
- list_del(&page->lru);
-}
-
-static inline bool __is_movable_balloon_page(struct page *page)
-{
- return false;
-}
-
-static inline bool balloon_page_movable(struct page *page)
-{
- return false;
-}
-
-static inline bool isolated_balloon_page(struct page *page)
-{
- return false;
-}
-
static inline bool balloon_page_isolate(struct page *page)
{
return false;
diff --git a/mm/Kconfig b/mm/Kconfig
index 72e0db0..e09cf0a 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -237,7 +237,7 @@ config MEMORY_BALLOON
config BALLOON_COMPACTION
bool "Allow for balloon memory compaction/migration"
def_bool y
- depends on COMPACTION && VIRTIO_BALLOON
+ depends on COMPACTION && MEMORY_BALLOON
help
Memory fragmentation introduced by ballooning might reduce
significantly the number of 2MB contiguous memory blocks that can be
diff --git a/mm/Makefile b/mm/Makefile
index a96e3a1..b2f18dc 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o balloon_compaction.o vmacache.o \
+ compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
+obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..a942081 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)
* Prevent concurrent compaction threads from isolating
* an already isolated balloon page by refcount check.
*/
- if (__is_movable_balloon_page(page) &&
- page_count(page) == 2) {
+ if (PageBalloon(page) && page_count(page) == 2) {
__isolate_balloon_page(page);
unlock_page(page);
return true;
@@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)
*/
lock_page(page);
- if (__is_movable_balloon_page(page)) {
+ if (PageBalloon(page)) {
__putback_balloon_page(page);
/* drop the extra ref count taken for page isolation */
put_page(page);
@@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,
*/
BUG_ON(!trylock_page(newpage));
- if (WARN_ON(!__is_movable_balloon_page(page))) {
+ if (WARN_ON(!PageBalloon(page))) {
dump_page(page, "not movable balloon page");
unlock_page(newpage);
return rc;
diff --git a/mm/compaction.c b/mm/compaction.c
index ad58f73..7d9d92e 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* Skip any other type of page
*/
if (!PageLRU(page)) {
- if (unlikely(balloon_page_movable(page))) {
- if (balloon_page_isolate(page)) {
- /* Successfully isolated */
- goto isolate_success;
- }
+ if (unlikely(PageBalloon(page)) &&
+ balloon_page_isolate(page)) {
+ /* Successfully isolated */
+ goto isolate_success;
}
continue;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 57c94f9..a4939b1 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)
list_del(&page->lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- if (unlikely(isolated_balloon_page(page)))
+ if (unlikely(PageBalloon(page)))
balloon_page_putback(page);
else
putback_lru_page(page);
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(__is_movable_balloon_page(page))) {
+ if (unlikely(PageBalloon(page))) {
/*
* A ballooned page does not need any special attention from
* physical to virtual reverse mapping procedures.
@@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
rc = __unmap_and_move(page, newpage, force, mode);
+#ifdef CONFIG_MEMORY_BALLOON
if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
/*
* A ballooned page has been migrated already.
@@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
balloon_page_free(page);
return MIGRATEPAGE_SUCCESS;
}
+#endif
out:
if (rc != -EAGAIN) {
/*
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 1a71b8b..88dd901 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,
list_for_each_entry_safe(page, next, page_list, lru) {
if (page_is_file_cache(page) && !PageDirty(page) &&
- !isolated_balloon_page(page)) {
+ !PageBalloon(page)) {
ClearPageActive(page);
list_move(&page->lru, &clean_pages);
}
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
with PageBalloon which is stored directly in the struct page.
All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
drivers/virtio/Kconfig | 1
include/linux/balloon_compaction.h | 135 ++++++------------------------------
mm/Kconfig | 2 -
mm/Makefile | 3 +
mm/balloon_compaction.c | 7 +-
mm/compaction.c | 9 +-
mm/migrate.c | 6 +-
mm/vmscan.c | 2 -
8 files changed, 39 insertions(+), 126 deletions(-)
diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index c6683f2..00b2286 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -25,6 +25,7 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
+ select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 284fc1d..09f8c5a 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -46,6 +46,8 @@
#include <linux/gfp.h>
#include <linux/err.h>
+#ifdef CONFIG_MEMORY_BALLOON
+
/*
* Balloon device information descriptor.
* This struct is used to allow the common balloon compaction interface
@@ -93,91 +95,6 @@ static inline void balloon_page_free(struct page *page)
__free_page(page);
}
-#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
-extern struct address_space
-*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops);
-
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- kfree(balloon_mapping);
-}
-
-/*
- * page_flags_cleared - helper to perform balloon @page ->flags tests.
- *
- * As balloon pages are obtained from buddy and we do not play with page->flags
- * at driver level (exception made when we get the page lock for compaction),
- * we can safely identify a ballooned page by checking if the
- * PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also
- * helps us skip ballooned pages that are locked for compaction or release, thus
- * mitigating their racy check at balloon_page_movable()
- */
-static inline bool page_flags_cleared(struct page *page)
-{
- return !(page->flags & PAGE_FLAGS_CHECK_AT_PREP);
-}
-
-/*
- * __is_movable_balloon_page - helper to perform @page mapping->flags tests
- */
-static inline bool __is_movable_balloon_page(struct page *page)
-{
- struct address_space *mapping = page->mapping;
- return !PageAnon(page) && mapping_balloon(mapping);
-}
-
-/*
- * balloon_page_movable - test page->mapping->flags to identify balloon pages
- * that can be moved by compaction/migration.
- *
- * This function is used at core compaction's page isolation scheme, therefore
- * most pages exposed to it are not enlisted as balloon pages and so, to avoid
- * undesired side effects like racing against __free_pages(), we cannot afford
- * holding the page locked while testing page->mapping->flags here.
- *
- * As we might return false positives in the case of a balloon page being just
- * released under us, the page->mapping->flags need to be re-tested later,
- * under the proper page lock, at the functions that will be coping with the
- * balloon page case.
- */
-static inline bool balloon_page_movable(struct page *page)
-{
- /*
- * Before dereferencing and testing mapping->flags, let's make sure
- * this is not a page that uses ->mapping in a different way
- */
- if (page_flags_cleared(page) && !page_mapped(page) &&
- page_count(page) == 1)
- return __is_movable_balloon_page(page);
-
- return false;
-}
-
-/*
- * isolated_balloon_page - identify an isolated balloon page on private
- * compaction/migration page lists.
- *
- * After a compaction thread isolates a balloon page for migration, it raises
- * the page refcount to prevent concurrent compaction threads from re-isolating
- * the same page. For that reason putback_movable_pages(), or other routines
- * that need to identify isolated balloon pages on private pagelists, cannot
- * rely on balloon_page_movable() to accomplish the task.
- */
-static inline bool isolated_balloon_page(struct page *page)
-{
- /* Already isolated balloon pages, by default, have a raised refcount */
- if (page_flags_cleared(page) && !page_mapped(page) &&
- page_count(page) >= 2)
- return __is_movable_balloon_page(page);
-
- return false;
-}
-
/*
* balloon_page_insert - insert a page into the balloon's page list and make
* the page->mapping assignment accordingly.
@@ -192,6 +109,8 @@ static inline void balloon_page_insert(struct page *page,
struct address_space *mapping,
struct list_head *head)
{
+ __SetPageBalloon(page);
+ inc_zone_page_state(page, NR_BALLOON_PAGES);
page->mapping = mapping;
list_add(&page->lru, head);
}
@@ -206,10 +125,29 @@ static inline void balloon_page_insert(struct page *page,
*/
static inline void balloon_page_delete(struct page *page)
{
+ __ClearPageBalloon(page);
+ dec_zone_page_state(page, NR_BALLOON_PAGES);
page->mapping = NULL;
list_del(&page->lru);
}
+#endif /* CONFIG_MEMORY_BALLOON */
+
+#ifdef CONFIG_BALLOON_COMPACTION
+
+extern bool balloon_page_isolate(struct page *page);
+extern void balloon_page_putback(struct page *page);
+extern int balloon_page_migrate(struct page *newpage,
+ struct page *page, enum migrate_mode mode);
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
+
/*
* balloon_page_device - get the b_dev_info descriptor for the balloon device
* that enqueues the given page.
@@ -246,33 +184,6 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
return;
}
-static inline void balloon_page_insert(struct page *page,
- struct address_space *mapping,
- struct list_head *head)
-{
- list_add(&page->lru, head);
-}
-
-static inline void balloon_page_delete(struct page *page)
-{
- list_del(&page->lru);
-}
-
-static inline bool __is_movable_balloon_page(struct page *page)
-{
- return false;
-}
-
-static inline bool balloon_page_movable(struct page *page)
-{
- return false;
-}
-
-static inline bool isolated_balloon_page(struct page *page)
-{
- return false;
-}
-
static inline bool balloon_page_isolate(struct page *page)
{
return false;
diff --git a/mm/Kconfig b/mm/Kconfig
index 72e0db0..e09cf0a 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -237,7 +237,7 @@ config MEMORY_BALLOON
config BALLOON_COMPACTION
bool "Allow for balloon memory compaction/migration"
def_bool y
- depends on COMPACTION && VIRTIO_BALLOON
+ depends on COMPACTION && MEMORY_BALLOON
help
Memory fragmentation introduced by ballooning might reduce
significantly the number of 2MB contiguous memory blocks that can be
diff --git a/mm/Makefile b/mm/Makefile
index a96e3a1..b2f18dc 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o balloon_compaction.o vmacache.o \
+ compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
+obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..a942081 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)
* Prevent concurrent compaction threads from isolating
* an already isolated balloon page by refcount check.
*/
- if (__is_movable_balloon_page(page) &&
- page_count(page) == 2) {
+ if (PageBalloon(page) && page_count(page) == 2) {
__isolate_balloon_page(page);
unlock_page(page);
return true;
@@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)
*/
lock_page(page);
- if (__is_movable_balloon_page(page)) {
+ if (PageBalloon(page)) {
__putback_balloon_page(page);
/* drop the extra ref count taken for page isolation */
put_page(page);
@@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,
*/
BUG_ON(!trylock_page(newpage));
- if (WARN_ON(!__is_movable_balloon_page(page))) {
+ if (WARN_ON(!PageBalloon(page))) {
dump_page(page, "not movable balloon page");
unlock_page(newpage);
return rc;
diff --git a/mm/compaction.c b/mm/compaction.c
index ad58f73..7d9d92e 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* Skip any other type of page
*/
if (!PageLRU(page)) {
- if (unlikely(balloon_page_movable(page))) {
- if (balloon_page_isolate(page)) {
- /* Successfully isolated */
- goto isolate_success;
- }
+ if (unlikely(PageBalloon(page)) &&
+ balloon_page_isolate(page)) {
+ /* Successfully isolated */
+ goto isolate_success;
}
continue;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 57c94f9..a4939b1 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)
list_del(&page->lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- if (unlikely(isolated_balloon_page(page)))
+ if (unlikely(PageBalloon(page)))
balloon_page_putback(page);
else
putback_lru_page(page);
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(__is_movable_balloon_page(page))) {
+ if (unlikely(PageBalloon(page))) {
/*
* A ballooned page does not need any special attention from
* physical to virtual reverse mapping procedures.
@@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
rc = __unmap_and_move(page, newpage, force, mode);
+#ifdef CONFIG_MEMORY_BALLOON
if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
/*
* A ballooned page has been migrated already.
@@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
balloon_page_free(page);
return MIGRATEPAGE_SUCCESS;
}
+#endif
out:
if (rc != -EAGAIN) {
/*
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 1a71b8b..88dd901 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,
list_for_each_entry_safe(page, next, page_list, lru) {
if (page_is_file_cache(page) && !PageDirty(page) &&
- !isolated_balloon_page(page)) {
+ !PageBalloon(page)) {
ClearPageActive(page);
list_move(&page->lru, &clean_pages);
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 6/6] mm/balloon_compaction: general cleanup
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
* move special branch for balloon migraion into migrate_pages
* remove special mapping for balloon and its flag AS_BALLOON_MAP
* embed struct balloon_dev_info into struct virtio_balloon
* cleanup balloon_page_dequeue, kill balloon_page_free
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
drivers/virtio/virtio_balloon.c | 77 ++++---------
include/linux/balloon_compaction.h | 127 ++++++---------------
include/linux/migrate.h | 11 --
include/linux/pagemap.h | 18 ---
mm/balloon_compaction.c | 214 ++++++++++++------------------------
mm/migrate.c | 29 +----
6 files changed, 134 insertions(+), 342 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 25ebe8e..c84d6a8 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -59,7 +59,7 @@ struct virtio_balloon
* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
* to num_pages above.
*/
- struct balloon_dev_info *vb_dev_info;
+ struct balloon_dev_info vb_dev_info;
/* Synchronize access/update to this struct virtio_balloon elements */
struct mutex balloon_lock;
@@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)
static void fill_balloon(struct virtio_balloon *vb, size_t num)
{
- struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
+ struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
/* Find pfns pointing at start of each page, get pages and free them. */
for (i = 0; i < num; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
struct page *page = balloon_pfn_to_page(pfns[i]);
- balloon_page_free(page);
adjust_managed_page_count(page, 1);
+ put_page(page);
}
}
static void leak_balloon(struct virtio_balloon *vb, size_t num)
{
struct page *page;
- struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
+ struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)
return 0;
}
-static const struct address_space_operations virtio_balloon_aops;
#ifdef CONFIG_BALLOON_COMPACTION
/*
* virtballoon_migratepage - perform the balloon page migration on behalf of
* a compation thread. (called under page lock)
- * @mapping: the page->mapping which will be assigned to the new migrated page.
+ * @vb_dev_info: the balloon device
* @newpage: page that will replace the isolated page after migration finishes.
* @page : the isolated (old) page that is about to be migrated to newpage.
* @mode : compaction mode -- not used for balloon page migration.
@@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;
* This function preforms the balloon page migration task.
* Called through balloon_mapping->a_ops->migratepage
*/
-static int virtballoon_migratepage(struct address_space *mapping,
+static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
struct page *newpage, struct page *page, enum migrate_mode mode)
{
- struct balloon_dev_info *vb_dev_info = balloon_page_device(page);
- struct virtio_balloon *vb;
+ struct virtio_balloon *vb = container_of(vb_dev_info,
+ struct virtio_balloon, vb_dev_info);
unsigned long flags;
- BUG_ON(!vb_dev_info);
-
- vb = vb_dev_info->balloon_device;
-
/*
* In order to avoid lock contention while migrating pages concurrently
* to leak_balloon() or fill_balloon() we just give up the balloon_lock
@@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,
if (!mutex_trylock(&vb->balloon_lock))
return -EAGAIN;
+ get_page(newpage); /* balloon reference */
+
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
- balloon_page_insert(newpage, mapping, &vb_dev_info->pages);
+ balloon_page_insert(vb_dev_info, newpage);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, newpage);
tell_host(vb, vb->inflate_vq);
- /*
- * balloon's page migration 2nd step -- deflate "page"
- *
- * It's safe to delete page->lru here because this page is at
- * an isolated migration list, and this step is expected to happen here
- */
- balloon_page_delete(page);
+ /* balloon's page migration 2nd step -- deflate "page" */
+ balloon_page_delete(page, true);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, page);
tell_host(vb, vb->deflate_vq);
mutex_unlock(&vb->balloon_lock);
- return MIGRATEPAGE_BALLOON_SUCCESS;
-}
+ put_page(page); /* balloon reference */
-/* define the balloon_mapping->a_ops callback to allow balloon page migration */
-static const struct address_space_operations virtio_balloon_aops = {
- .migratepage = virtballoon_migratepage,
-};
+ return MIGRATEPAGE_SUCCESS;
+}
#endif /* CONFIG_BALLOON_COMPACTION */
static int virtballoon_probe(struct virtio_device *vdev)
{
struct virtio_balloon *vb;
- struct address_space *vb_mapping;
- struct balloon_dev_info *vb_devinfo;
int err;
vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
@@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev)
vb->vdev = vdev;
vb->need_stats_update = 0;
- vb_devinfo = balloon_devinfo_alloc(vb);
- if (IS_ERR(vb_devinfo)) {
- err = PTR_ERR(vb_devinfo);
- goto out_free_vb;
- }
-
- vb_mapping = balloon_mapping_alloc(vb_devinfo,
- (balloon_compaction_check()) ?
- &virtio_balloon_aops : NULL);
- if (IS_ERR(vb_mapping)) {
- /*
- * IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP
- * This means !CONFIG_BALLOON_COMPACTION, otherwise we get off.
- */
- err = PTR_ERR(vb_mapping);
- if (err != -EOPNOTSUPP)
- goto out_free_vb_devinfo;
- }
-
- vb->vb_dev_info = vb_devinfo;
+ balloon_devinfo_init(&vb->vb_dev_info);
+#ifdef CONFIG_BALLOON_COMPACTION
+ vb->vb_dev_info.migratepage = virtballoon_migratepage;
+#endif
err = init_vqs(vb);
if (err)
- goto out_free_vb_mapping;
+ goto out_free_vb;
vb->thread = kthread_run(balloon, vb, "vballoon");
if (IS_ERR(vb->thread)) {
@@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
out_del_vqs:
vdev->config->del_vqs(vdev);
-out_free_vb_mapping:
- balloon_mapping_free(vb_mapping);
-out_free_vb_devinfo:
- balloon_devinfo_free(vb_devinfo);
out_free_vb:
kfree(vb);
out:
@@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)
kthread_stop(vb->thread);
remove_common(vb);
- balloon_mapping_free(vb->vb_dev_info->mapping);
- balloon_devinfo_free(vb->vb_dev_info);
kfree(vb);
}
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 09f8c5a..ad112fcc6 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -56,96 +56,58 @@
* balloon driver as a page book-keeper for its registered balloon devices.
*/
struct balloon_dev_info {
- void *balloon_device; /* balloon device descriptor */
- struct address_space *mapping; /* balloon special page->mapping */
unsigned long isolated_pages; /* # of isolated pages for migration */
spinlock_t pages_lock; /* Protection to pages list */
struct list_head pages; /* Pages enqueued & handled to Host */
+ int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
+ struct page *page, enum migrate_mode mode);
};
-extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
-extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
-extern struct balloon_dev_info *balloon_devinfo_alloc(
- void *balloon_dev_descriptor);
-
-static inline void balloon_devinfo_free(struct balloon_dev_info *b_dev_info)
+static inline void balloon_devinfo_init(struct balloon_dev_info *b_dev_info)
{
- kfree(b_dev_info);
+ b_dev_info->isolated_pages = 0;
+ spin_lock_init(&b_dev_info->pages_lock);
+ INIT_LIST_HEAD(&b_dev_info->pages);
+ b_dev_info->migratepage = NULL;
}
-/*
- * balloon_page_free - release a balloon page back to the page free lists
- * @page: ballooned page to be set free
- *
- * This function must be used to properly set free an isolated/dequeued balloon
- * page at the end of a sucessful page migration, or at the balloon driver's
- * page release procedure.
- */
-static inline void balloon_page_free(struct page *page)
-{
- /*
- * Balloon pages always get an extra refcount before being isolated
- * and before being dequeued to help on sorting out fortuite colisions
- * between a thread attempting to isolate and another thread attempting
- * to release the very same balloon page.
- *
- * Before we handle the page back to Buddy, lets drop its extra refcnt.
- */
- put_page(page);
- __free_page(page);
-}
+extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
+extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
/*
- * balloon_page_insert - insert a page into the balloon's page list and make
- * the page->mapping assignment accordingly.
- * @page : page to be assigned as a 'balloon page'
- * @mapping : allocated special 'balloon_mapping'
- * @head : balloon's device page list head
+ * balloon_page_insert - insert a page into the balloon's page list,
+ * mark and account it accordingly.
+ * @b_dev_info : pinter to ballon device
+ * @page : page to be assigned as a 'balloon page'
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
* pages list is held before inserting a page into the balloon device.
*/
-static inline void balloon_page_insert(struct page *page,
- struct address_space *mapping,
- struct list_head *head)
+static inline void
+balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
inc_zone_page_state(page, NR_BALLOON_PAGES);
- page->mapping = mapping;
- list_add(&page->lru, head);
+ set_page_private(page, (unsigned long)b_dev_info);
+ list_add(&page->lru, &b_dev_info->pages);
}
/*
* balloon_page_delete - delete a page from balloon's page list and clear
- * the page->mapping assignement accordingly.
+ * the ballon page mark accordingly.
* @page : page to be released from balloon's page list
+ * @isolated: already isolated, do not delete from list
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
* pages list is held before deleting a page from the balloon device.
*/
-static inline void balloon_page_delete(struct page *page)
+static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
dec_zone_page_state(page, NR_BALLOON_PAGES);
- page->mapping = NULL;
- list_del(&page->lru);
-}
-
-#endif /* CONFIG_MEMORY_BALLOON */
-
-#ifdef CONFIG_BALLOON_COMPACTION
-
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
-extern struct address_space
-*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops);
-
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- kfree(balloon_mapping);
+ set_page_private(page, 0);
+ if (!isolated)
+ list_del(&page->lru);
}
/*
@@ -154,35 +116,25 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
*/
static inline struct balloon_dev_info *balloon_page_device(struct page *page)
{
- struct address_space *mapping = page->mapping;
- if (likely(mapping))
- return mapping->private_data;
-
- return NULL;
+ return (struct balloon_dev_info *)page_private(page);
}
-static inline gfp_t balloon_mapping_gfp_mask(void)
-{
- return GFP_HIGHUSER_MOVABLE;
-}
+#endif /* CONFIG_MEMORY_BALLOON */
-static inline bool balloon_compaction_check(void)
-{
- return true;
-}
+#ifdef CONFIG_BALLOON_COMPACTION
+extern bool balloon_page_isolate(struct page *page);
+extern void balloon_page_putback(struct page *page);
-#else /* !CONFIG_BALLOON_COMPACTION */
+int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
+ unsigned long private, struct page *page,
+ int force, enum migrate_mode mode);
-static inline void *balloon_mapping_alloc(void *balloon_device,
- const struct address_space_operations *a_ops)
+static inline gfp_t balloon_mapping_gfp_mask(void)
{
- return ERR_PTR(-EOPNOTSUPP);
+ return GFP_HIGHUSER_MOVABLE;
}
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- return;
-}
+#else /* !CONFIG_BALLOON_COMPACTION */
static inline bool balloon_page_isolate(struct page *page)
{
@@ -194,10 +146,11 @@ static inline void balloon_page_putback(struct page *page)
return;
}
-static inline int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode)
+static inline int balloon_page_migrate(new_page_t get_new_page,
+ free_page_t put_new_page, unsigned long private,
+ struct page *page, int force, enum migrate_mode mode)
{
- return 0;
+ return -EAGAIN;
}
static inline gfp_t balloon_mapping_gfp_mask(void)
@@ -205,9 +158,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
return GFP_HIGHUSER;
}
-static inline bool balloon_compaction_check(void)
-{
- return false;
-}
#endif /* CONFIG_BALLOON_COMPACTION */
#endif /* _LINUX_BALLOON_COMPACTION_H */
diff --git a/include/linux/migrate.h b/include/linux/migrate.h
index 0a4604a..cf90776 100644
--- a/include/linux/migrate.h
+++ b/include/linux/migrate.h
@@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);
* Return values from addresss_space_operations.migratepage():
* - negative errno on page migration failure;
* - zero on page migration success;
- *
- * The balloon page migration introduces this special case where a 'distinct'
- * return code is used to flag a successful page migration to unmap_and_move().
- * This approach is necessary because page migration can race against balloon
- * deflation procedure, and for such case we could introduce a nasty page leak
- * if a successfully migrated balloon page gets released concurrently with
- * migration's unmap_and_move() wrap-up steps.
*/
#define MIGRATEPAGE_SUCCESS 0
-#define MIGRATEPAGE_BALLOON_SUCCESS 1 /* special ret code for balloon page
- * sucessful migration case.
- */
+
enum migrate_reason {
MR_COMPACTION,
MR_MEMORY_FAILURE,
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 5ba1813..210b46b 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -24,8 +24,7 @@ enum mapping_flags {
AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
- AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
- AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */
+ AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
};
static inline void mapping_set_error(struct address_space *mapping, int error)
@@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)
return !!mapping;
}
-static inline void mapping_set_balloon(struct address_space *mapping)
-{
- set_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
-static inline void mapping_clear_balloon(struct address_space *mapping)
-{
- clear_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
-static inline int mapping_balloon(struct address_space *mapping)
-{
- return mapping && test_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
static inline void mapping_set_exiting(struct address_space *mapping)
{
set_bit(AS_EXITING, &mapping->flags);
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index a942081..3c8cb7a 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -11,32 +11,6 @@
#include <linux/balloon_compaction.h>
/*
- * balloon_devinfo_alloc - allocates a balloon device information descriptor.
- * @balloon_dev_descriptor: pointer to reference the balloon device which
- * this struct balloon_dev_info will be servicing.
- *
- * Driver must call it to properly allocate and initialize an instance of
- * struct balloon_dev_info which will be used to reference a balloon device
- * as well as to keep track of the balloon device page list.
- */
-struct balloon_dev_info *balloon_devinfo_alloc(void *balloon_dev_descriptor)
-{
- struct balloon_dev_info *b_dev_info;
- b_dev_info = kmalloc(sizeof(*b_dev_info), GFP_KERNEL);
- if (!b_dev_info)
- return ERR_PTR(-ENOMEM);
-
- b_dev_info->balloon_device = balloon_dev_descriptor;
- b_dev_info->mapping = NULL;
- b_dev_info->isolated_pages = 0;
- spin_lock_init(&b_dev_info->pages_lock);
- INIT_LIST_HEAD(&b_dev_info->pages);
-
- return b_dev_info;
-}
-EXPORT_SYMBOL_GPL(balloon_devinfo_alloc);
-
-/*
* balloon_page_enqueue - allocates a new page and inserts it into the balloon
* page list.
* @b_dev_info: balloon device decriptor where we will insert a new page to
@@ -61,7 +35,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)
*/
BUG_ON(!trylock_page(page));
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- balloon_page_insert(page, b_dev_info->mapping, &b_dev_info->pages);
+ balloon_page_insert(b_dev_info, page);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -81,12 +55,10 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
*/
struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
{
- struct page *page, *tmp;
+ struct page *page;
unsigned long flags;
- bool dequeued_page;
- dequeued_page = false;
- list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) {
+ list_for_each_entry(page, &b_dev_info->pages, lru) {
/*
* Block others from accessing the 'page' while we get around
* establishing additional references and preparing the 'page'
@@ -94,98 +66,32 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
*/
if (trylock_page(page)) {
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- /*
- * Raise the page refcount here to prevent any wrong
- * attempt to isolate this page, in case of coliding
- * with balloon_page_isolate() just after we release
- * the page lock.
- *
- * balloon_page_free() will take care of dropping
- * this extra refcount later.
- */
- get_page(page);
- balloon_page_delete(page);
+ balloon_page_delete(page, false);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
- dequeued_page = true;
- break;
+ return page;
}
}
- if (!dequeued_page) {
- /*
- * If we are unable to dequeue a balloon page because the page
- * list is empty and there is no isolated pages, then something
- * went out of track and some balloon pages are lost.
- * BUG() here, otherwise the balloon driver may get stuck into
- * an infinite loop while attempting to release all its pages.
- */
- spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- if (unlikely(list_empty(&b_dev_info->pages) &&
- !b_dev_info->isolated_pages))
- BUG();
- spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
- page = NULL;
- }
- return page;
+ /*
+ * If we are unable to dequeue a balloon page because the page
+ * list is empty and there is no isolated pages, then something
+ * went out of track and some balloon pages are lost.
+ * BUG() here, otherwise the balloon driver may get stuck into
+ * an infinite loop while attempting to release all its pages.
+ */
+ spin_lock_irqsave(&b_dev_info->pages_lock, flags);
+ BUG_ON(list_empty(&b_dev_info->pages) && !b_dev_info->isolated_pages);
+ spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
+ return NULL;
}
EXPORT_SYMBOL_GPL(balloon_page_dequeue);
#ifdef CONFIG_BALLOON_COMPACTION
-/*
- * balloon_mapping_alloc - allocates a special ->mapping for ballooned pages.
- * @b_dev_info: holds the balloon device information descriptor.
- * @a_ops: balloon_mapping address_space_operations descriptor.
- *
- * Driver must call it to properly allocate and initialize an instance of
- * struct address_space which will be used as the special page->mapping for
- * balloon device enlisted page instances.
- */
-struct address_space *balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops)
-{
- struct address_space *mapping;
-
- mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
- if (!mapping)
- return ERR_PTR(-ENOMEM);
-
- /*
- * Give a clean 'zeroed' status to all elements of this special
- * balloon page->mapping struct address_space instance.
- */
- address_space_init_once(mapping);
-
- /*
- * Set mapping->flags appropriately, to allow balloon pages
- * ->mapping identification.
- */
- mapping_set_balloon(mapping);
- mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
-
- /* balloon's page->mapping->a_ops callback descriptor */
- mapping->a_ops = a_ops;
-
- /*
- * Establish a pointer reference back to the balloon device descriptor
- * this particular page->mapping will be servicing.
- * This is used by compaction / migration procedures to identify and
- * access the balloon device pageset while isolating / migrating pages.
- *
- * As some balloon drivers can register multiple balloon devices
- * for a single guest, this also helps compaction / migration to
- * properly deal with multiple balloon pagesets, when required.
- */
- mapping->private_data = b_dev_info;
- b_dev_info->mapping = mapping;
-
- return mapping;
-}
-EXPORT_SYMBOL_GPL(balloon_mapping_alloc);
static inline void __isolate_balloon_page(struct page *page)
{
- struct balloon_dev_info *b_dev_info = page->mapping->private_data;
+ struct balloon_dev_info *b_dev_info = balloon_page_device(page);
unsigned long flags;
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
list_del(&page->lru);
@@ -195,7 +101,7 @@ static inline void __isolate_balloon_page(struct page *page)
static inline void __putback_balloon_page(struct page *page)
{
- struct balloon_dev_info *b_dev_info = page->mapping->private_data;
+ struct balloon_dev_info *b_dev_info = balloon_page_device(page);
unsigned long flags;
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
list_add(&page->lru, &b_dev_info->pages);
@@ -203,12 +109,6 @@ static inline void __putback_balloon_page(struct page *page)
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
}
-static inline int __migrate_balloon_page(struct address_space *mapping,
- struct page *newpage, struct page *page, enum migrate_mode mode)
-{
- return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
-}
-
/* __isolate_lru_page() counterpart for a ballooned page */
bool balloon_page_isolate(struct page *page)
{
@@ -251,6 +151,57 @@ bool balloon_page_isolate(struct page *page)
return false;
}
+int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
+ unsigned long private, struct page *page,
+ int force, enum migrate_mode mode)
+{
+ struct balloon_dev_info *balloon = balloon_page_device(page);
+ struct page *newpage;
+ int *result = NULL;
+ int rc = -EAGAIN;
+
+ if (!balloon || !balloon->migratepage)
+ return -EAGAIN;
+
+ newpage = get_new_page(page, private, &result);
+ if (!newpage)
+ return -ENOMEM;
+
+ if (!trylock_page(newpage))
+ BUG();
+
+ if (!trylock_page(page)) {
+ if (!force || mode != MIGRATE_SYNC)
+ goto out;
+ lock_page(page);
+ }
+
+ rc = balloon->migratepage(balloon, newpage, page, mode);
+
+ unlock_page(page);
+out:
+ unlock_page(newpage);
+
+ if (rc != -EAGAIN) {
+ dec_zone_page_state(page, NR_ISOLATED_FILE);
+ list_del(&page->lru);
+ put_page(page);
+ }
+
+ if (rc != MIGRATEPAGE_SUCCESS && put_new_page)
+ put_new_page(newpage, private);
+ else
+ put_page(newpage);
+
+ if (result) {
+ if (rc)
+ *result = rc;
+ else
+ *result = page_to_nid(newpage);
+ }
+ return rc;
+}
+
/* putback_lru_page() counterpart for a ballooned page */
void balloon_page_putback(struct page *page)
{
@@ -271,31 +222,4 @@ void balloon_page_putback(struct page *page)
unlock_page(page);
}
-/* move_to_new_page() counterpart for a ballooned page */
-int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode)
-{
- struct address_space *mapping;
- int rc = -EAGAIN;
-
- /*
- * Block others from accessing the 'newpage' when we get around to
- * establishing additional references. We should be the only one
- * holding a reference to the 'newpage' at this point.
- */
- BUG_ON(!trylock_page(newpage));
-
- if (WARN_ON(!PageBalloon(page))) {
- dump_page(page, "not movable balloon page");
- unlock_page(newpage);
- return rc;
- }
-
- mapping = page->mapping;
- if (mapping)
- rc = __migrate_balloon_page(mapping, newpage, page, mode);
-
- unlock_page(newpage);
- return rc;
-}
#endif /* CONFIG_BALLOON_COMPACTION */
diff --git a/mm/migrate.c b/mm/migrate.c
index a4939b1..e6d0d2d 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(PageBalloon(page))) {
- /*
- * A ballooned page does not need any special attention from
- * physical to virtual reverse mapping procedures.
- * Skip any attempt to unmap PTEs or to remap swap cache,
- * in order to avoid burning cycles at rmap level, and perform
- * the page migration right away (proteced by page lock).
- */
- rc = balloon_page_migrate(newpage, page, mode);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
@@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
rc = __unmap_and_move(page, newpage, force, mode);
-#ifdef CONFIG_MEMORY_BALLOON
- if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
- /*
- * A ballooned page has been migrated already.
- * Now, it's the time to wrap-up counters,
- * handle the page back to Buddy and return.
- */
- dec_zone_page_state(page, NR_ISOLATED_ANON +
- page_is_file_cache(page));
- balloon_page_free(page);
- return MIGRATEPAGE_SUCCESS;
- }
-#endif
out:
if (rc != -EAGAIN) {
/*
@@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
rc = unmap_and_move_huge_page(get_new_page,
put_new_page, private, page,
pass > 2, mode);
+ else if (PageBalloon(page))
+ rc = balloon_page_migrate(get_new_page,
+ put_new_page, private,
+ page, pass > 2, mode);
else
rc = unmap_and_move(get_new_page, put_new_page,
private, page, pass > 2, mode);
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH v2 6/6] mm/balloon_compaction: general cleanup
@ 2014-08-30 16:41 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-08-30 16:41 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
* move special branch for balloon migraion into migrate_pages
* remove special mapping for balloon and its flag AS_BALLOON_MAP
* embed struct balloon_dev_info into struct virtio_balloon
* cleanup balloon_page_dequeue, kill balloon_page_free
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
---
drivers/virtio/virtio_balloon.c | 77 ++++---------
include/linux/balloon_compaction.h | 127 ++++++---------------
include/linux/migrate.h | 11 --
include/linux/pagemap.h | 18 ---
mm/balloon_compaction.c | 214 ++++++++++++------------------------
mm/migrate.c | 29 +----
6 files changed, 134 insertions(+), 342 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 25ebe8e..c84d6a8 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -59,7 +59,7 @@ struct virtio_balloon
* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
* to num_pages above.
*/
- struct balloon_dev_info *vb_dev_info;
+ struct balloon_dev_info vb_dev_info;
/* Synchronize access/update to this struct virtio_balloon elements */
struct mutex balloon_lock;
@@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)
static void fill_balloon(struct virtio_balloon *vb, size_t num)
{
- struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
+ struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
/* Find pfns pointing at start of each page, get pages and free them. */
for (i = 0; i < num; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
struct page *page = balloon_pfn_to_page(pfns[i]);
- balloon_page_free(page);
adjust_managed_page_count(page, 1);
+ put_page(page);
}
}
static void leak_balloon(struct virtio_balloon *vb, size_t num)
{
struct page *page;
- struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
+ struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)
return 0;
}
-static const struct address_space_operations virtio_balloon_aops;
#ifdef CONFIG_BALLOON_COMPACTION
/*
* virtballoon_migratepage - perform the balloon page migration on behalf of
* a compation thread. (called under page lock)
- * @mapping: the page->mapping which will be assigned to the new migrated page.
+ * @vb_dev_info: the balloon device
* @newpage: page that will replace the isolated page after migration finishes.
* @page : the isolated (old) page that is about to be migrated to newpage.
* @mode : compaction mode -- not used for balloon page migration.
@@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;
* This function preforms the balloon page migration task.
* Called through balloon_mapping->a_ops->migratepage
*/
-static int virtballoon_migratepage(struct address_space *mapping,
+static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
struct page *newpage, struct page *page, enum migrate_mode mode)
{
- struct balloon_dev_info *vb_dev_info = balloon_page_device(page);
- struct virtio_balloon *vb;
+ struct virtio_balloon *vb = container_of(vb_dev_info,
+ struct virtio_balloon, vb_dev_info);
unsigned long flags;
- BUG_ON(!vb_dev_info);
-
- vb = vb_dev_info->balloon_device;
-
/*
* In order to avoid lock contention while migrating pages concurrently
* to leak_balloon() or fill_balloon() we just give up the balloon_lock
@@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,
if (!mutex_trylock(&vb->balloon_lock))
return -EAGAIN;
+ get_page(newpage); /* balloon reference */
+
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
- balloon_page_insert(newpage, mapping, &vb_dev_info->pages);
+ balloon_page_insert(vb_dev_info, newpage);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, newpage);
tell_host(vb, vb->inflate_vq);
- /*
- * balloon's page migration 2nd step -- deflate "page"
- *
- * It's safe to delete page->lru here because this page is at
- * an isolated migration list, and this step is expected to happen here
- */
- balloon_page_delete(page);
+ /* balloon's page migration 2nd step -- deflate "page" */
+ balloon_page_delete(page, true);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, page);
tell_host(vb, vb->deflate_vq);
mutex_unlock(&vb->balloon_lock);
- return MIGRATEPAGE_BALLOON_SUCCESS;
-}
+ put_page(page); /* balloon reference */
-/* define the balloon_mapping->a_ops callback to allow balloon page migration */
-static const struct address_space_operations virtio_balloon_aops = {
- .migratepage = virtballoon_migratepage,
-};
+ return MIGRATEPAGE_SUCCESS;
+}
#endif /* CONFIG_BALLOON_COMPACTION */
static int virtballoon_probe(struct virtio_device *vdev)
{
struct virtio_balloon *vb;
- struct address_space *vb_mapping;
- struct balloon_dev_info *vb_devinfo;
int err;
vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
@@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev)
vb->vdev = vdev;
vb->need_stats_update = 0;
- vb_devinfo = balloon_devinfo_alloc(vb);
- if (IS_ERR(vb_devinfo)) {
- err = PTR_ERR(vb_devinfo);
- goto out_free_vb;
- }
-
- vb_mapping = balloon_mapping_alloc(vb_devinfo,
- (balloon_compaction_check()) ?
- &virtio_balloon_aops : NULL);
- if (IS_ERR(vb_mapping)) {
- /*
- * IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP
- * This means !CONFIG_BALLOON_COMPACTION, otherwise we get off.
- */
- err = PTR_ERR(vb_mapping);
- if (err != -EOPNOTSUPP)
- goto out_free_vb_devinfo;
- }
-
- vb->vb_dev_info = vb_devinfo;
+ balloon_devinfo_init(&vb->vb_dev_info);
+#ifdef CONFIG_BALLOON_COMPACTION
+ vb->vb_dev_info.migratepage = virtballoon_migratepage;
+#endif
err = init_vqs(vb);
if (err)
- goto out_free_vb_mapping;
+ goto out_free_vb;
vb->thread = kthread_run(balloon, vb, "vballoon");
if (IS_ERR(vb->thread)) {
@@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
out_del_vqs:
vdev->config->del_vqs(vdev);
-out_free_vb_mapping:
- balloon_mapping_free(vb_mapping);
-out_free_vb_devinfo:
- balloon_devinfo_free(vb_devinfo);
out_free_vb:
kfree(vb);
out:
@@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)
kthread_stop(vb->thread);
remove_common(vb);
- balloon_mapping_free(vb->vb_dev_info->mapping);
- balloon_devinfo_free(vb->vb_dev_info);
kfree(vb);
}
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index 09f8c5a..ad112fcc6 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -56,96 +56,58 @@
* balloon driver as a page book-keeper for its registered balloon devices.
*/
struct balloon_dev_info {
- void *balloon_device; /* balloon device descriptor */
- struct address_space *mapping; /* balloon special page->mapping */
unsigned long isolated_pages; /* # of isolated pages for migration */
spinlock_t pages_lock; /* Protection to pages list */
struct list_head pages; /* Pages enqueued & handled to Host */
+ int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
+ struct page *page, enum migrate_mode mode);
};
-extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
-extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
-extern struct balloon_dev_info *balloon_devinfo_alloc(
- void *balloon_dev_descriptor);
-
-static inline void balloon_devinfo_free(struct balloon_dev_info *b_dev_info)
+static inline void balloon_devinfo_init(struct balloon_dev_info *b_dev_info)
{
- kfree(b_dev_info);
+ b_dev_info->isolated_pages = 0;
+ spin_lock_init(&b_dev_info->pages_lock);
+ INIT_LIST_HEAD(&b_dev_info->pages);
+ b_dev_info->migratepage = NULL;
}
-/*
- * balloon_page_free - release a balloon page back to the page free lists
- * @page: ballooned page to be set free
- *
- * This function must be used to properly set free an isolated/dequeued balloon
- * page at the end of a sucessful page migration, or at the balloon driver's
- * page release procedure.
- */
-static inline void balloon_page_free(struct page *page)
-{
- /*
- * Balloon pages always get an extra refcount before being isolated
- * and before being dequeued to help on sorting out fortuite colisions
- * between a thread attempting to isolate and another thread attempting
- * to release the very same balloon page.
- *
- * Before we handle the page back to Buddy, lets drop its extra refcnt.
- */
- put_page(page);
- __free_page(page);
-}
+extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
+extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
/*
- * balloon_page_insert - insert a page into the balloon's page list and make
- * the page->mapping assignment accordingly.
- * @page : page to be assigned as a 'balloon page'
- * @mapping : allocated special 'balloon_mapping'
- * @head : balloon's device page list head
+ * balloon_page_insert - insert a page into the balloon's page list,
+ * mark and account it accordingly.
+ * @b_dev_info : pinter to ballon device
+ * @page : page to be assigned as a 'balloon page'
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
* pages list is held before inserting a page into the balloon device.
*/
-static inline void balloon_page_insert(struct page *page,
- struct address_space *mapping,
- struct list_head *head)
+static inline void
+balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
inc_zone_page_state(page, NR_BALLOON_PAGES);
- page->mapping = mapping;
- list_add(&page->lru, head);
+ set_page_private(page, (unsigned long)b_dev_info);
+ list_add(&page->lru, &b_dev_info->pages);
}
/*
* balloon_page_delete - delete a page from balloon's page list and clear
- * the page->mapping assignement accordingly.
+ * the ballon page mark accordingly.
* @page : page to be released from balloon's page list
+ * @isolated: already isolated, do not delete from list
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
* pages list is held before deleting a page from the balloon device.
*/
-static inline void balloon_page_delete(struct page *page)
+static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
dec_zone_page_state(page, NR_BALLOON_PAGES);
- page->mapping = NULL;
- list_del(&page->lru);
-}
-
-#endif /* CONFIG_MEMORY_BALLOON */
-
-#ifdef CONFIG_BALLOON_COMPACTION
-
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
-extern struct address_space
-*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops);
-
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- kfree(balloon_mapping);
+ set_page_private(page, 0);
+ if (!isolated)
+ list_del(&page->lru);
}
/*
@@ -154,35 +116,25 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
*/
static inline struct balloon_dev_info *balloon_page_device(struct page *page)
{
- struct address_space *mapping = page->mapping;
- if (likely(mapping))
- return mapping->private_data;
-
- return NULL;
+ return (struct balloon_dev_info *)page_private(page);
}
-static inline gfp_t balloon_mapping_gfp_mask(void)
-{
- return GFP_HIGHUSER_MOVABLE;
-}
+#endif /* CONFIG_MEMORY_BALLOON */
-static inline bool balloon_compaction_check(void)
-{
- return true;
-}
+#ifdef CONFIG_BALLOON_COMPACTION
+extern bool balloon_page_isolate(struct page *page);
+extern void balloon_page_putback(struct page *page);
-#else /* !CONFIG_BALLOON_COMPACTION */
+int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
+ unsigned long private, struct page *page,
+ int force, enum migrate_mode mode);
-static inline void *balloon_mapping_alloc(void *balloon_device,
- const struct address_space_operations *a_ops)
+static inline gfp_t balloon_mapping_gfp_mask(void)
{
- return ERR_PTR(-EOPNOTSUPP);
+ return GFP_HIGHUSER_MOVABLE;
}
-static inline void balloon_mapping_free(struct address_space *balloon_mapping)
-{
- return;
-}
+#else /* !CONFIG_BALLOON_COMPACTION */
static inline bool balloon_page_isolate(struct page *page)
{
@@ -194,10 +146,11 @@ static inline void balloon_page_putback(struct page *page)
return;
}
-static inline int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode)
+static inline int balloon_page_migrate(new_page_t get_new_page,
+ free_page_t put_new_page, unsigned long private,
+ struct page *page, int force, enum migrate_mode mode)
{
- return 0;
+ return -EAGAIN;
}
static inline gfp_t balloon_mapping_gfp_mask(void)
@@ -205,9 +158,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
return GFP_HIGHUSER;
}
-static inline bool balloon_compaction_check(void)
-{
- return false;
-}
#endif /* CONFIG_BALLOON_COMPACTION */
#endif /* _LINUX_BALLOON_COMPACTION_H */
diff --git a/include/linux/migrate.h b/include/linux/migrate.h
index 0a4604a..cf90776 100644
--- a/include/linux/migrate.h
+++ b/include/linux/migrate.h
@@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);
* Return values from addresss_space_operations.migratepage():
* - negative errno on page migration failure;
* - zero on page migration success;
- *
- * The balloon page migration introduces this special case where a 'distinct'
- * return code is used to flag a successful page migration to unmap_and_move().
- * This approach is necessary because page migration can race against balloon
- * deflation procedure, and for such case we could introduce a nasty page leak
- * if a successfully migrated balloon page gets released concurrently with
- * migration's unmap_and_move() wrap-up steps.
*/
#define MIGRATEPAGE_SUCCESS 0
-#define MIGRATEPAGE_BALLOON_SUCCESS 1 /* special ret code for balloon page
- * sucessful migration case.
- */
+
enum migrate_reason {
MR_COMPACTION,
MR_MEMORY_FAILURE,
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 5ba1813..210b46b 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -24,8 +24,7 @@ enum mapping_flags {
AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
- AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
- AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */
+ AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
};
static inline void mapping_set_error(struct address_space *mapping, int error)
@@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)
return !!mapping;
}
-static inline void mapping_set_balloon(struct address_space *mapping)
-{
- set_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
-static inline void mapping_clear_balloon(struct address_space *mapping)
-{
- clear_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
-static inline int mapping_balloon(struct address_space *mapping)
-{
- return mapping && test_bit(AS_BALLOON_MAP, &mapping->flags);
-}
-
static inline void mapping_set_exiting(struct address_space *mapping)
{
set_bit(AS_EXITING, &mapping->flags);
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index a942081..3c8cb7a 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -11,32 +11,6 @@
#include <linux/balloon_compaction.h>
/*
- * balloon_devinfo_alloc - allocates a balloon device information descriptor.
- * @balloon_dev_descriptor: pointer to reference the balloon device which
- * this struct balloon_dev_info will be servicing.
- *
- * Driver must call it to properly allocate and initialize an instance of
- * struct balloon_dev_info which will be used to reference a balloon device
- * as well as to keep track of the balloon device page list.
- */
-struct balloon_dev_info *balloon_devinfo_alloc(void *balloon_dev_descriptor)
-{
- struct balloon_dev_info *b_dev_info;
- b_dev_info = kmalloc(sizeof(*b_dev_info), GFP_KERNEL);
- if (!b_dev_info)
- return ERR_PTR(-ENOMEM);
-
- b_dev_info->balloon_device = balloon_dev_descriptor;
- b_dev_info->mapping = NULL;
- b_dev_info->isolated_pages = 0;
- spin_lock_init(&b_dev_info->pages_lock);
- INIT_LIST_HEAD(&b_dev_info->pages);
-
- return b_dev_info;
-}
-EXPORT_SYMBOL_GPL(balloon_devinfo_alloc);
-
-/*
* balloon_page_enqueue - allocates a new page and inserts it into the balloon
* page list.
* @b_dev_info: balloon device decriptor where we will insert a new page to
@@ -61,7 +35,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)
*/
BUG_ON(!trylock_page(page));
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- balloon_page_insert(page, b_dev_info->mapping, &b_dev_info->pages);
+ balloon_page_insert(b_dev_info, page);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -81,12 +55,10 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
*/
struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
{
- struct page *page, *tmp;
+ struct page *page;
unsigned long flags;
- bool dequeued_page;
- dequeued_page = false;
- list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) {
+ list_for_each_entry(page, &b_dev_info->pages, lru) {
/*
* Block others from accessing the 'page' while we get around
* establishing additional references and preparing the 'page'
@@ -94,98 +66,32 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
*/
if (trylock_page(page)) {
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- /*
- * Raise the page refcount here to prevent any wrong
- * attempt to isolate this page, in case of coliding
- * with balloon_page_isolate() just after we release
- * the page lock.
- *
- * balloon_page_free() will take care of dropping
- * this extra refcount later.
- */
- get_page(page);
- balloon_page_delete(page);
+ balloon_page_delete(page, false);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
- dequeued_page = true;
- break;
+ return page;
}
}
- if (!dequeued_page) {
- /*
- * If we are unable to dequeue a balloon page because the page
- * list is empty and there is no isolated pages, then something
- * went out of track and some balloon pages are lost.
- * BUG() here, otherwise the balloon driver may get stuck into
- * an infinite loop while attempting to release all its pages.
- */
- spin_lock_irqsave(&b_dev_info->pages_lock, flags);
- if (unlikely(list_empty(&b_dev_info->pages) &&
- !b_dev_info->isolated_pages))
- BUG();
- spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
- page = NULL;
- }
- return page;
+ /*
+ * If we are unable to dequeue a balloon page because the page
+ * list is empty and there is no isolated pages, then something
+ * went out of track and some balloon pages are lost.
+ * BUG() here, otherwise the balloon driver may get stuck into
+ * an infinite loop while attempting to release all its pages.
+ */
+ spin_lock_irqsave(&b_dev_info->pages_lock, flags);
+ BUG_ON(list_empty(&b_dev_info->pages) && !b_dev_info->isolated_pages);
+ spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
+ return NULL;
}
EXPORT_SYMBOL_GPL(balloon_page_dequeue);
#ifdef CONFIG_BALLOON_COMPACTION
-/*
- * balloon_mapping_alloc - allocates a special ->mapping for ballooned pages.
- * @b_dev_info: holds the balloon device information descriptor.
- * @a_ops: balloon_mapping address_space_operations descriptor.
- *
- * Driver must call it to properly allocate and initialize an instance of
- * struct address_space which will be used as the special page->mapping for
- * balloon device enlisted page instances.
- */
-struct address_space *balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
- const struct address_space_operations *a_ops)
-{
- struct address_space *mapping;
-
- mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
- if (!mapping)
- return ERR_PTR(-ENOMEM);
-
- /*
- * Give a clean 'zeroed' status to all elements of this special
- * balloon page->mapping struct address_space instance.
- */
- address_space_init_once(mapping);
-
- /*
- * Set mapping->flags appropriately, to allow balloon pages
- * ->mapping identification.
- */
- mapping_set_balloon(mapping);
- mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
-
- /* balloon's page->mapping->a_ops callback descriptor */
- mapping->a_ops = a_ops;
-
- /*
- * Establish a pointer reference back to the balloon device descriptor
- * this particular page->mapping will be servicing.
- * This is used by compaction / migration procedures to identify and
- * access the balloon device pageset while isolating / migrating pages.
- *
- * As some balloon drivers can register multiple balloon devices
- * for a single guest, this also helps compaction / migration to
- * properly deal with multiple balloon pagesets, when required.
- */
- mapping->private_data = b_dev_info;
- b_dev_info->mapping = mapping;
-
- return mapping;
-}
-EXPORT_SYMBOL_GPL(balloon_mapping_alloc);
static inline void __isolate_balloon_page(struct page *page)
{
- struct balloon_dev_info *b_dev_info = page->mapping->private_data;
+ struct balloon_dev_info *b_dev_info = balloon_page_device(page);
unsigned long flags;
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
list_del(&page->lru);
@@ -195,7 +101,7 @@ static inline void __isolate_balloon_page(struct page *page)
static inline void __putback_balloon_page(struct page *page)
{
- struct balloon_dev_info *b_dev_info = page->mapping->private_data;
+ struct balloon_dev_info *b_dev_info = balloon_page_device(page);
unsigned long flags;
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
list_add(&page->lru, &b_dev_info->pages);
@@ -203,12 +109,6 @@ static inline void __putback_balloon_page(struct page *page)
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
}
-static inline int __migrate_balloon_page(struct address_space *mapping,
- struct page *newpage, struct page *page, enum migrate_mode mode)
-{
- return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
-}
-
/* __isolate_lru_page() counterpart for a ballooned page */
bool balloon_page_isolate(struct page *page)
{
@@ -251,6 +151,57 @@ bool balloon_page_isolate(struct page *page)
return false;
}
+int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
+ unsigned long private, struct page *page,
+ int force, enum migrate_mode mode)
+{
+ struct balloon_dev_info *balloon = balloon_page_device(page);
+ struct page *newpage;
+ int *result = NULL;
+ int rc = -EAGAIN;
+
+ if (!balloon || !balloon->migratepage)
+ return -EAGAIN;
+
+ newpage = get_new_page(page, private, &result);
+ if (!newpage)
+ return -ENOMEM;
+
+ if (!trylock_page(newpage))
+ BUG();
+
+ if (!trylock_page(page)) {
+ if (!force || mode != MIGRATE_SYNC)
+ goto out;
+ lock_page(page);
+ }
+
+ rc = balloon->migratepage(balloon, newpage, page, mode);
+
+ unlock_page(page);
+out:
+ unlock_page(newpage);
+
+ if (rc != -EAGAIN) {
+ dec_zone_page_state(page, NR_ISOLATED_FILE);
+ list_del(&page->lru);
+ put_page(page);
+ }
+
+ if (rc != MIGRATEPAGE_SUCCESS && put_new_page)
+ put_new_page(newpage, private);
+ else
+ put_page(newpage);
+
+ if (result) {
+ if (rc)
+ *result = rc;
+ else
+ *result = page_to_nid(newpage);
+ }
+ return rc;
+}
+
/* putback_lru_page() counterpart for a ballooned page */
void balloon_page_putback(struct page *page)
{
@@ -271,31 +222,4 @@ void balloon_page_putback(struct page *page)
unlock_page(page);
}
-/* move_to_new_page() counterpart for a ballooned page */
-int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode)
-{
- struct address_space *mapping;
- int rc = -EAGAIN;
-
- /*
- * Block others from accessing the 'newpage' when we get around to
- * establishing additional references. We should be the only one
- * holding a reference to the 'newpage' at this point.
- */
- BUG_ON(!trylock_page(newpage));
-
- if (WARN_ON(!PageBalloon(page))) {
- dump_page(page, "not movable balloon page");
- unlock_page(newpage);
- return rc;
- }
-
- mapping = page->mapping;
- if (mapping)
- rc = __migrate_balloon_page(mapping, newpage, page, mode);
-
- unlock_page(newpage);
- return rc;
-}
#endif /* CONFIG_BALLOON_COMPACTION */
diff --git a/mm/migrate.c b/mm/migrate.c
index a4939b1..e6d0d2d 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
}
}
- if (unlikely(PageBalloon(page))) {
- /*
- * A ballooned page does not need any special attention from
- * physical to virtual reverse mapping procedures.
- * Skip any attempt to unmap PTEs or to remap swap cache,
- * in order to avoid burning cycles at rmap level, and perform
- * the page migration right away (proteced by page lock).
- */
- rc = balloon_page_migrate(newpage, page, mode);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
@@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
rc = __unmap_and_move(page, newpage, force, mode);
-#ifdef CONFIG_MEMORY_BALLOON
- if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
- /*
- * A ballooned page has been migrated already.
- * Now, it's the time to wrap-up counters,
- * handle the page back to Buddy and return.
- */
- dec_zone_page_state(page, NR_ISOLATED_ANON +
- page_is_file_cache(page));
- balloon_page_free(page);
- return MIGRATEPAGE_SUCCESS;
- }
-#endif
out:
if (rc != -EAGAIN) {
/*
@@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
rc = unmap_and_move_huge_page(get_new_page,
put_new_page, private, page,
pass > 2, mode);
+ else if (PageBalloon(page))
+ rc = balloon_page_migrate(get_new_page,
+ put_new_page, private,
+ page, pass > 2, mode);
else
rc = unmap_and_move(get_new_page, put_new_page,
private, page, pass > 2, mode);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 12:29 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:29 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:09PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> Sasha Levin reported KASAN splash inside isolate_migratepages_range().
> Problem is in function __is_movable_balloon_page() which tests AS_BALLOON_MAP
> in page->mapping->flags. This function has no protection against anonymous
> pages. As result it tried to check address space flags in inside anon-vma.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Reported-by: Sasha Levin <sasha.levin@oracle.com>
> Link: http://lkml.kernel.org/p/53E6CEAA.9020105@oracle.com
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> include/linux/balloon_compaction.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 089743a..53d482e 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)
> static inline bool __is_movable_balloon_page(struct page *page)
> {
> struct address_space *mapping = page->mapping;
> - return mapping_balloon(mapping);
> + return !PageAnon(page) && mapping_balloon(mapping);
> }
>
> /*
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages
@ 2014-09-02 12:29 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:29 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:09PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> Sasha Levin reported KASAN splash inside isolate_migratepages_range().
> Problem is in function __is_movable_balloon_page() which tests AS_BALLOON_MAP
> in page->mapping->flags. This function has no protection against anonymous
> pages. As result it tried to check address space flags in inside anon-vma.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Reported-by: Sasha Levin <sasha.levin@oracle.com>
> Link: http://lkml.kernel.org/p/53E6CEAA.9020105@oracle.com
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> include/linux/balloon_compaction.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 089743a..53d482e 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)
> static inline bool __is_movable_balloon_page(struct page *page)
> {
> struct address_space *mapping = page->mapping;
> - return mapping_balloon(mapping);
> + return !PageAnon(page) && mapping_balloon(mapping);
> }
>
> /*
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 12:31 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:31 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:13PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> Proper testing shows yet another problem in balloon migration: it works only
> once for each page. balloon_page_movable() check page flags and page_count.
> In __unmap_and_move page is locked, reference counter is elevated, so
> balloon_page_movable() _always_ fails here. As result in __unmap_and_move()
> migration goes to the normal migration path.
>
> Balloon ->migratepage() is so special, it returns MIGRATEPAGE_BALLOON_SUCCESS
> instead of MIGRATEPAGE_SUCCESS. After that in move_to_new_page() successfully
> migrated page got NULL into its mapping pointer and loses connectivity with
> balloon and ability for further migration.
>
> It's safe to use __is_movable_balloon_page here: page is isolated and pinned.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> include/linux/balloon_compaction.h | 5 +++++
> mm/migrate.c | 2 +-
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 53d482e..284fc1d 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)
> list_del(&page->lru);
> }
>
> +static inline bool __is_movable_balloon_page(struct page *page)
> +{
> + return false;
> +}
> +
> static inline bool balloon_page_movable(struct page *page)
> {
> return false;
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 905b1aa..57c94f9 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(balloon_page_movable(page))) {
> + if (unlikely(__is_movable_balloon_page(page))) {
> /*
> * A ballooned page does not need any special attention from
> * physical to virtual reverse mapping procedures.
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path
@ 2014-09-02 12:31 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:31 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:13PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> Proper testing shows yet another problem in balloon migration: it works only
> once for each page. balloon_page_movable() check page flags and page_count.
> In __unmap_and_move page is locked, reference counter is elevated, so
> balloon_page_movable() _always_ fails here. As result in __unmap_and_move()
> migration goes to the normal migration path.
>
> Balloon ->migratepage() is so special, it returns MIGRATEPAGE_BALLOON_SUCCESS
> instead of MIGRATEPAGE_SUCCESS. After that in move_to_new_page() successfully
> migrated page got NULL into its mapping pointer and loses connectivity with
> balloon and ability for further migration.
>
> It's safe to use __is_movable_balloon_page here: page is isolated and pinned.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> include/linux/balloon_compaction.h | 5 +++++
> mm/migrate.c | 2 +-
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 53d482e..284fc1d 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)
> list_del(&page->lru);
> }
>
> +static inline bool __is_movable_balloon_page(struct page *page)
> +{
> + return false;
> +}
> +
> static inline bool balloon_page_movable(struct page *page)
> {
> return false;
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 905b1aa..57c94f9 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(balloon_page_movable(page))) {
> + if (unlikely(__is_movable_balloon_page(page))) {
> /*
> * A ballooned page does not need any special attention from
> * physical to virtual reverse mapping procedures.
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 12:32 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:32 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:17PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> LRU-lock isn't required for balloon page isolation. This check makes migration
> of some ballooned pages mostly impossible because isolate_migratepages_range()
> drops LRU lock periodically.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> mm/compaction.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 73466e1..ad58f73 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> */
> if (!PageLRU(page)) {
> if (unlikely(balloon_page_movable(page))) {
> - if (locked && balloon_page_isolate(page)) {
> + if (balloon_page_isolate(page)) {
> /* Successfully isolated */
> goto isolate_success;
> }
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock
@ 2014-09-02 12:32 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:32 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:17PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> LRU-lock isn't required for balloon page isolation. This check makes migration
> of some ballooned pages mostly impossible because isolate_migratepages_range()
> drops LRU lock periodically.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> Cc: stable <stable@vger.kernel.org> # v3.8
> ---
> mm/compaction.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 73466e1..ad58f73 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> */
> if (!PageLRU(page)) {
> if (unlikely(balloon_page_movable(page))) {
> - if (locked && balloon_page_isolate(page)) {
> + if (balloon_page_isolate(page)) {
> /* Successfully isolated */
> goto isolate_success;
> }
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 12:53 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:53 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:20PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
> state of page->_mapcount counter. There is no conflict because ballooned pages
> cannot be mapped and cannot be in buddy allocator.
>
> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
> userspace via /proc/kpageflags as KPF_BALLOON.
>
> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
> ballooning driver which wants use this feature.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> Documentation/filesystems/proc.txt | 2 ++
> drivers/base/node.c | 16 ++++++++++------
> fs/proc/meminfo.c | 6 ++++++
> fs/proc/page.c | 3 +++
> include/linux/mm.h | 20 ++++++++++++++++++++
> include/linux/mmzone.h | 3 +++
> include/uapi/linux/kernel-page-flags.h | 1 +
> mm/Kconfig | 5 +++++
> mm/vmstat.c | 8 +++++++-
> tools/vm/page-types.c | 1 +
> 10 files changed, 58 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
> index eb8a10e..154a345 100644
> --- a/Documentation/filesystems/proc.txt
> +++ b/Documentation/filesystems/proc.txt
> @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB
> VmallocUsed: 428 kB
> VmallocChunk: 111088 kB
> AnonHugePages: 49152 kB
> +BalloonPages: 0 kB
>
> MemTotal: Total usable ram (i.e. physical ram minus a few reserved
> bits and the kernel binary code)
> @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new
> Writeback: Memory which is actively being written back to the disk
> AnonPages: Non-file backed pages mapped into userspace page tables
> AnonHugePages: Non-file backed huge pages mapped into userspace page tables
> +BalloonPages: Memory which was ballooned, not included into MemTotal
> Mapped: files which have been mmaped, such as libraries
> Slab: in-kernel data structures cache
> SReclaimable: Part of Slab, that might be reclaimed, such as caches
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index c6d3ae0..59e565c 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> "Node %d AnonHugePages: %8lu kB\n"
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + "Node %d BalloonPages: %8lu kB\n"
> +#endif
> ,
> nid, K(node_page_state(nid, NR_FILE_DIRTY)),
> nid, K(node_page_state(nid, NR_WRITEBACK)),
> @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev,
> nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) +
> node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
> nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)),
> -#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))
> - , nid,
> - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) *
> - HPAGE_PMD_NR));
> -#else
> - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)));
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> + ,nid, K(node_page_state(nid,
> + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)
> +#endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + ,nid, K(node_page_state(nid, NR_BALLOON_PAGES))
> #endif
> + );
> n += hugetlb_report_node_meminfo(nid, buf + n);
> return n;
> }
> diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
> index aa1eee0..f897fbf 100644
> --- a/fs/proc/meminfo.c
> +++ b/fs/proc/meminfo.c
> @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> "AnonHugePages: %8lu kB\n"
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + "BalloonPages: %8lu kB\n"
> +#endif
> ,
> K(i.totalram),
> K(i.freeram),
> @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) *
> HPAGE_PMD_NR)
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + ,K(global_page_state(NR_BALLOON_PAGES))
> +#endif
> );
>
> hugetlb_report_meminfo(m);
> diff --git a/fs/proc/page.c b/fs/proc/page.c
> index e647c55..1e3187d 100644
> --- a/fs/proc/page.c
> +++ b/fs/proc/page.c
> @@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)
> if (PageBuddy(page))
> u |= 1 << KPF_BUDDY;
>
> + if (PageBalloon(page))
> + u |= 1 << KPF_BALLOON;
> +
> u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
>
> u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index da79328..a9a27ea 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)
> atomic_set(&page->_mapcount, -1);
> }
>
> +#define PAGE_BALLOON_MAPCOUNT_VALUE (-256)
> +
> +static inline int PageBalloon(struct page *page)
> +{
> + return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
> + atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
> +}
> +
> +static inline void __SetPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> + atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> +}
> +
> +static inline void __ClearPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(!PageBalloon(page), page);
> + atomic_set(&page->_mapcount, -1);
> +}
> +
> void put_page(struct page *page);
> void put_pages_list(struct list_head *pages);
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 318df70..d88fd01 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -157,6 +157,9 @@ enum zone_stat_item {
> WORKINGSET_NODERECLAIM,
> NR_ANON_TRANSPARENT_HUGEPAGES,
> NR_FREE_CMA_PAGES,
> +#ifdef CONFIG_MEMORY_BALLOON
> + NR_BALLOON_PAGES,
> +#endif
> NR_VM_ZONE_STAT_ITEMS };
>
> /*
> diff --git a/include/uapi/linux/kernel-page-flags.h b/include/uapi/linux/kernel-page-flags.h
> index 5116a0e..2f96d23 100644
> --- a/include/uapi/linux/kernel-page-flags.h
> +++ b/include/uapi/linux/kernel-page-flags.h
> @@ -31,6 +31,7 @@
>
> #define KPF_KSM 21
> #define KPF_THP 22
> +#define KPF_BALLOON 23
>
>
> #endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 886db21..72e0db0 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK
> boolean
>
> #
> +# support for memory ballooning
> +config MEMORY_BALLOON
> + boolean
> +
> +#
> # support for memory balloon compaction
> config BALLOON_COMPACTION
> bool "Allow for balloon memory compaction/migration"
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 84c688d..ff7bd29 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
> TEXT_FOR_HIGHMEM(xx) xx "_movable",
>
> const char * const vmstat_text[] = {
> - /* Zoned VM counters */
> + /* enum zone_stat_item countes */
> "nr_free_pages",
> "nr_alloc_batch",
> "nr_inactive_anon",
> @@ -794,10 +794,16 @@ const char * const vmstat_text[] = {
> "workingset_nodereclaim",
> "nr_anon_transparent_hugepages",
> "nr_free_cma",
> +#ifdef CONFIG_MEMORY_BALLOON
> + "nr_balloon_pages",
> +#endif
> +
> + /* enum writeback_stat_item counters */
> "nr_dirty_threshold",
> "nr_dirty_background_threshold",
>
> #ifdef CONFIG_VM_EVENT_COUNTERS
> + /* enum vm_event_item counters */
> "pgpgin",
> "pgpgout",
> "pswpin",
> diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
> index c4d6d2e..264fbc2 100644
> --- a/tools/vm/page-types.c
> +++ b/tools/vm/page-types.c
> @@ -132,6 +132,7 @@ static const char * const page_flag_names[] = {
> [KPF_NOPAGE] = "n:nopage",
> [KPF_KSM] = "x:ksm",
> [KPF_THP] = "t:thp",
> + [KPF_BALLOON] = "o:balloon",
>
> [KPF_RESERVED] = "r:reserved",
> [KPF_MLOCKED] = "m:mlocked",
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-02 12:53 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:53 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:20PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
> state of page->_mapcount counter. There is no conflict because ballooned pages
> cannot be mapped and cannot be in buddy allocator.
>
> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
> userspace via /proc/kpageflags as KPF_BALLOON.
>
> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
> ballooning driver which wants use this feature.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> Documentation/filesystems/proc.txt | 2 ++
> drivers/base/node.c | 16 ++++++++++------
> fs/proc/meminfo.c | 6 ++++++
> fs/proc/page.c | 3 +++
> include/linux/mm.h | 20 ++++++++++++++++++++
> include/linux/mmzone.h | 3 +++
> include/uapi/linux/kernel-page-flags.h | 1 +
> mm/Kconfig | 5 +++++
> mm/vmstat.c | 8 +++++++-
> tools/vm/page-types.c | 1 +
> 10 files changed, 58 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
> index eb8a10e..154a345 100644
> --- a/Documentation/filesystems/proc.txt
> +++ b/Documentation/filesystems/proc.txt
> @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB
> VmallocUsed: 428 kB
> VmallocChunk: 111088 kB
> AnonHugePages: 49152 kB
> +BalloonPages: 0 kB
>
> MemTotal: Total usable ram (i.e. physical ram minus a few reserved
> bits and the kernel binary code)
> @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new
> Writeback: Memory which is actively being written back to the disk
> AnonPages: Non-file backed pages mapped into userspace page tables
> AnonHugePages: Non-file backed huge pages mapped into userspace page tables
> +BalloonPages: Memory which was ballooned, not included into MemTotal
> Mapped: files which have been mmaped, such as libraries
> Slab: in-kernel data structures cache
> SReclaimable: Part of Slab, that might be reclaimed, such as caches
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index c6d3ae0..59e565c 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> "Node %d AnonHugePages: %8lu kB\n"
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + "Node %d BalloonPages: %8lu kB\n"
> +#endif
> ,
> nid, K(node_page_state(nid, NR_FILE_DIRTY)),
> nid, K(node_page_state(nid, NR_WRITEBACK)),
> @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev,
> nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) +
> node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
> nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)),
> -#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))
> - , nid,
> - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) *
> - HPAGE_PMD_NR));
> -#else
> - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)));
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> + ,nid, K(node_page_state(nid,
> + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)
> +#endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + ,nid, K(node_page_state(nid, NR_BALLOON_PAGES))
> #endif
> + );
> n += hugetlb_report_node_meminfo(nid, buf + n);
> return n;
> }
> diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
> index aa1eee0..f897fbf 100644
> --- a/fs/proc/meminfo.c
> +++ b/fs/proc/meminfo.c
> @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> "AnonHugePages: %8lu kB\n"
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + "BalloonPages: %8lu kB\n"
> +#endif
> ,
> K(i.totalram),
> K(i.freeram),
> @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) *
> HPAGE_PMD_NR)
> #endif
> +#ifdef CONFIG_MEMORY_BALLOON
> + ,K(global_page_state(NR_BALLOON_PAGES))
> +#endif
> );
>
> hugetlb_report_meminfo(m);
> diff --git a/fs/proc/page.c b/fs/proc/page.c
> index e647c55..1e3187d 100644
> --- a/fs/proc/page.c
> +++ b/fs/proc/page.c
> @@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)
> if (PageBuddy(page))
> u |= 1 << KPF_BUDDY;
>
> + if (PageBalloon(page))
> + u |= 1 << KPF_BALLOON;
> +
> u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
>
> u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index da79328..a9a27ea 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)
> atomic_set(&page->_mapcount, -1);
> }
>
> +#define PAGE_BALLOON_MAPCOUNT_VALUE (-256)
> +
> +static inline int PageBalloon(struct page *page)
> +{
> + return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
> + atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
> +}
> +
> +static inline void __SetPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> + atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> +}
> +
> +static inline void __ClearPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(!PageBalloon(page), page);
> + atomic_set(&page->_mapcount, -1);
> +}
> +
> void put_page(struct page *page);
> void put_pages_list(struct list_head *pages);
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 318df70..d88fd01 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -157,6 +157,9 @@ enum zone_stat_item {
> WORKINGSET_NODERECLAIM,
> NR_ANON_TRANSPARENT_HUGEPAGES,
> NR_FREE_CMA_PAGES,
> +#ifdef CONFIG_MEMORY_BALLOON
> + NR_BALLOON_PAGES,
> +#endif
> NR_VM_ZONE_STAT_ITEMS };
>
> /*
> diff --git a/include/uapi/linux/kernel-page-flags.h b/include/uapi/linux/kernel-page-flags.h
> index 5116a0e..2f96d23 100644
> --- a/include/uapi/linux/kernel-page-flags.h
> +++ b/include/uapi/linux/kernel-page-flags.h
> @@ -31,6 +31,7 @@
>
> #define KPF_KSM 21
> #define KPF_THP 22
> +#define KPF_BALLOON 23
>
>
> #endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 886db21..72e0db0 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK
> boolean
>
> #
> +# support for memory ballooning
> +config MEMORY_BALLOON
> + boolean
> +
> +#
> # support for memory balloon compaction
> config BALLOON_COMPACTION
> bool "Allow for balloon memory compaction/migration"
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 84c688d..ff7bd29 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
> TEXT_FOR_HIGHMEM(xx) xx "_movable",
>
> const char * const vmstat_text[] = {
> - /* Zoned VM counters */
> + /* enum zone_stat_item countes */
> "nr_free_pages",
> "nr_alloc_batch",
> "nr_inactive_anon",
> @@ -794,10 +794,16 @@ const char * const vmstat_text[] = {
> "workingset_nodereclaim",
> "nr_anon_transparent_hugepages",
> "nr_free_cma",
> +#ifdef CONFIG_MEMORY_BALLOON
> + "nr_balloon_pages",
> +#endif
> +
> + /* enum writeback_stat_item counters */
> "nr_dirty_threshold",
> "nr_dirty_background_threshold",
>
> #ifdef CONFIG_VM_EVENT_COUNTERS
> + /* enum vm_event_item counters */
> "pgpgin",
> "pgpgout",
> "pswpin",
> diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
> index c4d6d2e..264fbc2 100644
> --- a/tools/vm/page-types.c
> +++ b/tools/vm/page-types.c
> @@ -132,6 +132,7 @@ static const char * const page_flag_names[] = {
> [KPF_NOPAGE] = "n:nopage",
> [KPF_KSM] = "x:ksm",
> [KPF_THP] = "t:thp",
> + [KPF_BALLOON] = "o:balloon",
>
> [KPF_RESERVED] = "r:reserved",
> [KPF_MLOCKED] = "m:mlocked",
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 12:57 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:57 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:23PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
> with PageBalloon which is stored directly in the struct page.
> All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> drivers/virtio/Kconfig | 1
> include/linux/balloon_compaction.h | 135 ++++++------------------------------
> mm/Kconfig | 2 -
> mm/Makefile | 3 +
> mm/balloon_compaction.c | 7 +-
> mm/compaction.c | 9 +-
> mm/migrate.c | 6 +-
> mm/vmscan.c | 2 -
> 8 files changed, 39 insertions(+), 126 deletions(-)
>
> diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
> index c6683f2..00b2286 100644
> --- a/drivers/virtio/Kconfig
> +++ b/drivers/virtio/Kconfig
> @@ -25,6 +25,7 @@ config VIRTIO_PCI
> config VIRTIO_BALLOON
> tristate "Virtio balloon driver"
> depends on VIRTIO
> + select MEMORY_BALLOON
> ---help---
> This driver supports increasing and decreasing the amount
> of memory within a KVM guest.
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 284fc1d..09f8c5a 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -46,6 +46,8 @@
> #include <linux/gfp.h>
> #include <linux/err.h>
>
> +#ifdef CONFIG_MEMORY_BALLOON
> +
> /*
> * Balloon device information descriptor.
> * This struct is used to allow the common balloon compaction interface
> @@ -93,91 +95,6 @@ static inline void balloon_page_free(struct page *page)
> __free_page(page);
> }
>
> -#ifdef CONFIG_BALLOON_COMPACTION
> -extern bool balloon_page_isolate(struct page *page);
> -extern void balloon_page_putback(struct page *page);
> -extern int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode);
> -extern struct address_space
> -*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops);
> -
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - kfree(balloon_mapping);
> -}
> -
> -/*
> - * page_flags_cleared - helper to perform balloon @page ->flags tests.
> - *
> - * As balloon pages are obtained from buddy and we do not play with page->flags
> - * at driver level (exception made when we get the page lock for compaction),
> - * we can safely identify a ballooned page by checking if the
> - * PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also
> - * helps us skip ballooned pages that are locked for compaction or release, thus
> - * mitigating their racy check at balloon_page_movable()
> - */
> -static inline bool page_flags_cleared(struct page *page)
> -{
> - return !(page->flags & PAGE_FLAGS_CHECK_AT_PREP);
> -}
> -
> -/*
> - * __is_movable_balloon_page - helper to perform @page mapping->flags tests
> - */
> -static inline bool __is_movable_balloon_page(struct page *page)
> -{
> - struct address_space *mapping = page->mapping;
> - return !PageAnon(page) && mapping_balloon(mapping);
> -}
> -
> -/*
> - * balloon_page_movable - test page->mapping->flags to identify balloon pages
> - * that can be moved by compaction/migration.
> - *
> - * This function is used at core compaction's page isolation scheme, therefore
> - * most pages exposed to it are not enlisted as balloon pages and so, to avoid
> - * undesired side effects like racing against __free_pages(), we cannot afford
> - * holding the page locked while testing page->mapping->flags here.
> - *
> - * As we might return false positives in the case of a balloon page being just
> - * released under us, the page->mapping->flags need to be re-tested later,
> - * under the proper page lock, at the functions that will be coping with the
> - * balloon page case.
> - */
> -static inline bool balloon_page_movable(struct page *page)
> -{
> - /*
> - * Before dereferencing and testing mapping->flags, let's make sure
> - * this is not a page that uses ->mapping in a different way
> - */
> - if (page_flags_cleared(page) && !page_mapped(page) &&
> - page_count(page) == 1)
> - return __is_movable_balloon_page(page);
> -
> - return false;
> -}
> -
> -/*
> - * isolated_balloon_page - identify an isolated balloon page on private
> - * compaction/migration page lists.
> - *
> - * After a compaction thread isolates a balloon page for migration, it raises
> - * the page refcount to prevent concurrent compaction threads from re-isolating
> - * the same page. For that reason putback_movable_pages(), or other routines
> - * that need to identify isolated balloon pages on private pagelists, cannot
> - * rely on balloon_page_movable() to accomplish the task.
> - */
> -static inline bool isolated_balloon_page(struct page *page)
> -{
> - /* Already isolated balloon pages, by default, have a raised refcount */
> - if (page_flags_cleared(page) && !page_mapped(page) &&
> - page_count(page) >= 2)
> - return __is_movable_balloon_page(page);
> -
> - return false;
> -}
> -
> /*
> * balloon_page_insert - insert a page into the balloon's page list and make
> * the page->mapping assignment accordingly.
> @@ -192,6 +109,8 @@ static inline void balloon_page_insert(struct page *page,
> struct address_space *mapping,
> struct list_head *head)
> {
> + __SetPageBalloon(page);
> + inc_zone_page_state(page, NR_BALLOON_PAGES);
> page->mapping = mapping;
> list_add(&page->lru, head);
> }
> @@ -206,10 +125,29 @@ static inline void balloon_page_insert(struct page *page,
> */
> static inline void balloon_page_delete(struct page *page)
> {
> + __ClearPageBalloon(page);
> + dec_zone_page_state(page, NR_BALLOON_PAGES);
> page->mapping = NULL;
> list_del(&page->lru);
> }
>
> +#endif /* CONFIG_MEMORY_BALLOON */
> +
> +#ifdef CONFIG_BALLOON_COMPACTION
> +
> +extern bool balloon_page_isolate(struct page *page);
> +extern void balloon_page_putback(struct page *page);
> +extern int balloon_page_migrate(struct page *newpage,
> + struct page *page, enum migrate_mode mode);
> +extern struct address_space
> +*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> + const struct address_space_operations *a_ops);
> +
> +static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> +{
> + kfree(balloon_mapping);
> +}
> +
> /*
> * balloon_page_device - get the b_dev_info descriptor for the balloon device
> * that enqueues the given page.
> @@ -246,33 +184,6 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> return;
> }
>
> -static inline void balloon_page_insert(struct page *page,
> - struct address_space *mapping,
> - struct list_head *head)
> -{
> - list_add(&page->lru, head);
> -}
> -
> -static inline void balloon_page_delete(struct page *page)
> -{
> - list_del(&page->lru);
> -}
> -
> -static inline bool __is_movable_balloon_page(struct page *page)
> -{
> - return false;
> -}
> -
> -static inline bool balloon_page_movable(struct page *page)
> -{
> - return false;
> -}
> -
> -static inline bool isolated_balloon_page(struct page *page)
> -{
> - return false;
> -}
> -
> static inline bool balloon_page_isolate(struct page *page)
> {
> return false;
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 72e0db0..e09cf0a 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -237,7 +237,7 @@ config MEMORY_BALLOON
> config BALLOON_COMPACTION
> bool "Allow for balloon memory compaction/migration"
> def_bool y
> - depends on COMPACTION && VIRTIO_BALLOON
> + depends on COMPACTION && MEMORY_BALLOON
> help
> Memory fragmentation introduced by ballooning might reduce
> significantly the number of 2MB contiguous memory blocks that can be
> diff --git a/mm/Makefile b/mm/Makefile
> index a96e3a1..b2f18dc 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
> readahead.o swap.o truncate.o vmscan.o shmem.o \
> util.o mmzone.o vmstat.o backing-dev.o \
> mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o balloon_compaction.o vmacache.o \
> + compaction.o vmacache.o \
> interval_tree.o list_lru.o workingset.o \
> iov_iter.o $(mmu-y)
>
> @@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> obj-$(CONFIG_CMA) += cma.o
> +obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index 6e45a50..a942081 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)
> * Prevent concurrent compaction threads from isolating
> * an already isolated balloon page by refcount check.
> */
> - if (__is_movable_balloon_page(page) &&
> - page_count(page) == 2) {
> + if (PageBalloon(page) && page_count(page) == 2) {
> __isolate_balloon_page(page);
> unlock_page(page);
> return true;
> @@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)
> */
> lock_page(page);
>
> - if (__is_movable_balloon_page(page)) {
> + if (PageBalloon(page)) {
> __putback_balloon_page(page);
> /* drop the extra ref count taken for page isolation */
> put_page(page);
> @@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,
> */
> BUG_ON(!trylock_page(newpage));
>
> - if (WARN_ON(!__is_movable_balloon_page(page))) {
> + if (WARN_ON(!PageBalloon(page))) {
> dump_page(page, "not movable balloon page");
> unlock_page(newpage);
> return rc;
> diff --git a/mm/compaction.c b/mm/compaction.c
> index ad58f73..7d9d92e 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> * Skip any other type of page
> */
> if (!PageLRU(page)) {
> - if (unlikely(balloon_page_movable(page))) {
> - if (balloon_page_isolate(page)) {
> - /* Successfully isolated */
> - goto isolate_success;
> - }
> + if (unlikely(PageBalloon(page)) &&
> + balloon_page_isolate(page)) {
> + /* Successfully isolated */
> + goto isolate_success;
> }
> continue;
> }
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 57c94f9..a4939b1 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)
> list_del(&page->lru);
> dec_zone_page_state(page, NR_ISOLATED_ANON +
> page_is_file_cache(page));
> - if (unlikely(isolated_balloon_page(page)))
> + if (unlikely(PageBalloon(page)))
> balloon_page_putback(page);
> else
> putback_lru_page(page);
> @@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(__is_movable_balloon_page(page))) {
> + if (unlikely(PageBalloon(page))) {
> /*
> * A ballooned page does not need any special attention from
> * physical to virtual reverse mapping procedures.
> @@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
>
> rc = __unmap_and_move(page, newpage, force, mode);
>
> +#ifdef CONFIG_MEMORY_BALLOON
> if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
> /*
> * A ballooned page has been migrated already.
> @@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
> balloon_page_free(page);
> return MIGRATEPAGE_SUCCESS;
> }
> +#endif
> out:
> if (rc != -EAGAIN) {
> /*
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 1a71b8b..88dd901 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,
>
> list_for_each_entry_safe(page, next, page_list, lru) {
> if (page_is_file_cache(page) && !PageDirty(page) &&
> - !isolated_balloon_page(page)) {
> + !PageBalloon(page)) {
> ClearPageActive(page);
> list_move(&page->lru, &clean_pages);
> }
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
@ 2014-09-02 12:57 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 12:57 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:23PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
> with PageBalloon which is stored directly in the struct page.
> All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> drivers/virtio/Kconfig | 1
> include/linux/balloon_compaction.h | 135 ++++++------------------------------
> mm/Kconfig | 2 -
> mm/Makefile | 3 +
> mm/balloon_compaction.c | 7 +-
> mm/compaction.c | 9 +-
> mm/migrate.c | 6 +-
> mm/vmscan.c | 2 -
> 8 files changed, 39 insertions(+), 126 deletions(-)
>
> diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
> index c6683f2..00b2286 100644
> --- a/drivers/virtio/Kconfig
> +++ b/drivers/virtio/Kconfig
> @@ -25,6 +25,7 @@ config VIRTIO_PCI
> config VIRTIO_BALLOON
> tristate "Virtio balloon driver"
> depends on VIRTIO
> + select MEMORY_BALLOON
> ---help---
> This driver supports increasing and decreasing the amount
> of memory within a KVM guest.
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 284fc1d..09f8c5a 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -46,6 +46,8 @@
> #include <linux/gfp.h>
> #include <linux/err.h>
>
> +#ifdef CONFIG_MEMORY_BALLOON
> +
> /*
> * Balloon device information descriptor.
> * This struct is used to allow the common balloon compaction interface
> @@ -93,91 +95,6 @@ static inline void balloon_page_free(struct page *page)
> __free_page(page);
> }
>
> -#ifdef CONFIG_BALLOON_COMPACTION
> -extern bool balloon_page_isolate(struct page *page);
> -extern void balloon_page_putback(struct page *page);
> -extern int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode);
> -extern struct address_space
> -*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops);
> -
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - kfree(balloon_mapping);
> -}
> -
> -/*
> - * page_flags_cleared - helper to perform balloon @page ->flags tests.
> - *
> - * As balloon pages are obtained from buddy and we do not play with page->flags
> - * at driver level (exception made when we get the page lock for compaction),
> - * we can safely identify a ballooned page by checking if the
> - * PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also
> - * helps us skip ballooned pages that are locked for compaction or release, thus
> - * mitigating their racy check at balloon_page_movable()
> - */
> -static inline bool page_flags_cleared(struct page *page)
> -{
> - return !(page->flags & PAGE_FLAGS_CHECK_AT_PREP);
> -}
> -
> -/*
> - * __is_movable_balloon_page - helper to perform @page mapping->flags tests
> - */
> -static inline bool __is_movable_balloon_page(struct page *page)
> -{
> - struct address_space *mapping = page->mapping;
> - return !PageAnon(page) && mapping_balloon(mapping);
> -}
> -
> -/*
> - * balloon_page_movable - test page->mapping->flags to identify balloon pages
> - * that can be moved by compaction/migration.
> - *
> - * This function is used at core compaction's page isolation scheme, therefore
> - * most pages exposed to it are not enlisted as balloon pages and so, to avoid
> - * undesired side effects like racing against __free_pages(), we cannot afford
> - * holding the page locked while testing page->mapping->flags here.
> - *
> - * As we might return false positives in the case of a balloon page being just
> - * released under us, the page->mapping->flags need to be re-tested later,
> - * under the proper page lock, at the functions that will be coping with the
> - * balloon page case.
> - */
> -static inline bool balloon_page_movable(struct page *page)
> -{
> - /*
> - * Before dereferencing and testing mapping->flags, let's make sure
> - * this is not a page that uses ->mapping in a different way
> - */
> - if (page_flags_cleared(page) && !page_mapped(page) &&
> - page_count(page) == 1)
> - return __is_movable_balloon_page(page);
> -
> - return false;
> -}
> -
> -/*
> - * isolated_balloon_page - identify an isolated balloon page on private
> - * compaction/migration page lists.
> - *
> - * After a compaction thread isolates a balloon page for migration, it raises
> - * the page refcount to prevent concurrent compaction threads from re-isolating
> - * the same page. For that reason putback_movable_pages(), or other routines
> - * that need to identify isolated balloon pages on private pagelists, cannot
> - * rely on balloon_page_movable() to accomplish the task.
> - */
> -static inline bool isolated_balloon_page(struct page *page)
> -{
> - /* Already isolated balloon pages, by default, have a raised refcount */
> - if (page_flags_cleared(page) && !page_mapped(page) &&
> - page_count(page) >= 2)
> - return __is_movable_balloon_page(page);
> -
> - return false;
> -}
> -
> /*
> * balloon_page_insert - insert a page into the balloon's page list and make
> * the page->mapping assignment accordingly.
> @@ -192,6 +109,8 @@ static inline void balloon_page_insert(struct page *page,
> struct address_space *mapping,
> struct list_head *head)
> {
> + __SetPageBalloon(page);
> + inc_zone_page_state(page, NR_BALLOON_PAGES);
> page->mapping = mapping;
> list_add(&page->lru, head);
> }
> @@ -206,10 +125,29 @@ static inline void balloon_page_insert(struct page *page,
> */
> static inline void balloon_page_delete(struct page *page)
> {
> + __ClearPageBalloon(page);
> + dec_zone_page_state(page, NR_BALLOON_PAGES);
> page->mapping = NULL;
> list_del(&page->lru);
> }
>
> +#endif /* CONFIG_MEMORY_BALLOON */
> +
> +#ifdef CONFIG_BALLOON_COMPACTION
> +
> +extern bool balloon_page_isolate(struct page *page);
> +extern void balloon_page_putback(struct page *page);
> +extern int balloon_page_migrate(struct page *newpage,
> + struct page *page, enum migrate_mode mode);
> +extern struct address_space
> +*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> + const struct address_space_operations *a_ops);
> +
> +static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> +{
> + kfree(balloon_mapping);
> +}
> +
> /*
> * balloon_page_device - get the b_dev_info descriptor for the balloon device
> * that enqueues the given page.
> @@ -246,33 +184,6 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> return;
> }
>
> -static inline void balloon_page_insert(struct page *page,
> - struct address_space *mapping,
> - struct list_head *head)
> -{
> - list_add(&page->lru, head);
> -}
> -
> -static inline void balloon_page_delete(struct page *page)
> -{
> - list_del(&page->lru);
> -}
> -
> -static inline bool __is_movable_balloon_page(struct page *page)
> -{
> - return false;
> -}
> -
> -static inline bool balloon_page_movable(struct page *page)
> -{
> - return false;
> -}
> -
> -static inline bool isolated_balloon_page(struct page *page)
> -{
> - return false;
> -}
> -
> static inline bool balloon_page_isolate(struct page *page)
> {
> return false;
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 72e0db0..e09cf0a 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -237,7 +237,7 @@ config MEMORY_BALLOON
> config BALLOON_COMPACTION
> bool "Allow for balloon memory compaction/migration"
> def_bool y
> - depends on COMPACTION && VIRTIO_BALLOON
> + depends on COMPACTION && MEMORY_BALLOON
> help
> Memory fragmentation introduced by ballooning might reduce
> significantly the number of 2MB contiguous memory blocks that can be
> diff --git a/mm/Makefile b/mm/Makefile
> index a96e3a1..b2f18dc 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
> readahead.o swap.o truncate.o vmscan.o shmem.o \
> util.o mmzone.o vmstat.o backing-dev.o \
> mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o balloon_compaction.o vmacache.o \
> + compaction.o vmacache.o \
> interval_tree.o list_lru.o workingset.o \
> iov_iter.o $(mmu-y)
>
> @@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> obj-$(CONFIG_CMA) += cma.o
> +obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index 6e45a50..a942081 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)
> * Prevent concurrent compaction threads from isolating
> * an already isolated balloon page by refcount check.
> */
> - if (__is_movable_balloon_page(page) &&
> - page_count(page) == 2) {
> + if (PageBalloon(page) && page_count(page) == 2) {
> __isolate_balloon_page(page);
> unlock_page(page);
> return true;
> @@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)
> */
> lock_page(page);
>
> - if (__is_movable_balloon_page(page)) {
> + if (PageBalloon(page)) {
> __putback_balloon_page(page);
> /* drop the extra ref count taken for page isolation */
> put_page(page);
> @@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,
> */
> BUG_ON(!trylock_page(newpage));
>
> - if (WARN_ON(!__is_movable_balloon_page(page))) {
> + if (WARN_ON(!PageBalloon(page))) {
> dump_page(page, "not movable balloon page");
> unlock_page(newpage);
> return rc;
> diff --git a/mm/compaction.c b/mm/compaction.c
> index ad58f73..7d9d92e 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> * Skip any other type of page
> */
> if (!PageLRU(page)) {
> - if (unlikely(balloon_page_movable(page))) {
> - if (balloon_page_isolate(page)) {
> - /* Successfully isolated */
> - goto isolate_success;
> - }
> + if (unlikely(PageBalloon(page)) &&
> + balloon_page_isolate(page)) {
> + /* Successfully isolated */
> + goto isolate_success;
> }
> continue;
> }
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 57c94f9..a4939b1 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)
> list_del(&page->lru);
> dec_zone_page_state(page, NR_ISOLATED_ANON +
> page_is_file_cache(page));
> - if (unlikely(isolated_balloon_page(page)))
> + if (unlikely(PageBalloon(page)))
> balloon_page_putback(page);
> else
> putback_lru_page(page);
> @@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(__is_movable_balloon_page(page))) {
> + if (unlikely(PageBalloon(page))) {
> /*
> * A ballooned page does not need any special attention from
> * physical to virtual reverse mapping procedures.
> @@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
>
> rc = __unmap_and_move(page, newpage, force, mode);
>
> +#ifdef CONFIG_MEMORY_BALLOON
> if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
> /*
> * A ballooned page has been migrated already.
> @@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
> balloon_page_free(page);
> return MIGRATEPAGE_SUCCESS;
> }
> +#endif
> out:
> if (rc != -EAGAIN) {
> /*
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 1a71b8b..88dd901 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,
>
> list_for_each_entry_safe(page, next, page_list, lru) {
> if (page_is_file_cache(page) && !PageDirty(page) &&
> - !isolated_balloon_page(page)) {
> + !PageBalloon(page)) {
> ClearPageActive(page);
> list_move(&page->lru, &clean_pages);
> }
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-02 13:09 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 13:09 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:27PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> * move special branch for balloon migraion into migrate_pages
> * remove special mapping for balloon and its flag AS_BALLOON_MAP
> * embed struct balloon_dev_info into struct virtio_balloon
> * cleanup balloon_page_dequeue, kill balloon_page_free
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> drivers/virtio/virtio_balloon.c | 77 ++++---------
> include/linux/balloon_compaction.h | 127 ++++++---------------
> include/linux/migrate.h | 11 --
> include/linux/pagemap.h | 18 ---
> mm/balloon_compaction.c | 214 ++++++++++++------------------------
> mm/migrate.c | 29 +----
> 6 files changed, 134 insertions(+), 342 deletions(-)
>
> diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
> index 25ebe8e..c84d6a8 100644
> --- a/drivers/virtio/virtio_balloon.c
> +++ b/drivers/virtio/virtio_balloon.c
> @@ -59,7 +59,7 @@ struct virtio_balloon
> * Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
> * to num_pages above.
> */
> - struct balloon_dev_info *vb_dev_info;
> + struct balloon_dev_info vb_dev_info;
>
> /* Synchronize access/update to this struct virtio_balloon elements */
> struct mutex balloon_lock;
> @@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)
>
> static void fill_balloon(struct virtio_balloon *vb, size_t num)
> {
> - struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
> + struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
>
> /* We can only do one array worth at a time. */
> num = min(num, ARRAY_SIZE(vb->pfns));
> @@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
> /* Find pfns pointing at start of each page, get pages and free them. */
> for (i = 0; i < num; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
> struct page *page = balloon_pfn_to_page(pfns[i]);
> - balloon_page_free(page);
> adjust_managed_page_count(page, 1);
> + put_page(page);
> }
> }
>
> static void leak_balloon(struct virtio_balloon *vb, size_t num)
> {
> struct page *page;
> - struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
> + struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
>
> /* We can only do one array worth at a time. */
> num = min(num, ARRAY_SIZE(vb->pfns));
> @@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)
> return 0;
> }
>
> -static const struct address_space_operations virtio_balloon_aops;
> #ifdef CONFIG_BALLOON_COMPACTION
> /*
> * virtballoon_migratepage - perform the balloon page migration on behalf of
> * a compation thread. (called under page lock)
> - * @mapping: the page->mapping which will be assigned to the new migrated page.
> + * @vb_dev_info: the balloon device
> * @newpage: page that will replace the isolated page after migration finishes.
> * @page : the isolated (old) page that is about to be migrated to newpage.
> * @mode : compaction mode -- not used for balloon page migration.
> @@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;
> * This function preforms the balloon page migration task.
> * Called through balloon_mapping->a_ops->migratepage
> */
> -static int virtballoon_migratepage(struct address_space *mapping,
> +static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
> struct page *newpage, struct page *page, enum migrate_mode mode)
> {
> - struct balloon_dev_info *vb_dev_info = balloon_page_device(page);
> - struct virtio_balloon *vb;
> + struct virtio_balloon *vb = container_of(vb_dev_info,
> + struct virtio_balloon, vb_dev_info);
> unsigned long flags;
>
> - BUG_ON(!vb_dev_info);
> -
> - vb = vb_dev_info->balloon_device;
> -
> /*
> * In order to avoid lock contention while migrating pages concurrently
> * to leak_balloon() or fill_balloon() we just give up the balloon_lock
> @@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,
> if (!mutex_trylock(&vb->balloon_lock))
> return -EAGAIN;
>
> + get_page(newpage); /* balloon reference */
> +
> /* balloon's page migration 1st step -- inflate "newpage" */
> spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
> - balloon_page_insert(newpage, mapping, &vb_dev_info->pages);
> + balloon_page_insert(vb_dev_info, newpage);
> vb_dev_info->isolated_pages--;
> spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> set_page_pfns(vb->pfns, newpage);
> tell_host(vb, vb->inflate_vq);
>
> - /*
> - * balloon's page migration 2nd step -- deflate "page"
> - *
> - * It's safe to delete page->lru here because this page is at
> - * an isolated migration list, and this step is expected to happen here
> - */
> - balloon_page_delete(page);
> + /* balloon's page migration 2nd step -- deflate "page" */
> + balloon_page_delete(page, true);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> set_page_pfns(vb->pfns, page);
> tell_host(vb, vb->deflate_vq);
>
> mutex_unlock(&vb->balloon_lock);
>
> - return MIGRATEPAGE_BALLOON_SUCCESS;
> -}
> + put_page(page); /* balloon reference */
>
> -/* define the balloon_mapping->a_ops callback to allow balloon page migration */
> -static const struct address_space_operations virtio_balloon_aops = {
> - .migratepage = virtballoon_migratepage,
> -};
> + return MIGRATEPAGE_SUCCESS;
> +}
> #endif /* CONFIG_BALLOON_COMPACTION */
>
> static int virtballoon_probe(struct virtio_device *vdev)
> {
> struct virtio_balloon *vb;
> - struct address_space *vb_mapping;
> - struct balloon_dev_info *vb_devinfo;
> int err;
>
> vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
> @@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev)
> vb->vdev = vdev;
> vb->need_stats_update = 0;
>
> - vb_devinfo = balloon_devinfo_alloc(vb);
> - if (IS_ERR(vb_devinfo)) {
> - err = PTR_ERR(vb_devinfo);
> - goto out_free_vb;
> - }
> -
> - vb_mapping = balloon_mapping_alloc(vb_devinfo,
> - (balloon_compaction_check()) ?
> - &virtio_balloon_aops : NULL);
> - if (IS_ERR(vb_mapping)) {
> - /*
> - * IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP
> - * This means !CONFIG_BALLOON_COMPACTION, otherwise we get off.
> - */
> - err = PTR_ERR(vb_mapping);
> - if (err != -EOPNOTSUPP)
> - goto out_free_vb_devinfo;
> - }
> -
> - vb->vb_dev_info = vb_devinfo;
> + balloon_devinfo_init(&vb->vb_dev_info);
> +#ifdef CONFIG_BALLOON_COMPACTION
> + vb->vb_dev_info.migratepage = virtballoon_migratepage;
> +#endif
>
> err = init_vqs(vb);
> if (err)
> - goto out_free_vb_mapping;
> + goto out_free_vb;
>
> vb->thread = kthread_run(balloon, vb, "vballoon");
> if (IS_ERR(vb->thread)) {
> @@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
>
> out_del_vqs:
> vdev->config->del_vqs(vdev);
> -out_free_vb_mapping:
> - balloon_mapping_free(vb_mapping);
> -out_free_vb_devinfo:
> - balloon_devinfo_free(vb_devinfo);
> out_free_vb:
> kfree(vb);
> out:
> @@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)
>
> kthread_stop(vb->thread);
> remove_common(vb);
> - balloon_mapping_free(vb->vb_dev_info->mapping);
> - balloon_devinfo_free(vb->vb_dev_info);
> kfree(vb);
> }
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 09f8c5a..ad112fcc6 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -56,96 +56,58 @@
> * balloon driver as a page book-keeper for its registered balloon devices.
> */
> struct balloon_dev_info {
> - void *balloon_device; /* balloon device descriptor */
> - struct address_space *mapping; /* balloon special page->mapping */
> unsigned long isolated_pages; /* # of isolated pages for migration */
> spinlock_t pages_lock; /* Protection to pages list */
> struct list_head pages; /* Pages enqueued & handled to Host */
> + int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
> + struct page *page, enum migrate_mode mode);
> };
>
> -extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
> -extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
> -extern struct balloon_dev_info *balloon_devinfo_alloc(
> - void *balloon_dev_descriptor);
> -
> -static inline void balloon_devinfo_free(struct balloon_dev_info *b_dev_info)
> +static inline void balloon_devinfo_init(struct balloon_dev_info *b_dev_info)
> {
> - kfree(b_dev_info);
> + b_dev_info->isolated_pages = 0;
> + spin_lock_init(&b_dev_info->pages_lock);
> + INIT_LIST_HEAD(&b_dev_info->pages);
> + b_dev_info->migratepage = NULL;
> }
>
> -/*
> - * balloon_page_free - release a balloon page back to the page free lists
> - * @page: ballooned page to be set free
> - *
> - * This function must be used to properly set free an isolated/dequeued balloon
> - * page at the end of a sucessful page migration, or at the balloon driver's
> - * page release procedure.
> - */
> -static inline void balloon_page_free(struct page *page)
> -{
> - /*
> - * Balloon pages always get an extra refcount before being isolated
> - * and before being dequeued to help on sorting out fortuite colisions
> - * between a thread attempting to isolate and another thread attempting
> - * to release the very same balloon page.
> - *
> - * Before we handle the page back to Buddy, lets drop its extra refcnt.
> - */
> - put_page(page);
> - __free_page(page);
> -}
> +extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
> +extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
>
> /*
> - * balloon_page_insert - insert a page into the balloon's page list and make
> - * the page->mapping assignment accordingly.
> - * @page : page to be assigned as a 'balloon page'
> - * @mapping : allocated special 'balloon_mapping'
> - * @head : balloon's device page list head
> + * balloon_page_insert - insert a page into the balloon's page list,
> + * mark and account it accordingly.
> + * @b_dev_info : pinter to ballon device
> + * @page : page to be assigned as a 'balloon page'
> *
> * Caller must ensure the page is locked and the spin_lock protecting balloon
> * pages list is held before inserting a page into the balloon device.
> */
> -static inline void balloon_page_insert(struct page *page,
> - struct address_space *mapping,
> - struct list_head *head)
> +static inline void
> +balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
> {
> __SetPageBalloon(page);
> inc_zone_page_state(page, NR_BALLOON_PAGES);
> - page->mapping = mapping;
> - list_add(&page->lru, head);
> + set_page_private(page, (unsigned long)b_dev_info);
> + list_add(&page->lru, &b_dev_info->pages);
> }
>
> /*
> * balloon_page_delete - delete a page from balloon's page list and clear
> - * the page->mapping assignement accordingly.
> + * the ballon page mark accordingly.
> * @page : page to be released from balloon's page list
> + * @isolated: already isolated, do not delete from list
> *
> * Caller must ensure the page is locked and the spin_lock protecting balloon
> * pages list is held before deleting a page from the balloon device.
> */
> -static inline void balloon_page_delete(struct page *page)
> +static inline void balloon_page_delete(struct page *page, bool isolated)
> {
> __ClearPageBalloon(page);
> dec_zone_page_state(page, NR_BALLOON_PAGES);
> - page->mapping = NULL;
> - list_del(&page->lru);
> -}
> -
> -#endif /* CONFIG_MEMORY_BALLOON */
> -
> -#ifdef CONFIG_BALLOON_COMPACTION
> -
> -extern bool balloon_page_isolate(struct page *page);
> -extern void balloon_page_putback(struct page *page);
> -extern int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode);
> -extern struct address_space
> -*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops);
> -
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - kfree(balloon_mapping);
> + set_page_private(page, 0);
> + if (!isolated)
> + list_del(&page->lru);
> }
>
> /*
> @@ -154,35 +116,25 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> */
> static inline struct balloon_dev_info *balloon_page_device(struct page *page)
> {
> - struct address_space *mapping = page->mapping;
> - if (likely(mapping))
> - return mapping->private_data;
> -
> - return NULL;
> + return (struct balloon_dev_info *)page_private(page);
> }
>
> -static inline gfp_t balloon_mapping_gfp_mask(void)
> -{
> - return GFP_HIGHUSER_MOVABLE;
> -}
> +#endif /* CONFIG_MEMORY_BALLOON */
>
> -static inline bool balloon_compaction_check(void)
> -{
> - return true;
> -}
> +#ifdef CONFIG_BALLOON_COMPACTION
> +extern bool balloon_page_isolate(struct page *page);
> +extern void balloon_page_putback(struct page *page);
>
> -#else /* !CONFIG_BALLOON_COMPACTION */
> +int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
> + unsigned long private, struct page *page,
> + int force, enum migrate_mode mode);
>
> -static inline void *balloon_mapping_alloc(void *balloon_device,
> - const struct address_space_operations *a_ops)
> +static inline gfp_t balloon_mapping_gfp_mask(void)
> {
> - return ERR_PTR(-EOPNOTSUPP);
> + return GFP_HIGHUSER_MOVABLE;
> }
>
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - return;
> -}
> +#else /* !CONFIG_BALLOON_COMPACTION */
>
> static inline bool balloon_page_isolate(struct page *page)
> {
> @@ -194,10 +146,11 @@ static inline void balloon_page_putback(struct page *page)
> return;
> }
>
> -static inline int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode)
> +static inline int balloon_page_migrate(new_page_t get_new_page,
> + free_page_t put_new_page, unsigned long private,
> + struct page *page, int force, enum migrate_mode mode)
> {
> - return 0;
> + return -EAGAIN;
> }
>
> static inline gfp_t balloon_mapping_gfp_mask(void)
> @@ -205,9 +158,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
> return GFP_HIGHUSER;
> }
>
> -static inline bool balloon_compaction_check(void)
> -{
> - return false;
> -}
> #endif /* CONFIG_BALLOON_COMPACTION */
> #endif /* _LINUX_BALLOON_COMPACTION_H */
> diff --git a/include/linux/migrate.h b/include/linux/migrate.h
> index 0a4604a..cf90776 100644
> --- a/include/linux/migrate.h
> +++ b/include/linux/migrate.h
> @@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);
> * Return values from addresss_space_operations.migratepage():
> * - negative errno on page migration failure;
> * - zero on page migration success;
> - *
> - * The balloon page migration introduces this special case where a 'distinct'
> - * return code is used to flag a successful page migration to unmap_and_move().
> - * This approach is necessary because page migration can race against balloon
> - * deflation procedure, and for such case we could introduce a nasty page leak
> - * if a successfully migrated balloon page gets released concurrently with
> - * migration's unmap_and_move() wrap-up steps.
> */
> #define MIGRATEPAGE_SUCCESS 0
> -#define MIGRATEPAGE_BALLOON_SUCCESS 1 /* special ret code for balloon page
> - * sucessful migration case.
> - */
> +
> enum migrate_reason {
> MR_COMPACTION,
> MR_MEMORY_FAILURE,
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index 5ba1813..210b46b 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -24,8 +24,7 @@ enum mapping_flags {
> AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
> AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
> AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
> - AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
> - AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */
> + AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
> };
>
> static inline void mapping_set_error(struct address_space *mapping, int error)
> @@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)
> return !!mapping;
> }
>
> -static inline void mapping_set_balloon(struct address_space *mapping)
> -{
> - set_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> -static inline void mapping_clear_balloon(struct address_space *mapping)
> -{
> - clear_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> -static inline int mapping_balloon(struct address_space *mapping)
> -{
> - return mapping && test_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> static inline void mapping_set_exiting(struct address_space *mapping)
> {
> set_bit(AS_EXITING, &mapping->flags);
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index a942081..3c8cb7a 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -11,32 +11,6 @@
> #include <linux/balloon_compaction.h>
>
> /*
> - * balloon_devinfo_alloc - allocates a balloon device information descriptor.
> - * @balloon_dev_descriptor: pointer to reference the balloon device which
> - * this struct balloon_dev_info will be servicing.
> - *
> - * Driver must call it to properly allocate and initialize an instance of
> - * struct balloon_dev_info which will be used to reference a balloon device
> - * as well as to keep track of the balloon device page list.
> - */
> -struct balloon_dev_info *balloon_devinfo_alloc(void *balloon_dev_descriptor)
> -{
> - struct balloon_dev_info *b_dev_info;
> - b_dev_info = kmalloc(sizeof(*b_dev_info), GFP_KERNEL);
> - if (!b_dev_info)
> - return ERR_PTR(-ENOMEM);
> -
> - b_dev_info->balloon_device = balloon_dev_descriptor;
> - b_dev_info->mapping = NULL;
> - b_dev_info->isolated_pages = 0;
> - spin_lock_init(&b_dev_info->pages_lock);
> - INIT_LIST_HEAD(&b_dev_info->pages);
> -
> - return b_dev_info;
> -}
> -EXPORT_SYMBOL_GPL(balloon_devinfo_alloc);
> -
> -/*
> * balloon_page_enqueue - allocates a new page and inserts it into the balloon
> * page list.
> * @b_dev_info: balloon device decriptor where we will insert a new page to
> @@ -61,7 +35,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)
> */
> BUG_ON(!trylock_page(page));
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - balloon_page_insert(page, b_dev_info->mapping, &b_dev_info->pages);
> + balloon_page_insert(b_dev_info, page);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> @@ -81,12 +55,10 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
> */
> struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
> {
> - struct page *page, *tmp;
> + struct page *page;
> unsigned long flags;
> - bool dequeued_page;
>
> - dequeued_page = false;
> - list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) {
> + list_for_each_entry(page, &b_dev_info->pages, lru) {
> /*
> * Block others from accessing the 'page' while we get around
> * establishing additional references and preparing the 'page'
> @@ -94,98 +66,32 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
> */
> if (trylock_page(page)) {
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - /*
> - * Raise the page refcount here to prevent any wrong
> - * attempt to isolate this page, in case of coliding
> - * with balloon_page_isolate() just after we release
> - * the page lock.
> - *
> - * balloon_page_free() will take care of dropping
> - * this extra refcount later.
> - */
> - get_page(page);
> - balloon_page_delete(page);
> + balloon_page_delete(page, false);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> - dequeued_page = true;
> - break;
> + return page;
> }
> }
>
> - if (!dequeued_page) {
> - /*
> - * If we are unable to dequeue a balloon page because the page
> - * list is empty and there is no isolated pages, then something
> - * went out of track and some balloon pages are lost.
> - * BUG() here, otherwise the balloon driver may get stuck into
> - * an infinite loop while attempting to release all its pages.
> - */
> - spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - if (unlikely(list_empty(&b_dev_info->pages) &&
> - !b_dev_info->isolated_pages))
> - BUG();
> - spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> - page = NULL;
> - }
> - return page;
> + /*
> + * If we are unable to dequeue a balloon page because the page
> + * list is empty and there is no isolated pages, then something
> + * went out of track and some balloon pages are lost.
> + * BUG() here, otherwise the balloon driver may get stuck into
> + * an infinite loop while attempting to release all its pages.
> + */
> + spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> + BUG_ON(list_empty(&b_dev_info->pages) && !b_dev_info->isolated_pages);
> + spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> + return NULL;
> }
> EXPORT_SYMBOL_GPL(balloon_page_dequeue);
>
> #ifdef CONFIG_BALLOON_COMPACTION
> -/*
> - * balloon_mapping_alloc - allocates a special ->mapping for ballooned pages.
> - * @b_dev_info: holds the balloon device information descriptor.
> - * @a_ops: balloon_mapping address_space_operations descriptor.
> - *
> - * Driver must call it to properly allocate and initialize an instance of
> - * struct address_space which will be used as the special page->mapping for
> - * balloon device enlisted page instances.
> - */
> -struct address_space *balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops)
> -{
> - struct address_space *mapping;
> -
> - mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
> - if (!mapping)
> - return ERR_PTR(-ENOMEM);
> -
> - /*
> - * Give a clean 'zeroed' status to all elements of this special
> - * balloon page->mapping struct address_space instance.
> - */
> - address_space_init_once(mapping);
> -
> - /*
> - * Set mapping->flags appropriately, to allow balloon pages
> - * ->mapping identification.
> - */
> - mapping_set_balloon(mapping);
> - mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
> -
> - /* balloon's page->mapping->a_ops callback descriptor */
> - mapping->a_ops = a_ops;
> -
> - /*
> - * Establish a pointer reference back to the balloon device descriptor
> - * this particular page->mapping will be servicing.
> - * This is used by compaction / migration procedures to identify and
> - * access the balloon device pageset while isolating / migrating pages.
> - *
> - * As some balloon drivers can register multiple balloon devices
> - * for a single guest, this also helps compaction / migration to
> - * properly deal with multiple balloon pagesets, when required.
> - */
> - mapping->private_data = b_dev_info;
> - b_dev_info->mapping = mapping;
> -
> - return mapping;
> -}
> -EXPORT_SYMBOL_GPL(balloon_mapping_alloc);
>
> static inline void __isolate_balloon_page(struct page *page)
> {
> - struct balloon_dev_info *b_dev_info = page->mapping->private_data;
> + struct balloon_dev_info *b_dev_info = balloon_page_device(page);
> unsigned long flags;
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> list_del(&page->lru);
> @@ -195,7 +101,7 @@ static inline void __isolate_balloon_page(struct page *page)
>
> static inline void __putback_balloon_page(struct page *page)
> {
> - struct balloon_dev_info *b_dev_info = page->mapping->private_data;
> + struct balloon_dev_info *b_dev_info = balloon_page_device(page);
> unsigned long flags;
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> list_add(&page->lru, &b_dev_info->pages);
> @@ -203,12 +109,6 @@ static inline void __putback_balloon_page(struct page *page)
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> }
>
> -static inline int __migrate_balloon_page(struct address_space *mapping,
> - struct page *newpage, struct page *page, enum migrate_mode mode)
> -{
> - return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
> -}
> -
> /* __isolate_lru_page() counterpart for a ballooned page */
> bool balloon_page_isolate(struct page *page)
> {
> @@ -251,6 +151,57 @@ bool balloon_page_isolate(struct page *page)
> return false;
> }
>
> +int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
> + unsigned long private, struct page *page,
> + int force, enum migrate_mode mode)
> +{
> + struct balloon_dev_info *balloon = balloon_page_device(page);
> + struct page *newpage;
> + int *result = NULL;
> + int rc = -EAGAIN;
> +
> + if (!balloon || !balloon->migratepage)
> + return -EAGAIN;
> +
> + newpage = get_new_page(page, private, &result);
> + if (!newpage)
> + return -ENOMEM;
> +
> + if (!trylock_page(newpage))
> + BUG();
> +
> + if (!trylock_page(page)) {
> + if (!force || mode != MIGRATE_SYNC)
> + goto out;
> + lock_page(page);
> + }
> +
> + rc = balloon->migratepage(balloon, newpage, page, mode);
> +
> + unlock_page(page);
> +out:
> + unlock_page(newpage);
> +
> + if (rc != -EAGAIN) {
> + dec_zone_page_state(page, NR_ISOLATED_FILE);
> + list_del(&page->lru);
> + put_page(page);
> + }
> +
> + if (rc != MIGRATEPAGE_SUCCESS && put_new_page)
> + put_new_page(newpage, private);
> + else
> + put_page(newpage);
> +
> + if (result) {
> + if (rc)
> + *result = rc;
> + else
> + *result = page_to_nid(newpage);
> + }
> + return rc;
> +}
> +
> /* putback_lru_page() counterpart for a ballooned page */
> void balloon_page_putback(struct page *page)
> {
> @@ -271,31 +222,4 @@ void balloon_page_putback(struct page *page)
> unlock_page(page);
> }
>
> -/* move_to_new_page() counterpart for a ballooned page */
> -int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode)
> -{
> - struct address_space *mapping;
> - int rc = -EAGAIN;
> -
> - /*
> - * Block others from accessing the 'newpage' when we get around to
> - * establishing additional references. We should be the only one
> - * holding a reference to the 'newpage' at this point.
> - */
> - BUG_ON(!trylock_page(newpage));
> -
> - if (WARN_ON(!PageBalloon(page))) {
> - dump_page(page, "not movable balloon page");
> - unlock_page(newpage);
> - return rc;
> - }
> -
> - mapping = page->mapping;
> - if (mapping)
> - rc = __migrate_balloon_page(mapping, newpage, page, mode);
> -
> - unlock_page(newpage);
> - return rc;
> -}
> #endif /* CONFIG_BALLOON_COMPACTION */
> diff --git a/mm/migrate.c b/mm/migrate.c
> index a4939b1..e6d0d2d 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(PageBalloon(page))) {
> - /*
> - * A ballooned page does not need any special attention from
> - * physical to virtual reverse mapping procedures.
> - * Skip any attempt to unmap PTEs or to remap swap cache,
> - * in order to avoid burning cycles at rmap level, and perform
> - * the page migration right away (proteced by page lock).
> - */
> - rc = balloon_page_migrate(newpage, page, mode);
> - goto out_unlock;
> - }
> -
> /*
> * Corner case handling:
> * 1. When a new swap-cache page is read into, it is added to the LRU
> @@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
>
> rc = __unmap_and_move(page, newpage, force, mode);
>
> -#ifdef CONFIG_MEMORY_BALLOON
> - if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
> - /*
> - * A ballooned page has been migrated already.
> - * Now, it's the time to wrap-up counters,
> - * handle the page back to Buddy and return.
> - */
> - dec_zone_page_state(page, NR_ISOLATED_ANON +
> - page_is_file_cache(page));
> - balloon_page_free(page);
> - return MIGRATEPAGE_SUCCESS;
> - }
> -#endif
> out:
> if (rc != -EAGAIN) {
> /*
> @@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
> rc = unmap_and_move_huge_page(get_new_page,
> put_new_page, private, page,
> pass > 2, mode);
> + else if (PageBalloon(page))
> + rc = balloon_page_migrate(get_new_page,
> + put_new_page, private,
> + page, pass > 2, mode);
> else
> rc = unmap_and_move(get_new_page, put_new_page,
> private, page, pass > 2, mode);
>
Acked-by: Rafael Aquini <aquini@redhat.com>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
@ 2014-09-02 13:09 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-02 13:09 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Aug 30, 2014 at 08:41:27PM +0400, Konstantin Khlebnikov wrote:
> From: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
>
> * move special branch for balloon migraion into migrate_pages
> * remove special mapping for balloon and its flag AS_BALLOON_MAP
> * embed struct balloon_dev_info into struct virtio_balloon
> * cleanup balloon_page_dequeue, kill balloon_page_free
>
> Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
> ---
> drivers/virtio/virtio_balloon.c | 77 ++++---------
> include/linux/balloon_compaction.h | 127 ++++++---------------
> include/linux/migrate.h | 11 --
> include/linux/pagemap.h | 18 ---
> mm/balloon_compaction.c | 214 ++++++++++++------------------------
> mm/migrate.c | 29 +----
> 6 files changed, 134 insertions(+), 342 deletions(-)
>
> diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
> index 25ebe8e..c84d6a8 100644
> --- a/drivers/virtio/virtio_balloon.c
> +++ b/drivers/virtio/virtio_balloon.c
> @@ -59,7 +59,7 @@ struct virtio_balloon
> * Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
> * to num_pages above.
> */
> - struct balloon_dev_info *vb_dev_info;
> + struct balloon_dev_info vb_dev_info;
>
> /* Synchronize access/update to this struct virtio_balloon elements */
> struct mutex balloon_lock;
> @@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)
>
> static void fill_balloon(struct virtio_balloon *vb, size_t num)
> {
> - struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
> + struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
>
> /* We can only do one array worth at a time. */
> num = min(num, ARRAY_SIZE(vb->pfns));
> @@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
> /* Find pfns pointing at start of each page, get pages and free them. */
> for (i = 0; i < num; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
> struct page *page = balloon_pfn_to_page(pfns[i]);
> - balloon_page_free(page);
> adjust_managed_page_count(page, 1);
> + put_page(page);
> }
> }
>
> static void leak_balloon(struct virtio_balloon *vb, size_t num)
> {
> struct page *page;
> - struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
> + struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
>
> /* We can only do one array worth at a time. */
> num = min(num, ARRAY_SIZE(vb->pfns));
> @@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)
> return 0;
> }
>
> -static const struct address_space_operations virtio_balloon_aops;
> #ifdef CONFIG_BALLOON_COMPACTION
> /*
> * virtballoon_migratepage - perform the balloon page migration on behalf of
> * a compation thread. (called under page lock)
> - * @mapping: the page->mapping which will be assigned to the new migrated page.
> + * @vb_dev_info: the balloon device
> * @newpage: page that will replace the isolated page after migration finishes.
> * @page : the isolated (old) page that is about to be migrated to newpage.
> * @mode : compaction mode -- not used for balloon page migration.
> @@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;
> * This function preforms the balloon page migration task.
> * Called through balloon_mapping->a_ops->migratepage
> */
> -static int virtballoon_migratepage(struct address_space *mapping,
> +static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
> struct page *newpage, struct page *page, enum migrate_mode mode)
> {
> - struct balloon_dev_info *vb_dev_info = balloon_page_device(page);
> - struct virtio_balloon *vb;
> + struct virtio_balloon *vb = container_of(vb_dev_info,
> + struct virtio_balloon, vb_dev_info);
> unsigned long flags;
>
> - BUG_ON(!vb_dev_info);
> -
> - vb = vb_dev_info->balloon_device;
> -
> /*
> * In order to avoid lock contention while migrating pages concurrently
> * to leak_balloon() or fill_balloon() we just give up the balloon_lock
> @@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,
> if (!mutex_trylock(&vb->balloon_lock))
> return -EAGAIN;
>
> + get_page(newpage); /* balloon reference */
> +
> /* balloon's page migration 1st step -- inflate "newpage" */
> spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
> - balloon_page_insert(newpage, mapping, &vb_dev_info->pages);
> + balloon_page_insert(vb_dev_info, newpage);
> vb_dev_info->isolated_pages--;
> spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> set_page_pfns(vb->pfns, newpage);
> tell_host(vb, vb->inflate_vq);
>
> - /*
> - * balloon's page migration 2nd step -- deflate "page"
> - *
> - * It's safe to delete page->lru here because this page is at
> - * an isolated migration list, and this step is expected to happen here
> - */
> - balloon_page_delete(page);
> + /* balloon's page migration 2nd step -- deflate "page" */
> + balloon_page_delete(page, true);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> set_page_pfns(vb->pfns, page);
> tell_host(vb, vb->deflate_vq);
>
> mutex_unlock(&vb->balloon_lock);
>
> - return MIGRATEPAGE_BALLOON_SUCCESS;
> -}
> + put_page(page); /* balloon reference */
>
> -/* define the balloon_mapping->a_ops callback to allow balloon page migration */
> -static const struct address_space_operations virtio_balloon_aops = {
> - .migratepage = virtballoon_migratepage,
> -};
> + return MIGRATEPAGE_SUCCESS;
> +}
> #endif /* CONFIG_BALLOON_COMPACTION */
>
> static int virtballoon_probe(struct virtio_device *vdev)
> {
> struct virtio_balloon *vb;
> - struct address_space *vb_mapping;
> - struct balloon_dev_info *vb_devinfo;
> int err;
>
> vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
> @@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev)
> vb->vdev = vdev;
> vb->need_stats_update = 0;
>
> - vb_devinfo = balloon_devinfo_alloc(vb);
> - if (IS_ERR(vb_devinfo)) {
> - err = PTR_ERR(vb_devinfo);
> - goto out_free_vb;
> - }
> -
> - vb_mapping = balloon_mapping_alloc(vb_devinfo,
> - (balloon_compaction_check()) ?
> - &virtio_balloon_aops : NULL);
> - if (IS_ERR(vb_mapping)) {
> - /*
> - * IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP
> - * This means !CONFIG_BALLOON_COMPACTION, otherwise we get off.
> - */
> - err = PTR_ERR(vb_mapping);
> - if (err != -EOPNOTSUPP)
> - goto out_free_vb_devinfo;
> - }
> -
> - vb->vb_dev_info = vb_devinfo;
> + balloon_devinfo_init(&vb->vb_dev_info);
> +#ifdef CONFIG_BALLOON_COMPACTION
> + vb->vb_dev_info.migratepage = virtballoon_migratepage;
> +#endif
>
> err = init_vqs(vb);
> if (err)
> - goto out_free_vb_mapping;
> + goto out_free_vb;
>
> vb->thread = kthread_run(balloon, vb, "vballoon");
> if (IS_ERR(vb->thread)) {
> @@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
>
> out_del_vqs:
> vdev->config->del_vqs(vdev);
> -out_free_vb_mapping:
> - balloon_mapping_free(vb_mapping);
> -out_free_vb_devinfo:
> - balloon_devinfo_free(vb_devinfo);
> out_free_vb:
> kfree(vb);
> out:
> @@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)
>
> kthread_stop(vb->thread);
> remove_common(vb);
> - balloon_mapping_free(vb->vb_dev_info->mapping);
> - balloon_devinfo_free(vb->vb_dev_info);
> kfree(vb);
> }
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> index 09f8c5a..ad112fcc6 100644
> --- a/include/linux/balloon_compaction.h
> +++ b/include/linux/balloon_compaction.h
> @@ -56,96 +56,58 @@
> * balloon driver as a page book-keeper for its registered balloon devices.
> */
> struct balloon_dev_info {
> - void *balloon_device; /* balloon device descriptor */
> - struct address_space *mapping; /* balloon special page->mapping */
> unsigned long isolated_pages; /* # of isolated pages for migration */
> spinlock_t pages_lock; /* Protection to pages list */
> struct list_head pages; /* Pages enqueued & handled to Host */
> + int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
> + struct page *page, enum migrate_mode mode);
> };
>
> -extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
> -extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
> -extern struct balloon_dev_info *balloon_devinfo_alloc(
> - void *balloon_dev_descriptor);
> -
> -static inline void balloon_devinfo_free(struct balloon_dev_info *b_dev_info)
> +static inline void balloon_devinfo_init(struct balloon_dev_info *b_dev_info)
> {
> - kfree(b_dev_info);
> + b_dev_info->isolated_pages = 0;
> + spin_lock_init(&b_dev_info->pages_lock);
> + INIT_LIST_HEAD(&b_dev_info->pages);
> + b_dev_info->migratepage = NULL;
> }
>
> -/*
> - * balloon_page_free - release a balloon page back to the page free lists
> - * @page: ballooned page to be set free
> - *
> - * This function must be used to properly set free an isolated/dequeued balloon
> - * page at the end of a sucessful page migration, or at the balloon driver's
> - * page release procedure.
> - */
> -static inline void balloon_page_free(struct page *page)
> -{
> - /*
> - * Balloon pages always get an extra refcount before being isolated
> - * and before being dequeued to help on sorting out fortuite colisions
> - * between a thread attempting to isolate and another thread attempting
> - * to release the very same balloon page.
> - *
> - * Before we handle the page back to Buddy, lets drop its extra refcnt.
> - */
> - put_page(page);
> - __free_page(page);
> -}
> +extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
> +extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
>
> /*
> - * balloon_page_insert - insert a page into the balloon's page list and make
> - * the page->mapping assignment accordingly.
> - * @page : page to be assigned as a 'balloon page'
> - * @mapping : allocated special 'balloon_mapping'
> - * @head : balloon's device page list head
> + * balloon_page_insert - insert a page into the balloon's page list,
> + * mark and account it accordingly.
> + * @b_dev_info : pinter to ballon device
> + * @page : page to be assigned as a 'balloon page'
> *
> * Caller must ensure the page is locked and the spin_lock protecting balloon
> * pages list is held before inserting a page into the balloon device.
> */
> -static inline void balloon_page_insert(struct page *page,
> - struct address_space *mapping,
> - struct list_head *head)
> +static inline void
> +balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
> {
> __SetPageBalloon(page);
> inc_zone_page_state(page, NR_BALLOON_PAGES);
> - page->mapping = mapping;
> - list_add(&page->lru, head);
> + set_page_private(page, (unsigned long)b_dev_info);
> + list_add(&page->lru, &b_dev_info->pages);
> }
>
> /*
> * balloon_page_delete - delete a page from balloon's page list and clear
> - * the page->mapping assignement accordingly.
> + * the ballon page mark accordingly.
> * @page : page to be released from balloon's page list
> + * @isolated: already isolated, do not delete from list
> *
> * Caller must ensure the page is locked and the spin_lock protecting balloon
> * pages list is held before deleting a page from the balloon device.
> */
> -static inline void balloon_page_delete(struct page *page)
> +static inline void balloon_page_delete(struct page *page, bool isolated)
> {
> __ClearPageBalloon(page);
> dec_zone_page_state(page, NR_BALLOON_PAGES);
> - page->mapping = NULL;
> - list_del(&page->lru);
> -}
> -
> -#endif /* CONFIG_MEMORY_BALLOON */
> -
> -#ifdef CONFIG_BALLOON_COMPACTION
> -
> -extern bool balloon_page_isolate(struct page *page);
> -extern void balloon_page_putback(struct page *page);
> -extern int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode);
> -extern struct address_space
> -*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops);
> -
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - kfree(balloon_mapping);
> + set_page_private(page, 0);
> + if (!isolated)
> + list_del(&page->lru);
> }
>
> /*
> @@ -154,35 +116,25 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> */
> static inline struct balloon_dev_info *balloon_page_device(struct page *page)
> {
> - struct address_space *mapping = page->mapping;
> - if (likely(mapping))
> - return mapping->private_data;
> -
> - return NULL;
> + return (struct balloon_dev_info *)page_private(page);
> }
>
> -static inline gfp_t balloon_mapping_gfp_mask(void)
> -{
> - return GFP_HIGHUSER_MOVABLE;
> -}
> +#endif /* CONFIG_MEMORY_BALLOON */
>
> -static inline bool balloon_compaction_check(void)
> -{
> - return true;
> -}
> +#ifdef CONFIG_BALLOON_COMPACTION
> +extern bool balloon_page_isolate(struct page *page);
> +extern void balloon_page_putback(struct page *page);
>
> -#else /* !CONFIG_BALLOON_COMPACTION */
> +int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
> + unsigned long private, struct page *page,
> + int force, enum migrate_mode mode);
>
> -static inline void *balloon_mapping_alloc(void *balloon_device,
> - const struct address_space_operations *a_ops)
> +static inline gfp_t balloon_mapping_gfp_mask(void)
> {
> - return ERR_PTR(-EOPNOTSUPP);
> + return GFP_HIGHUSER_MOVABLE;
> }
>
> -static inline void balloon_mapping_free(struct address_space *balloon_mapping)
> -{
> - return;
> -}
> +#else /* !CONFIG_BALLOON_COMPACTION */
>
> static inline bool balloon_page_isolate(struct page *page)
> {
> @@ -194,10 +146,11 @@ static inline void balloon_page_putback(struct page *page)
> return;
> }
>
> -static inline int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode)
> +static inline int balloon_page_migrate(new_page_t get_new_page,
> + free_page_t put_new_page, unsigned long private,
> + struct page *page, int force, enum migrate_mode mode)
> {
> - return 0;
> + return -EAGAIN;
> }
>
> static inline gfp_t balloon_mapping_gfp_mask(void)
> @@ -205,9 +158,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
> return GFP_HIGHUSER;
> }
>
> -static inline bool balloon_compaction_check(void)
> -{
> - return false;
> -}
> #endif /* CONFIG_BALLOON_COMPACTION */
> #endif /* _LINUX_BALLOON_COMPACTION_H */
> diff --git a/include/linux/migrate.h b/include/linux/migrate.h
> index 0a4604a..cf90776 100644
> --- a/include/linux/migrate.h
> +++ b/include/linux/migrate.h
> @@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);
> * Return values from addresss_space_operations.migratepage():
> * - negative errno on page migration failure;
> * - zero on page migration success;
> - *
> - * The balloon page migration introduces this special case where a 'distinct'
> - * return code is used to flag a successful page migration to unmap_and_move().
> - * This approach is necessary because page migration can race against balloon
> - * deflation procedure, and for such case we could introduce a nasty page leak
> - * if a successfully migrated balloon page gets released concurrently with
> - * migration's unmap_and_move() wrap-up steps.
> */
> #define MIGRATEPAGE_SUCCESS 0
> -#define MIGRATEPAGE_BALLOON_SUCCESS 1 /* special ret code for balloon page
> - * sucessful migration case.
> - */
> +
> enum migrate_reason {
> MR_COMPACTION,
> MR_MEMORY_FAILURE,
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index 5ba1813..210b46b 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -24,8 +24,7 @@ enum mapping_flags {
> AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
> AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
> AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
> - AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
> - AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */
> + AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
> };
>
> static inline void mapping_set_error(struct address_space *mapping, int error)
> @@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)
> return !!mapping;
> }
>
> -static inline void mapping_set_balloon(struct address_space *mapping)
> -{
> - set_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> -static inline void mapping_clear_balloon(struct address_space *mapping)
> -{
> - clear_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> -static inline int mapping_balloon(struct address_space *mapping)
> -{
> - return mapping && test_bit(AS_BALLOON_MAP, &mapping->flags);
> -}
> -
> static inline void mapping_set_exiting(struct address_space *mapping)
> {
> set_bit(AS_EXITING, &mapping->flags);
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index a942081..3c8cb7a 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -11,32 +11,6 @@
> #include <linux/balloon_compaction.h>
>
> /*
> - * balloon_devinfo_alloc - allocates a balloon device information descriptor.
> - * @balloon_dev_descriptor: pointer to reference the balloon device which
> - * this struct balloon_dev_info will be servicing.
> - *
> - * Driver must call it to properly allocate and initialize an instance of
> - * struct balloon_dev_info which will be used to reference a balloon device
> - * as well as to keep track of the balloon device page list.
> - */
> -struct balloon_dev_info *balloon_devinfo_alloc(void *balloon_dev_descriptor)
> -{
> - struct balloon_dev_info *b_dev_info;
> - b_dev_info = kmalloc(sizeof(*b_dev_info), GFP_KERNEL);
> - if (!b_dev_info)
> - return ERR_PTR(-ENOMEM);
> -
> - b_dev_info->balloon_device = balloon_dev_descriptor;
> - b_dev_info->mapping = NULL;
> - b_dev_info->isolated_pages = 0;
> - spin_lock_init(&b_dev_info->pages_lock);
> - INIT_LIST_HEAD(&b_dev_info->pages);
> -
> - return b_dev_info;
> -}
> -EXPORT_SYMBOL_GPL(balloon_devinfo_alloc);
> -
> -/*
> * balloon_page_enqueue - allocates a new page and inserts it into the balloon
> * page list.
> * @b_dev_info: balloon device decriptor where we will insert a new page to
> @@ -61,7 +35,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)
> */
> BUG_ON(!trylock_page(page));
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - balloon_page_insert(page, b_dev_info->mapping, &b_dev_info->pages);
> + balloon_page_insert(b_dev_info, page);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> @@ -81,12 +55,10 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
> */
> struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
> {
> - struct page *page, *tmp;
> + struct page *page;
> unsigned long flags;
> - bool dequeued_page;
>
> - dequeued_page = false;
> - list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) {
> + list_for_each_entry(page, &b_dev_info->pages, lru) {
> /*
> * Block others from accessing the 'page' while we get around
> * establishing additional references and preparing the 'page'
> @@ -94,98 +66,32 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
> */
> if (trylock_page(page)) {
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - /*
> - * Raise the page refcount here to prevent any wrong
> - * attempt to isolate this page, in case of coliding
> - * with balloon_page_isolate() just after we release
> - * the page lock.
> - *
> - * balloon_page_free() will take care of dropping
> - * this extra refcount later.
> - */
> - get_page(page);
> - balloon_page_delete(page);
> + balloon_page_delete(page, false);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> - dequeued_page = true;
> - break;
> + return page;
> }
> }
>
> - if (!dequeued_page) {
> - /*
> - * If we are unable to dequeue a balloon page because the page
> - * list is empty and there is no isolated pages, then something
> - * went out of track and some balloon pages are lost.
> - * BUG() here, otherwise the balloon driver may get stuck into
> - * an infinite loop while attempting to release all its pages.
> - */
> - spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> - if (unlikely(list_empty(&b_dev_info->pages) &&
> - !b_dev_info->isolated_pages))
> - BUG();
> - spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> - page = NULL;
> - }
> - return page;
> + /*
> + * If we are unable to dequeue a balloon page because the page
> + * list is empty and there is no isolated pages, then something
> + * went out of track and some balloon pages are lost.
> + * BUG() here, otherwise the balloon driver may get stuck into
> + * an infinite loop while attempting to release all its pages.
> + */
> + spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> + BUG_ON(list_empty(&b_dev_info->pages) && !b_dev_info->isolated_pages);
> + spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> + return NULL;
> }
> EXPORT_SYMBOL_GPL(balloon_page_dequeue);
>
> #ifdef CONFIG_BALLOON_COMPACTION
> -/*
> - * balloon_mapping_alloc - allocates a special ->mapping for ballooned pages.
> - * @b_dev_info: holds the balloon device information descriptor.
> - * @a_ops: balloon_mapping address_space_operations descriptor.
> - *
> - * Driver must call it to properly allocate and initialize an instance of
> - * struct address_space which will be used as the special page->mapping for
> - * balloon device enlisted page instances.
> - */
> -struct address_space *balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
> - const struct address_space_operations *a_ops)
> -{
> - struct address_space *mapping;
> -
> - mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
> - if (!mapping)
> - return ERR_PTR(-ENOMEM);
> -
> - /*
> - * Give a clean 'zeroed' status to all elements of this special
> - * balloon page->mapping struct address_space instance.
> - */
> - address_space_init_once(mapping);
> -
> - /*
> - * Set mapping->flags appropriately, to allow balloon pages
> - * ->mapping identification.
> - */
> - mapping_set_balloon(mapping);
> - mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
> -
> - /* balloon's page->mapping->a_ops callback descriptor */
> - mapping->a_ops = a_ops;
> -
> - /*
> - * Establish a pointer reference back to the balloon device descriptor
> - * this particular page->mapping will be servicing.
> - * This is used by compaction / migration procedures to identify and
> - * access the balloon device pageset while isolating / migrating pages.
> - *
> - * As some balloon drivers can register multiple balloon devices
> - * for a single guest, this also helps compaction / migration to
> - * properly deal with multiple balloon pagesets, when required.
> - */
> - mapping->private_data = b_dev_info;
> - b_dev_info->mapping = mapping;
> -
> - return mapping;
> -}
> -EXPORT_SYMBOL_GPL(balloon_mapping_alloc);
>
> static inline void __isolate_balloon_page(struct page *page)
> {
> - struct balloon_dev_info *b_dev_info = page->mapping->private_data;
> + struct balloon_dev_info *b_dev_info = balloon_page_device(page);
> unsigned long flags;
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> list_del(&page->lru);
> @@ -195,7 +101,7 @@ static inline void __isolate_balloon_page(struct page *page)
>
> static inline void __putback_balloon_page(struct page *page)
> {
> - struct balloon_dev_info *b_dev_info = page->mapping->private_data;
> + struct balloon_dev_info *b_dev_info = balloon_page_device(page);
> unsigned long flags;
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> list_add(&page->lru, &b_dev_info->pages);
> @@ -203,12 +109,6 @@ static inline void __putback_balloon_page(struct page *page)
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> }
>
> -static inline int __migrate_balloon_page(struct address_space *mapping,
> - struct page *newpage, struct page *page, enum migrate_mode mode)
> -{
> - return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
> -}
> -
> /* __isolate_lru_page() counterpart for a ballooned page */
> bool balloon_page_isolate(struct page *page)
> {
> @@ -251,6 +151,57 @@ bool balloon_page_isolate(struct page *page)
> return false;
> }
>
> +int balloon_page_migrate(new_page_t get_new_page, free_page_t put_new_page,
> + unsigned long private, struct page *page,
> + int force, enum migrate_mode mode)
> +{
> + struct balloon_dev_info *balloon = balloon_page_device(page);
> + struct page *newpage;
> + int *result = NULL;
> + int rc = -EAGAIN;
> +
> + if (!balloon || !balloon->migratepage)
> + return -EAGAIN;
> +
> + newpage = get_new_page(page, private, &result);
> + if (!newpage)
> + return -ENOMEM;
> +
> + if (!trylock_page(newpage))
> + BUG();
> +
> + if (!trylock_page(page)) {
> + if (!force || mode != MIGRATE_SYNC)
> + goto out;
> + lock_page(page);
> + }
> +
> + rc = balloon->migratepage(balloon, newpage, page, mode);
> +
> + unlock_page(page);
> +out:
> + unlock_page(newpage);
> +
> + if (rc != -EAGAIN) {
> + dec_zone_page_state(page, NR_ISOLATED_FILE);
> + list_del(&page->lru);
> + put_page(page);
> + }
> +
> + if (rc != MIGRATEPAGE_SUCCESS && put_new_page)
> + put_new_page(newpage, private);
> + else
> + put_page(newpage);
> +
> + if (result) {
> + if (rc)
> + *result = rc;
> + else
> + *result = page_to_nid(newpage);
> + }
> + return rc;
> +}
> +
> /* putback_lru_page() counterpart for a ballooned page */
> void balloon_page_putback(struct page *page)
> {
> @@ -271,31 +222,4 @@ void balloon_page_putback(struct page *page)
> unlock_page(page);
> }
>
> -/* move_to_new_page() counterpart for a ballooned page */
> -int balloon_page_migrate(struct page *newpage,
> - struct page *page, enum migrate_mode mode)
> -{
> - struct address_space *mapping;
> - int rc = -EAGAIN;
> -
> - /*
> - * Block others from accessing the 'newpage' when we get around to
> - * establishing additional references. We should be the only one
> - * holding a reference to the 'newpage' at this point.
> - */
> - BUG_ON(!trylock_page(newpage));
> -
> - if (WARN_ON(!PageBalloon(page))) {
> - dump_page(page, "not movable balloon page");
> - unlock_page(newpage);
> - return rc;
> - }
> -
> - mapping = page->mapping;
> - if (mapping)
> - rc = __migrate_balloon_page(mapping, newpage, page, mode);
> -
> - unlock_page(newpage);
> - return rc;
> -}
> #endif /* CONFIG_BALLOON_COMPACTION */
> diff --git a/mm/migrate.c b/mm/migrate.c
> index a4939b1..e6d0d2d 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> }
> }
>
> - if (unlikely(PageBalloon(page))) {
> - /*
> - * A ballooned page does not need any special attention from
> - * physical to virtual reverse mapping procedures.
> - * Skip any attempt to unmap PTEs or to remap swap cache,
> - * in order to avoid burning cycles at rmap level, and perform
> - * the page migration right away (proteced by page lock).
> - */
> - rc = balloon_page_migrate(newpage, page, mode);
> - goto out_unlock;
> - }
> -
> /*
> * Corner case handling:
> * 1. When a new swap-cache page is read into, it is added to the LRU
> @@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,
>
> rc = __unmap_and_move(page, newpage, force, mode);
>
> -#ifdef CONFIG_MEMORY_BALLOON
> - if (unlikely(rc == MIGRATEPAGE_BALLOON_SUCCESS)) {
> - /*
> - * A ballooned page has been migrated already.
> - * Now, it's the time to wrap-up counters,
> - * handle the page back to Buddy and return.
> - */
> - dec_zone_page_state(page, NR_ISOLATED_ANON +
> - page_is_file_cache(page));
> - balloon_page_free(page);
> - return MIGRATEPAGE_SUCCESS;
> - }
> -#endif
> out:
> if (rc != -EAGAIN) {
> /*
> @@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
> rc = unmap_and_move_huge_page(get_new_page,
> put_new_page, private, page,
> pass > 2, mode);
> + else if (PageBalloon(page))
> + rc = balloon_page_migrate(get_new_page,
> + put_new_page, private,
> + page, pass > 2, mode);
> else
> rc = unmap_and_move(get_new_page, put_new_page,
> private, page, pass > 2, mode);
>
Acked-by: Rafael Aquini <aquini@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-12 23:51 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-12 23:51 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
> state of page->_mapcount counter. There is no conflict because ballooned pages
> cannot be mapped and cannot be in buddy allocator.
>
> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
> userspace via /proc/kpageflags as KPF_BALLOON.
>
> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
> ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a
pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
changes were not changelogged?
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Consuming another page flag is a big deal. We keep on nearly running
out and one day we'll run out for real. page-flags-layout.h is
incomprehensible. How many flags do we have left (worst-case) with this
change? Is there no other way? Needs extraordinary justification,
please.
drivers/virtio/Kconfig | 1 -
include/linux/mm.h | 14 ++++++++++++--
mm/Makefile | 3 +--
mm/balloon_compaction.c | 16 ----------------
4 files changed, 13 insertions(+), 21 deletions(-)
diff -puN drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 drivers/virtio/Kconfig
--- a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/drivers/virtio/Kconfig
@@ -25,7 +25,6 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
- select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff -puN include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 include/linux/mm.h
--- a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/include/linux/mm.h
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
}
-void __SetPageBalloon(struct page *page);
-void __ClearPageBalloon(struct page *page);
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
+ atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(&page->_mapcount, -1);
+}
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff -puN mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/Makefile
--- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o vmacache.o \
+ compaction.o balloon_compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
-obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff -puN mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/balloon_compaction.c
--- a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/balloon_compaction.c
@@ -10,22 +10,6 @@
#include <linux/export.h>
#include <linux/balloon_compaction.h>
-void __SetPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
- atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__SetPageBalloon);
-
-void __ClearPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(!PageBalloon(page), page);
- atomic_set(&page->_mapcount, -1);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__ClearPageBalloon);
-
/*
* balloon_devinfo_alloc - allocates a balloon device information descriptor.
* @balloon_dev_descriptor: pointer to reference the balloon device which
_
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-12 23:51 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-12 23:51 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
> state of page->_mapcount counter. There is no conflict because ballooned pages
> cannot be mapped and cannot be in buddy allocator.
>
> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
> userspace via /proc/kpageflags as KPF_BALLOON.
>
> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
> ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a
pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
changes were not changelogged?
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Consuming another page flag is a big deal. We keep on nearly running
out and one day we'll run out for real. page-flags-layout.h is
incomprehensible. How many flags do we have left (worst-case) with this
change? Is there no other way? Needs extraordinary justification,
please.
drivers/virtio/Kconfig | 1 -
include/linux/mm.h | 14 ++++++++++++--
mm/Makefile | 3 +--
mm/balloon_compaction.c | 16 ----------------
4 files changed, 13 insertions(+), 21 deletions(-)
diff -puN drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 drivers/virtio/Kconfig
--- a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/drivers/virtio/Kconfig
@@ -25,7 +25,6 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
- select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff -puN include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 include/linux/mm.h
--- a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/include/linux/mm.h
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
}
-void __SetPageBalloon(struct page *page);
-void __ClearPageBalloon(struct page *page);
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
+ atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(&page->_mapcount, -1);
+}
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff -puN mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/Makefile
--- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o vmacache.o \
+ compaction.o balloon_compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
-obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff -puN mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/balloon_compaction.c
--- a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/balloon_compaction.c
@@ -10,22 +10,6 @@
#include <linux/export.h>
#include <linux/balloon_compaction.h>
-void __SetPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
- atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__SetPageBalloon);
-
-void __ClearPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(!PageBalloon(page), page);
- atomic_set(&page->_mapcount, -1);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__ClearPageBalloon);
-
/*
* balloon_devinfo_alloc - allocates a balloon device information descriptor.
* @balloon_dev_descriptor: pointer to reference the balloon device which
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-12 23:57 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-12 23:57 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
> with PageBalloon which is stored directly in the struct page.
> All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
argh, now the bogus Kconfig/Makefile changes get removed again.
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
@ 2014-09-12 23:57 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-12 23:57 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> This patch replaces checking AS_BALLOON_MAP in page->mapping->flags
> with PageBalloon which is stored directly in the struct page.
> All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
argh, now the bogus Kconfig/Makefile changes get removed again.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-13 0:04 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:04 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> * move special branch for balloon migraion into migrate_pages
> * remove special mapping for balloon and its flag AS_BALLOON_MAP
> * embed struct balloon_dev_info into struct virtio_balloon
> * cleanup balloon_page_dequeue, kill balloon_page_free
Not sure what's going on here - your include/linux/balloon_compaction.h
seems significantly different from mine.
I think I'll just drop this patch - it's quite inconvenient to have a
large "general cleanup" coming after a stack of significant functional
changes. It makes review, debug, fix, merge and reversion harder.
Let's worry about it later.
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
@ 2014-09-13 0:04 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:04 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> * move special branch for balloon migraion into migrate_pages
> * remove special mapping for balloon and its flag AS_BALLOON_MAP
> * embed struct balloon_dev_info into struct virtio_balloon
> * cleanup balloon_page_dequeue, kill balloon_page_free
Not sure what's going on here - your include/linux/balloon_compaction.h
seems significantly different from mine.
I think I'll just drop this patch - it's quite inconvenient to have a
large "general cleanup" coming after a stack of significant functional
changes. It makes review, debug, fix, merge and reversion harder.
Let's worry about it later.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
2014-09-13 0:04 ` Andrew Morton
@ 2014-09-13 0:06 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:06 UTC (permalink / raw)
To: Konstantin Khlebnikov, Konstantin Khlebnikov, Rafael Aquini,
Linux Kernel Mailing List, linux-mm, Andrey Ryabinin,
Sasha Levin
On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
> > * move special branch for balloon migraion into migrate_pages
> > * remove special mapping for balloon and its flag AS_BALLOON_MAP
> > * embed struct balloon_dev_info into struct virtio_balloon
> > * cleanup balloon_page_dequeue, kill balloon_page_free
>
> Not sure what's going on here - your include/linux/balloon_compaction.h
> seems significantly different from mine.
OK, I worked it out.
> I think I'll just drop this patch - it's quite inconvenient to have a
> large "general cleanup" coming after a stack of significant functional
> changes. It makes review, debug, fix, merge and reversion harder.
> Let's worry about it later.
But I'm still thinking we should defer this one?
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
@ 2014-09-13 0:06 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:06 UTC (permalink / raw)
To: Konstantin Khlebnikov, Konstantin Khlebnikov, Rafael Aquini,
Linux Kernel Mailing List, linux-mm, Andrey Ryabinin,
Sasha Levin
On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
> > * move special branch for balloon migraion into migrate_pages
> > * remove special mapping for balloon and its flag AS_BALLOON_MAP
> > * embed struct balloon_dev_info into struct virtio_balloon
> > * cleanup balloon_page_dequeue, kill balloon_page_free
>
> Not sure what's going on here - your include/linux/balloon_compaction.h
> seems significantly different from mine.
OK, I worked it out.
> I think I'll just drop this patch - it's quite inconvenient to have a
> large "general cleanup" coming after a stack of significant functional
> changes. It makes review, debug, fix, merge and reversion harder.
> Let's worry about it later.
But I'm still thinking we should defer this one?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
2014-08-30 16:41 ` Konstantin Khlebnikov
@ 2014-09-13 0:09 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:09 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
> defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
> For stable kernels first three patches should be enough.
>
> changes since v1:
>
> mm/balloon_compaction: ignore anonymous pages
> * no changes
>
> mm/balloon_compaction: keep ballooned pages away from normal migration path
> * fix compilation without CONFIG_BALLOON_COMPACTION
>
> mm/balloon_compaction: isolate balloon pages without lru_lock
> * no changes
>
> mm: introduce common page state for ballooned memory
> * move __Set/ClearPageBalloon into linux/mm.h
> * remove inc/dec_zone_page_state from __Set/ClearPageBalloon
>
> mm/balloon_compaction: use common page ballooning
> * call inc/dec_zone_page_state from balloon_page_insert/delete
>
> mm/balloon_compaction: general cleanup
> * fix compilation without CONFIG_MIGRATION
> * fix compilation without CONFIG_BALLOON_COMPACTION
>
The patch "selftests/vm/transhuge-stress: stress test for memory
compaction" has silently and mysteriously vanished?
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
@ 2014-09-13 0:09 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 0:09 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
> defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
> For stable kernels first three patches should be enough.
>
> changes since v1:
>
> mm/balloon_compaction: ignore anonymous pages
> * no changes
>
> mm/balloon_compaction: keep ballooned pages away from normal migration path
> * fix compilation without CONFIG_BALLOON_COMPACTION
>
> mm/balloon_compaction: isolate balloon pages without lru_lock
> * no changes
>
> mm: introduce common page state for ballooned memory
> * move __Set/ClearPageBalloon into linux/mm.h
> * remove inc/dec_zone_page_state from __Set/ClearPageBalloon
>
> mm/balloon_compaction: use common page ballooning
> * call inc/dec_zone_page_state from balloon_page_insert/delete
>
> mm/balloon_compaction: general cleanup
> * fix compilation without CONFIG_MIGRATION
> * fix compilation without CONFIG_BALLOON_COMPACTION
>
The patch "selftests/vm/transhuge-stress: stress test for memory
compaction" has silently and mysteriously vanished?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
2014-09-13 0:09 ` Andrew Morton
@ 2014-09-13 5:01 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:01 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 4:09 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
>> defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
>> For stable kernels first three patches should be enough.
>>
>> changes since v1:
>>
>> mm/balloon_compaction: ignore anonymous pages
>> * no changes
>>
>> mm/balloon_compaction: keep ballooned pages away from normal migration path
>> * fix compilation without CONFIG_BALLOON_COMPACTION
>>
>> mm/balloon_compaction: isolate balloon pages without lru_lock
>> * no changes
>>
>> mm: introduce common page state for ballooned memory
>> * move __Set/ClearPageBalloon into linux/mm.h
>> * remove inc/dec_zone_page_state from __Set/ClearPageBalloon
>>
>> mm/balloon_compaction: use common page ballooning
>> * call inc/dec_zone_page_state from balloon_page_insert/delete
>>
>> mm/balloon_compaction: general cleanup
>> * fix compilation without CONFIG_MIGRATION
>> * fix compilation without CONFIG_BALLOON_COMPACTION
>>
>
> The patch "selftests/vm/transhuge-stress: stress test for memory
> compaction" has silently and mysteriously vanished?
>
It's unchanged and has no direct connection to this patchset.
So I've dropped it. If you like it you can keep the v1 version.
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups
@ 2014-09-13 5:01 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:01 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 4:09 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> I've checked compilation of linux-next/x86 for allnoconfig, defconfig and
>> defconfig + kvmconfig + virtio-balloon with and without balloon-compaction.
>> For stable kernels first three patches should be enough.
>>
>> changes since v1:
>>
>> mm/balloon_compaction: ignore anonymous pages
>> * no changes
>>
>> mm/balloon_compaction: keep ballooned pages away from normal migration path
>> * fix compilation without CONFIG_BALLOON_COMPACTION
>>
>> mm/balloon_compaction: isolate balloon pages without lru_lock
>> * no changes
>>
>> mm: introduce common page state for ballooned memory
>> * move __Set/ClearPageBalloon into linux/mm.h
>> * remove inc/dec_zone_page_state from __Set/ClearPageBalloon
>>
>> mm/balloon_compaction: use common page ballooning
>> * call inc/dec_zone_page_state from balloon_page_insert/delete
>>
>> mm/balloon_compaction: general cleanup
>> * fix compilation without CONFIG_MIGRATION
>> * fix compilation without CONFIG_BALLOON_COMPACTION
>>
>
> The patch "selftests/vm/transhuge-stress: stress test for memory
> compaction" has silently and mysteriously vanished?
>
It's unchanged and has no direct connection to this patchset.
So I've dropped it. If you like it you can keep the v1 version.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-12 23:51 ` Andrew Morton
@ 2014-09-13 5:26 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:26 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 3:51 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
>> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
>> state of page->_mapcount counter. There is no conflict because ballooned pages
>> cannot be mapped and cannot be in buddy allocator.
>>
>> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
>> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
>> userspace via /proc/kpageflags as KPF_BALLOON.
>>
>> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
>> ballooning driver which wants use this feature.
>
> The delta from the (fixed) v1 is below.
>
> What's up with those Kconfig/Makefile changes? We're now including a
> pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
> changes were not changelogged?
It's been moved into next patch, and not mentioned in changes. Sorry.
>
> Did we really need to put the BalloonPages count into per-zone vmstat,
> global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
>
> Consuming another page flag is a big deal. We keep on nearly running
> out and one day we'll run out for real. page-flags-layout.h is
> incomprehensible. How many flags do we have left (worst-case) with this
> change? Is there no other way? Needs extraordinary justification,
> please.
PageBalloon is not a page flags, it's like PageBuddy -- special state
of _mapcount (-256 in this case).
The same was in v1 and is written in the comment above.
>
> drivers/virtio/Kconfig | 1 -
> include/linux/mm.h | 14 ++++++++++++--
> mm/Makefile | 3 +--
> mm/balloon_compaction.c | 16 ----------------
> 4 files changed, 13 insertions(+), 21 deletions(-)
>
> diff -puN drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 drivers/virtio/Kconfig
> --- a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/drivers/virtio/Kconfig
> @@ -25,7 +25,6 @@ config VIRTIO_PCI
> config VIRTIO_BALLOON
> tristate "Virtio balloon driver"
> depends on VIRTIO
> - select MEMORY_BALLOON
> ---help---
> This driver supports increasing and decreasing the amount
> of memory within a KVM guest.
> diff -puN include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 include/linux/mm.h
> --- a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/include/linux/mm.h
> @@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
> return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
> atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
> }
> -void __SetPageBalloon(struct page *page);
> -void __ClearPageBalloon(struct page *page);
> +
> +static inline void __SetPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> + atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> +}
> +
> +static inline void __ClearPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(!PageBalloon(page), page);
> + atomic_set(&page->_mapcount, -1);
> +}
>
> void put_page(struct page *page);
> void put_pages_list(struct list_head *pages);
> diff -puN mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/Makefile
> --- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
> readahead.o swap.o truncate.o vmscan.o shmem.o \
> util.o mmzone.o vmstat.o backing-dev.o \
> mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o vmacache.o \
> + compaction.o balloon_compaction.o vmacache.o \
> interval_tree.o list_lru.o workingset.o \
> iov_iter.o $(mmu-y)
>
> @@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> obj-$(CONFIG_CMA) += cma.o
> -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
> diff -puN mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/balloon_compaction.c
> --- a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/balloon_compaction.c
> @@ -10,22 +10,6 @@
> #include <linux/export.h>
> #include <linux/balloon_compaction.h>
>
> -void __SetPageBalloon(struct page *page)
> -{
> - VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> - atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> - inc_zone_page_state(page, NR_BALLOON_PAGES);
> -}
> -EXPORT_SYMBOL(__SetPageBalloon);
> -
> -void __ClearPageBalloon(struct page *page)
> -{
> - VM_BUG_ON_PAGE(!PageBalloon(page), page);
> - atomic_set(&page->_mapcount, -1);
> - dec_zone_page_state(page, NR_BALLOON_PAGES);
> -}
> -EXPORT_SYMBOL(__ClearPageBalloon);
> -
> /*
> * balloon_devinfo_alloc - allocates a balloon device information descriptor.
> * @balloon_dev_descriptor: pointer to reference the balloon device which
> _
>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-13 5:26 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:26 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 3:51 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
>> Like PageBuddy() PageBalloon() looks like page-flag but actually this is special
>> state of page->_mapcount counter. There is no conflict because ballooned pages
>> cannot be mapped and cannot be in buddy allocator.
>>
>> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them
>> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
>> userspace via /proc/kpageflags as KPF_BALLOON.
>>
>> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
>> ballooning driver which wants use this feature.
>
> The delta from the (fixed) v1 is below.
>
> What's up with those Kconfig/Makefile changes? We're now including a
> pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
> changes were not changelogged?
It's been moved into next patch, and not mentioned in changes. Sorry.
>
> Did we really need to put the BalloonPages count into per-zone vmstat,
> global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
>
> Consuming another page flag is a big deal. We keep on nearly running
> out and one day we'll run out for real. page-flags-layout.h is
> incomprehensible. How many flags do we have left (worst-case) with this
> change? Is there no other way? Needs extraordinary justification,
> please.
PageBalloon is not a page flags, it's like PageBuddy -- special state
of _mapcount (-256 in this case).
The same was in v1 and is written in the comment above.
>
> drivers/virtio/Kconfig | 1 -
> include/linux/mm.h | 14 ++++++++++++--
> mm/Makefile | 3 +--
> mm/balloon_compaction.c | 16 ----------------
> 4 files changed, 13 insertions(+), 21 deletions(-)
>
> diff -puN drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 drivers/virtio/Kconfig
> --- a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/drivers/virtio/Kconfig
> @@ -25,7 +25,6 @@ config VIRTIO_PCI
> config VIRTIO_BALLOON
> tristate "Virtio balloon driver"
> depends on VIRTIO
> - select MEMORY_BALLOON
> ---help---
> This driver supports increasing and decreasing the amount
> of memory within a KVM guest.
> diff -puN include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 include/linux/mm.h
> --- a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/include/linux/mm.h
> @@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
> return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
> atomic_read(&page->_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
> }
> -void __SetPageBalloon(struct page *page);
> -void __ClearPageBalloon(struct page *page);
> +
> +static inline void __SetPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> + atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> +}
> +
> +static inline void __ClearPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(!PageBalloon(page), page);
> + atomic_set(&page->_mapcount, -1);
> +}
>
> void put_page(struct page *page);
> void put_pages_list(struct list_head *pages);
> diff -puN mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/Makefile
> --- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
> readahead.o swap.o truncate.o vmscan.o shmem.o \
> util.o mmzone.o vmstat.o backing-dev.o \
> mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o vmacache.o \
> + compaction.o balloon_compaction.o vmacache.o \
> interval_tree.o list_lru.o workingset.o \
> iov_iter.o $(mmu-y)
>
> @@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> obj-$(CONFIG_CMA) += cma.o
> -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
> diff -puN mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 mm/balloon_compaction.c
> --- a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/balloon_compaction.c
> @@ -10,22 +10,6 @@
> #include <linux/export.h>
> #include <linux/balloon_compaction.h>
>
> -void __SetPageBalloon(struct page *page)
> -{
> - VM_BUG_ON_PAGE(atomic_read(&page->_mapcount) != -1, page);
> - atomic_set(&page->_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> - inc_zone_page_state(page, NR_BALLOON_PAGES);
> -}
> -EXPORT_SYMBOL(__SetPageBalloon);
> -
> -void __ClearPageBalloon(struct page *page)
> -{
> - VM_BUG_ON_PAGE(!PageBalloon(page), page);
> - atomic_set(&page->_mapcount, -1);
> - dec_zone_page_state(page, NR_BALLOON_PAGES);
> -}
> -EXPORT_SYMBOL(__ClearPageBalloon);
> -
> /*
> * balloon_devinfo_alloc - allocates a balloon device information descriptor.
> * @balloon_dev_descriptor: pointer to reference the balloon device which
> _
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-13 5:26 ` Konstantin Khlebnikov
@ 2014-09-13 5:42 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 5:42 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> > Did we really need to put the BalloonPages count into per-zone vmstat,
> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> > important?
>
> Balloon grabs random pages, their distribution among numa nodes might
> be important.
> But I know nobody who uses numa-aware vm together with ballooning.
>
> Probably it's better to drop per-zone vmstat and line from meminfo,
> global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if
there is a demonstrated need.
> >
> > Consuming another page flag is a big deal. We keep on nearly running
> > out and one day we'll run out for real. page-flags-layout.h is
> > incomprehensible. How many flags do we have left (worst-case) with this
> > change? Is there no other way? Needs extraordinary justification,
> > please.
>
> PageBalloon is not a page flags, it's like PageBuddy -- special state
> of _mapcount (-256 in this case).
> The same was in v1 and is written in the comment above.
oop sorry, I got confused about KPF_BALLOON.
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-13 5:42 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-13 5:42 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> > Did we really need to put the BalloonPages count into per-zone vmstat,
> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> > important?
>
> Balloon grabs random pages, their distribution among numa nodes might
> be important.
> But I know nobody who uses numa-aware vm together with ballooning.
>
> Probably it's better to drop per-zone vmstat and line from meminfo,
> global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if
there is a demonstrated need.
> >
> > Consuming another page flag is a big deal. We keep on nearly running
> > out and one day we'll run out for real. page-flags-layout.h is
> > incomprehensible. How many flags do we have left (worst-case) with this
> > change? Is there no other way? Needs extraordinary justification,
> > please.
>
> PageBalloon is not a page flags, it's like PageBuddy -- special state
> of _mapcount (-256 in this case).
> The same was in v1 and is written in the comment above.
oop sorry, I got confused about KPF_BALLOON.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
2014-09-13 0:06 ` Andrew Morton
@ 2014-09-13 5:43 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:43 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 4:06 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
>> On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>>
>> > * move special branch for balloon migraion into migrate_pages
>> > * remove special mapping for balloon and its flag AS_BALLOON_MAP
>> > * embed struct balloon_dev_info into struct virtio_balloon
>> > * cleanup balloon_page_dequeue, kill balloon_page_free
>>
>> Not sure what's going on here - your include/linux/balloon_compaction.h
>> seems significantly different from mine.
>
> OK, I worked it out.
>
>> I think I'll just drop this patch - it's quite inconvenient to have a
>> large "general cleanup" coming after a stack of significant functional
>> changes. It makes review, debug, fix, merge and reversion harder.
>> Let's worry about it later.
>
> But I'm still thinking we should defer this one?
It seems in this case massive cleanup before fixies leads to much bigger mess.
I've separated fixes specially for merging into stable kernels.
The rest patches mostly rewrites this code using new approatch.
I don't know how to make it more reviewable, it's easier to write a
new version from the scratch.
Probably rework should be untied form fixes and sent as separate patchset.
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
@ 2014-09-13 5:43 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 5:43 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 13, 2014 at 4:06 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
>> On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>>
>> > * move special branch for balloon migraion into migrate_pages
>> > * remove special mapping for balloon and its flag AS_BALLOON_MAP
>> > * embed struct balloon_dev_info into struct virtio_balloon
>> > * cleanup balloon_page_dequeue, kill balloon_page_free
>>
>> Not sure what's going on here - your include/linux/balloon_compaction.h
>> seems significantly different from mine.
>
> OK, I worked it out.
>
>> I think I'll just drop this patch - it's quite inconvenient to have a
>> large "general cleanup" coming after a stack of significant functional
>> changes. It makes review, debug, fix, merge and reversion harder.
>> Let's worry about it later.
>
> But I'm still thinking we should defer this one?
It seems in this case massive cleanup before fixies leads to much bigger mess.
I've separated fixes specially for merging into stable kernels.
The rest patches mostly rewrites this code using new approatch.
I don't know how to make it more reviewable, it's easier to write a
new version from the scratch.
Probably rework should be untied form fixes and sent as separate patchset.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-13 5:42 ` Andrew Morton
(?)
@ 2014-09-13 8:22 ` Konstantin Khlebnikov
2014-09-19 21:35 ` Andrew Morton
-1 siblings, 1 reply; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-13 8:22 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]
On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> >
>> > Did we really need to put the BalloonPages count into per-zone vmstat,
>> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> > important?
>>
>> Balloon grabs random pages, their distribution among numa nodes might
>> be important.
>> But I know nobody who uses numa-aware vm together with ballooning.
>>
>> Probably it's better to drop per-zone vmstat and line from meminfo,
>> global vmstat counter should be enough.
>
> Yes, the less we add the better - we can always add stuff later if
> there is a demonstrated need.
Ok. (I guess incremental patches are more convenient for you)
Here is two fixes which remove redundant per-zone counters and adds
three vmstat counters: "balloon_inflate", "balloon_deflate" and
"balloon_migrate".
This statistic seems more useful than current state snapshot.
Size of balloon is just a difference between "inflate" and "deflate".
>
>> >
>> > Consuming another page flag is a big deal. We keep on nearly running
>> > out and one day we'll run out for real. page-flags-layout.h is
>> > incomprehensible. How many flags do we have left (worst-case) with this
>> > change? Is there no other way? Needs extraordinary justification,
>> > please.
>>
>> PageBalloon is not a page flags, it's like PageBuddy -- special state
>> of _mapcount (-256 in this case).
>> The same was in v1 and is written in the comment above.
>
> oop sorry, I got confused about KPF_BALLOON.
[-- Attachment #2: mm-balloon_compaction-use-common-page-ballooning-use-vmstat-counters --]
[-- Type: application/octet-stream, Size: 2701 bytes --]
mm-balloon_compaction-use-common-page-ballooning-use-vmstat-counters
From: Konstantin Khlebnikov <koct9i@gmail.com>
fix for mm-balloon_compaction-use-common-page-ballooning-v2
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
---
drivers/virtio/virtio_balloon.c | 1 +
include/linux/balloon_compaction.h | 2 --
mm/balloon_compaction.c | 2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index c84d6a8..6b3444b 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
index ad112fcc6..d69f2ae 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(&page->lru, &b_dev_info->pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(&page->lru);
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 3c8cb7a..c536350 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)
BUG_ON(!trylock_page(page));
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_insert(b_dev_info, page);
+ __count_vm_event(BALLOON_INFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
if (trylock_page(page)) {
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_delete(page, false);
+ __count_vm_event(BALLOON_DEFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
[-- Attachment #3: mm-introduce-common-page-state-for-ballooned-memory-vmstat-counters --]
[-- Type: application/octet-stream, Size: 5083 bytes --]
mm-introduce-common-page-state-for-ballooned-memory-vmstat-counters
From: Konstantin Khlebnikov <koct9i@gmail.com>
fix for mm-introduce-common-page-state-for-ballooned-memory-fix-v2
This reverts per-zone balloon counters and removes them from meminfo, zoneinfo.
Instead of that this patch adds three /proc/vmstat counters:
"balloon_inflate", "balloon_deflate" and "balloon_migrate".
Current size of balloon is (balloon_inflate - balloon_deflate) pages.
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
---
Documentation/filesystems/proc.txt | 2 --
drivers/base/node.c | 6 ------
fs/proc/meminfo.c | 6 ------
include/linux/mmzone.h | 3 ---
include/linux/vm_event_item.h | 7 +++++++
mm/vmstat.c | 10 +++++++---
6 files changed, 14 insertions(+), 20 deletions(-)
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index 154a345..eb8a10e 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -796,7 +796,6 @@ VmallocTotal: 112216 kB
VmallocUsed: 428 kB
VmallocChunk: 111088 kB
AnonHugePages: 49152 kB
-BalloonPages: 0 kB
MemTotal: Total usable ram (i.e. physical ram minus a few reserved
bits and the kernel binary code)
@@ -839,7 +838,6 @@ MemAvailable: An estimate of how much memory is available for starting new
Writeback: Memory which is actively being written back to the disk
AnonPages: Non-file backed pages mapped into userspace page tables
AnonHugePages: Non-file backed huge pages mapped into userspace page tables
-BalloonPages: Memory which was ballooned, not included into MemTotal
Mapped: files which have been mmaped, such as libraries
Slab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as caches
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 1f8d8b1..f0cdb02 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -120,9 +120,6 @@ static ssize_t node_read_meminfo(struct device *dev,
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"Node %d AnonHugePages: %8lu kB\n"
#endif
-#ifdef CONFIG_MEMORY_BALLOON
- "Node %d BalloonPages: %8lu kB\n"
-#endif
,
nid, K(node_page_state(nid, NR_FILE_DIRTY)),
nid, K(node_page_state(nid, NR_WRITEBACK)),
@@ -144,9 +141,6 @@ static ssize_t node_read_meminfo(struct device *dev,
,nid, K(node_page_state(nid,
NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)
#endif
-#ifdef CONFIG_MEMORY_BALLOON
- ,nid, K(node_page_state(nid, NR_BALLOON_PAGES))
-#endif
);
n += hugetlb_report_node_meminfo(nid, buf + n);
return n;
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index f897fbf..aa1eee0 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -138,9 +138,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
"AnonHugePages: %8lu kB\n"
#endif
-#ifdef CONFIG_MEMORY_BALLOON
- "BalloonPages: %8lu kB\n"
-#endif
,
K(i.totalram),
K(i.freeram),
@@ -196,9 +193,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) *
HPAGE_PMD_NR)
#endif
-#ifdef CONFIG_MEMORY_BALLOON
- ,K(global_page_state(NR_BALLOON_PAGES))
-#endif
);
hugetlb_report_meminfo(m);
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index fbbff5c..48bf12e 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -157,9 +157,6 @@ enum zone_stat_item {
WORKINGSET_NODERECLAIM,
NR_ANON_TRANSPARENT_HUGEPAGES,
NR_FREE_CMA_PAGES,
-#ifdef CONFIG_MEMORY_BALLOON
- NR_BALLOON_PAGES,
-#endif
NR_VM_ZONE_STAT_ITEMS };
/*
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index ced9234..730334c 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -72,6 +72,13 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
THP_ZERO_PAGE_ALLOC,
THP_ZERO_PAGE_ALLOC_FAILED,
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ BALLOON_INFLATE,
+ BALLOON_DEFLATE,
+#ifdef CONFIG_BALLOON_COMPACTION
+ BALLOON_MIGRATE,
+#endif
+#endif
#ifdef CONFIG_DEBUG_TLBFLUSH
#ifdef CONFIG_SMP
NR_TLB_REMOTE_FLUSH, /* cpu tried to flush others' tlbs */
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 87273cd..5da8834 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -794,9 +794,6 @@ const char * const vmstat_text[] = {
"workingset_nodereclaim",
"nr_anon_transparent_hugepages",
"nr_free_cma",
-#ifdef CONFIG_MEMORY_BALLOON
- "nr_balloon_pages",
-#endif
/* enum writeback_stat_item counters */
"nr_dirty_threshold",
@@ -882,6 +879,13 @@ const char * const vmstat_text[] = {
"thp_zero_page_alloc",
"thp_zero_page_alloc_failed",
#endif
+#ifdef CONFIG_MEMORY_BALLOON
+ "balloon_inflate",
+ "balloon_deflate",
+#ifdef CONFIG_BALLOON_COMPACTION
+ "balloon_migrate",
+#endif
+#endif /* CONFIG_MEMORY_BALLOON */
#ifdef CONFIG_DEBUG_TLBFLUSH
#ifdef CONFIG_SMP
"nr_tlb_remote_flush",
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-13 5:26 ` Konstantin Khlebnikov
@ 2014-09-13 14:03 ` Sasha Levin
-1 siblings, 0 replies; 63+ messages in thread
From: Sasha Levin @ 2014-09-13 14:03 UTC (permalink / raw)
To: Konstantin Khlebnikov, Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin
On 09/13/2014 01:26 AM, Konstantin Khlebnikov wrote:
>> Did we really need to put the BalloonPages count into per-zone vmstat,
>> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> > important?
> Balloon grabs random pages, their distribution among numa nodes might
> be important.
> But I know nobody who uses numa-aware vm together with ballooning.
*cough*. me?
Obviously there's no need to keep that per-zone vmstat just for me, but
right now NUMA on KVM works just fine and does a good job of catching
NUMA issues without having to run on actual metal.
Thanks,
Sasha
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-13 14:03 ` Sasha Levin
0 siblings, 0 replies; 63+ messages in thread
From: Sasha Levin @ 2014-09-13 14:03 UTC (permalink / raw)
To: Konstantin Khlebnikov, Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin
On 09/13/2014 01:26 AM, Konstantin Khlebnikov wrote:
>> Did we really need to put the BalloonPages count into per-zone vmstat,
>> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> > important?
> Balloon grabs random pages, their distribution among numa nodes might
> be important.
> But I know nobody who uses numa-aware vm together with ballooning.
*cough*. me?
Obviously there's no need to keep that per-zone vmstat just for me, but
right now NUMA on KVM works just fine and does a good job of catching
NUMA issues without having to run on actual metal.
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-13 8:22 ` Konstantin Khlebnikov
@ 2014-09-19 21:35 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-19 21:35 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
> <akpm@linux-foundation.org> wrote:
> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> >> >
> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> >> > important?
> >>
> >> Balloon grabs random pages, their distribution among numa nodes might
> >> be important.
> >> But I know nobody who uses numa-aware vm together with ballooning.
> >>
> >> Probably it's better to drop per-zone vmstat and line from meminfo,
> >> global vmstat counter should be enough.
> >
> > Yes, the less we add the better - we can always add stuff later if
> > there is a demonstrated need.
>
> Ok. (I guess incremental patches are more convenient for you)
> Here is two fixes which remove redundant per-zone counters and adds
> three vmstat counters: "balloon_inflate", "balloon_deflate" and
> "balloon_migrate".
>
> This statistic seems more useful than current state snapshot.
> Size of balloon is just a difference between "inflate" and "deflate".
This is a complete mess.
Your second patch (which is actually the first one) called "fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
has a changelog.
I eventually worked out that your first patch (whcih is actually the
second one) called "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
mm-balloon_compaction-general-cleanup.patch has been applied. Does it
actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
I can't tell, because the "general cleanup" is in the way.
So I'm going to send "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
separately, but it has no changelog at all.
Please always write changelogs. Please never send more than one patch
in a single email. Please be *consistent* in filenames, patch titles,
email subjects, etc.
Please send me a changelog for the below patch.
From: Konstantin Khlebnikov <koct9i@gmail.com>
Subject: mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
drivers/virtio/virtio_balloon.c | 1 +
include/linux/balloon_compaction.h | 2 --
mm/balloon_compaction.c | 2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff -puN drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 drivers/virtio/virtio_balloon.c
--- a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff -puN include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 include/linux/balloon_compaction.h
--- a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(&page->lru, &b_dev_info->pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(&page->lru);
diff -puN mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 mm/balloon_compaction.c
--- a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/mm/balloon_compaction.c
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
BUG_ON(!trylock_page(page));
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_insert(b_dev_info, page);
+ __count_vm_event(BALLOON_INFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
if (trylock_page(page)) {
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_delete(page, false);
+ __count_vm_event(BALLOON_DEFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
_
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-19 21:35 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-19 21:35 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
> <akpm@linux-foundation.org> wrote:
> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> >> >
> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> >> > important?
> >>
> >> Balloon grabs random pages, their distribution among numa nodes might
> >> be important.
> >> But I know nobody who uses numa-aware vm together with ballooning.
> >>
> >> Probably it's better to drop per-zone vmstat and line from meminfo,
> >> global vmstat counter should be enough.
> >
> > Yes, the less we add the better - we can always add stuff later if
> > there is a demonstrated need.
>
> Ok. (I guess incremental patches are more convenient for you)
> Here is two fixes which remove redundant per-zone counters and adds
> three vmstat counters: "balloon_inflate", "balloon_deflate" and
> "balloon_migrate".
>
> This statistic seems more useful than current state snapshot.
> Size of balloon is just a difference between "inflate" and "deflate".
This is a complete mess.
Your second patch (which is actually the first one) called "fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
has a changelog.
I eventually worked out that your first patch (whcih is actually the
second one) called "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
mm-balloon_compaction-general-cleanup.patch has been applied. Does it
actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
I can't tell, because the "general cleanup" is in the way.
So I'm going to send "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
separately, but it has no changelog at all.
Please always write changelogs. Please never send more than one patch
in a single email. Please be *consistent* in filenames, patch titles,
email subjects, etc.
Please send me a changelog for the below patch.
From: Konstantin Khlebnikov <koct9i@gmail.com>
Subject: mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
drivers/virtio/virtio_balloon.c | 1 +
include/linux/balloon_compaction.h | 2 --
mm/balloon_compaction.c | 2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff -puN drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 drivers/virtio/virtio_balloon.c
--- a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff -puN include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 include/linux/balloon_compaction.h
--- a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(&page->lru, &b_dev_info->pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(&page->lru);
diff -puN mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 mm/balloon_compaction.c
--- a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/mm/balloon_compaction.c
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
BUG_ON(!trylock_page(page));
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_insert(b_dev_info, page);
+ __count_vm_event(BALLOON_INFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
if (trylock_page(page)) {
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
balloon_page_delete(page, false);
+ __count_vm_event(BALLOON_DEFLATE);
spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
unlock_page(page);
return page;
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-19 21:35 ` Andrew Morton
@ 2014-09-20 5:25 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-20 5:25 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 20, 2014 at 1:35 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
>> <akpm@linux-foundation.org> wrote:
>> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >
>> >> >
>> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
>> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> >> > important?
>> >>
>> >> Balloon grabs random pages, their distribution among numa nodes might
>> >> be important.
>> >> But I know nobody who uses numa-aware vm together with ballooning.
>> >>
>> >> Probably it's better to drop per-zone vmstat and line from meminfo,
>> >> global vmstat counter should be enough.
>> >
>> > Yes, the less we add the better - we can always add stuff later if
>> > there is a demonstrated need.
>>
>> Ok. (I guess incremental patches are more convenient for you)
>> Here is two fixes which remove redundant per-zone counters and adds
>> three vmstat counters: "balloon_inflate", "balloon_deflate" and
>> "balloon_migrate".
>>
>> This statistic seems more useful than current state snapshot.
>> Size of balloon is just a difference between "inflate" and "deflate".
>
> This is a complete mess.
nod
>
> Your second patch (which is actually the first one) called "fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
> fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
> has a changelog.
>
> I eventually worked out that your first patch (whcih is actually the
> second one) called "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
> mm-balloon_compaction-general-cleanup.patch has been applied. Does it
> actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
> I can't tell, because the "general cleanup" is in the way.
Oops I did it again.
>
> So I'm going to send "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
>
> Please always write changelogs. Please never send more than one patch
> in a single email. Please be *consistent* in filenames, patch titles,
> email subjects, etc.
That patch is supposed to be merged into patch as a fix.
>
> Please send me a changelog for the below patch.
Ok, sure.
>
From: Konstantin Khlebnikov <koct9i@gmail.com>
Subject: mm/balloon_compaction: use vmstat counters
This is fix for "mm/balloon_compaction: use common page ballooning".
it reverts per-zone balloon counters and replaces them with vmstat counters:
"balloon_inflate", "balloon_deflate" and "balloon_migrate".
Per-zone balloon counters have been reverted after discussion but reverting
them from balloon_compaction conflicts with massive cleanup in this code.
Thus this change ends up as a separate patch. Sorry for the mess.
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
> drivers/virtio/virtio_balloon.c | 1 +
> include/linux/balloon_compaction.h | 2 --
> mm/balloon_compaction.c | 2 ++
> 3 files changed, 3 insertions(+), 2 deletions(-)
>
> diff -puN drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 drivers/virtio/virtio_balloon.c
> --- a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/drivers/virtio/virtio_balloon.c
> @@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
> /* balloon's page migration 1st step -- inflate "newpage" */
> spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
> balloon_page_insert(vb_dev_info, newpage);
> + __count_vm_event(BALLOON_MIGRATE);
> vb_dev_info->isolated_pages--;
> spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> diff -puN include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 include/linux/balloon_compaction.h
> --- a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/include/linux/balloon_compaction.h
> @@ -87,7 +87,6 @@ static inline void
> balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
> {
> __SetPageBalloon(page);
> - inc_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, (unsigned long)b_dev_info);
> list_add(&page->lru, &b_dev_info->pages);
> }
> @@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
> static inline void balloon_page_delete(struct page *page, bool isolated)
> {
> __ClearPageBalloon(page);
> - dec_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, 0);
> if (!isolated)
> list_del(&page->lru);
> diff -puN mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 mm/balloon_compaction.c
> --- a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/mm/balloon_compaction.c
> @@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
> BUG_ON(!trylock_page(page));
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> balloon_page_insert(b_dev_info, page);
> + __count_vm_event(BALLOON_INFLATE);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> @@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
> if (trylock_page(page)) {
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> balloon_page_delete(page, false);
> + __count_vm_event(BALLOON_DEFLATE);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> _
>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-20 5:25 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-20 5:25 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 20, 2014 at 1:35 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
>> <akpm@linux-foundation.org> wrote:
>> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >
>> >> >
>> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
>> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> >> > important?
>> >>
>> >> Balloon grabs random pages, their distribution among numa nodes might
>> >> be important.
>> >> But I know nobody who uses numa-aware vm together with ballooning.
>> >>
>> >> Probably it's better to drop per-zone vmstat and line from meminfo,
>> >> global vmstat counter should be enough.
>> >
>> > Yes, the less we add the better - we can always add stuff later if
>> > there is a demonstrated need.
>>
>> Ok. (I guess incremental patches are more convenient for you)
>> Here is two fixes which remove redundant per-zone counters and adds
>> three vmstat counters: "balloon_inflate", "balloon_deflate" and
>> "balloon_migrate".
>>
>> This statistic seems more useful than current state snapshot.
>> Size of balloon is just a difference between "inflate" and "deflate".
>
> This is a complete mess.
nod
>
> Your second patch (which is actually the first one) called "fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
> fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
> has a changelog.
>
> I eventually worked out that your first patch (whcih is actually the
> second one) called "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
> mm-balloon_compaction-general-cleanup.patch has been applied. Does it
> actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
> I can't tell, because the "general cleanup" is in the way.
Oops I did it again.
>
> So I'm going to send "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
>
> Please always write changelogs. Please never send more than one patch
> in a single email. Please be *consistent* in filenames, patch titles,
> email subjects, etc.
That patch is supposed to be merged into patch as a fix.
>
> Please send me a changelog for the below patch.
Ok, sure.
>
From: Konstantin Khlebnikov <koct9i@gmail.com>
Subject: mm/balloon_compaction: use vmstat counters
This is fix for "mm/balloon_compaction: use common page ballooning".
it reverts per-zone balloon counters and replaces them with vmstat counters:
"balloon_inflate", "balloon_deflate" and "balloon_migrate".
Per-zone balloon counters have been reverted after discussion but reverting
them from balloon_compaction conflicts with massive cleanup in this code.
Thus this change ends up as a separate patch. Sorry for the mess.
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
> drivers/virtio/virtio_balloon.c | 1 +
> include/linux/balloon_compaction.h | 2 --
> mm/balloon_compaction.c | 2 ++
> 3 files changed, 3 insertions(+), 2 deletions(-)
>
> diff -puN drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 drivers/virtio/virtio_balloon.c
> --- a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/drivers/virtio/virtio_balloon.c
> @@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
> /* balloon's page migration 1st step -- inflate "newpage" */
> spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
> balloon_page_insert(vb_dev_info, newpage);
> + __count_vm_event(BALLOON_MIGRATE);
> vb_dev_info->isolated_pages--;
> spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> diff -puN include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 include/linux/balloon_compaction.h
> --- a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/include/linux/balloon_compaction.h
> @@ -87,7 +87,6 @@ static inline void
> balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
> {
> __SetPageBalloon(page);
> - inc_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, (unsigned long)b_dev_info);
> list_add(&page->lru, &b_dev_info->pages);
> }
> @@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
> static inline void balloon_page_delete(struct page *page, bool isolated)
> {
> __ClearPageBalloon(page);
> - dec_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, 0);
> if (!isolated)
> list_del(&page->lru);
> diff -puN mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 mm/balloon_compaction.c
> --- a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/mm/balloon_compaction.c
> @@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
> BUG_ON(!trylock_page(page));
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> balloon_page_insert(b_dev_info, page);
> + __count_vm_event(BALLOON_INFLATE);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> @@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
> if (trylock_page(page)) {
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> balloon_page_delete(page, false);
> + __count_vm_event(BALLOON_DEFLATE);
> spin_unlock_irqrestore(&b_dev_info->pages_lock, flags);
> unlock_page(page);
> return page;
> _
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-20 5:25 ` Konstantin Khlebnikov
@ 2014-09-20 6:23 ` Andrew Morton
-1 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-20 6:23 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> > So I'm going to send "fix for
> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> > separately, but it has no changelog at all.
>
> Probably it would be better if you drop everything except actually
> fixes and stresstest. This is gone too far, now balloon won't compile
> in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-20 6:23 ` Andrew Morton
0 siblings, 0 replies; 63+ messages in thread
From: Andrew Morton @ 2014-09-20 6:23 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> > So I'm going to send "fix for
> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> > separately, but it has no changelog at all.
>
> Probably it would be better if you drop everything except actually
> fixes and stresstest. This is gone too far, now balloon won't compile
> in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-20 6:23 ` Andrew Morton
@ 2014-09-22 18:40 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 18:40 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> >
>> > So I'm going to send "fix for
>> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> > separately, but it has no changelog at all.
>>
>> Probably it would be better if you drop everything except actually
>> fixes and stresstest. This is gone too far, now balloon won't compile
>> in the middle of patchset. Just tell me and I'll redo the rest.
>
> I think it's best if I drop everything:
>
> mm-balloon_compaction-ignore-anonymous-pages.patch
> mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> mm-introduce-common-page-state-for-ballooned-memory.patch
> mm-balloon_compaction-use-common-page-ballooning.patch
> mm-balloon_compaction-general-cleanup.patch
> mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>
> Please go through it and send out a new version?
>
>
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Probably it's really easier to rewrite it than to fix bugs one by one =/
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-22 18:40 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 18:40 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov, Rafael Aquini, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>
>> >
>> > So I'm going to send "fix for
>> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> > separately, but it has no changelog at all.
>>
>> Probably it would be better if you drop everything except actually
>> fixes and stresstest. This is gone too far, now balloon won't compile
>> in the middle of patchset. Just tell me and I'll redo the rest.
>
> I think it's best if I drop everything:
>
> mm-balloon_compaction-ignore-anonymous-pages.patch
> mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> mm-introduce-common-page-state-for-ballooned-memory.patch
> mm-balloon_compaction-use-common-page-ballooning.patch
> mm-balloon_compaction-general-cleanup.patch
> mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>
> Please go through it and send out a new version?
>
>
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Probably it's really easier to rewrite it than to fix bugs one by one =/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-22 18:40 ` Konstantin Khlebnikov
@ 2014-09-22 19:22 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-22 19:22 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> <akpm@linux-foundation.org> wrote:
> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> >> >
> >> > So I'm going to send "fix for
> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> > separately, but it has no changelog at all.
> >>
> >> Probably it would be better if you drop everything except actually
> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >
> > I think it's best if I drop everything:
> >
> > mm-balloon_compaction-ignore-anonymous-pages.patch
> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> > mm-introduce-common-page-state-for-ballooned-memory.patch
> > mm-balloon_compaction-use-common-page-ballooning.patch
> > mm-balloon_compaction-general-cleanup.patch
> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >
> > Please go through it and send out a new version?
> >
> >
>
> I've found yet another bug in this code. It seems here is a nest.
> balloon_page_dequeue can race with balloon_page_isolate:
> balloon_page_isolate can remove page from list between
> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
> both of them lock page using trylock_page so race is tight but it is
> not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
> Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid
of such bugs changes the fact we still have to address them in the actual placed
code as we go on finding them. That's why I thought your inital changeset fine,
with patches for stable going first and code overhaul for next following them up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+ * Skip dequeue attempt for this page to a later round
+ * if balloon_page_isolate() has sucessfully isolated
+ * it just before we got the page lock here.
+ */
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
/*
* Raise the page refcount here to prevent any
* wrong
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-22 19:22 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-22 19:22 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> <akpm@linux-foundation.org> wrote:
> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >
> >> >
> >> > So I'm going to send "fix for
> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> > separately, but it has no changelog at all.
> >>
> >> Probably it would be better if you drop everything except actually
> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >
> > I think it's best if I drop everything:
> >
> > mm-balloon_compaction-ignore-anonymous-pages.patch
> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> > mm-introduce-common-page-state-for-ballooned-memory.patch
> > mm-balloon_compaction-use-common-page-ballooning.patch
> > mm-balloon_compaction-general-cleanup.patch
> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >
> > Please go through it and send out a new version?
> >
> >
>
> I've found yet another bug in this code. It seems here is a nest.
> balloon_page_dequeue can race with balloon_page_isolate:
> balloon_page_isolate can remove page from list between
> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
> both of them lock page using trylock_page so race is tight but it is
> not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
> Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid
of such bugs changes the fact we still have to address them in the actual placed
code as we go on finding them. That's why I thought your inital changeset fine,
with patches for stable going first and code overhaul for next following them up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+ * Skip dequeue attempt for this page to a later round
+ * if balloon_page_isolate() has sucessfully isolated
+ * it just before we got the page lock here.
+ */
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(&b_dev_info->pages_lock, flags);
/*
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-22 19:22 ` Rafael Aquini
@ 2014-09-22 20:06 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 20:06 UTC (permalink / raw)
To: Rafael Aquini
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
> On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> <akpm@linux-foundation.org> wrote:
>> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >
>> >> >
>> >> > So I'm going to send "fix for
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> > separately, but it has no changelog at all.
>> >>
>> >> Probably it would be better if you drop everything except actually
>> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >
>> > I think it's best if I drop everything:
>> >
>> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> > mm-balloon_compaction-use-common-page-ballooning.patch
>> > mm-balloon_compaction-general-cleanup.patch
>> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >
>> > Please go through it and send out a new version?
>> >
>> >
>>
>> I've found yet another bug in this code. It seems here is a nest.
>> balloon_page_dequeue can race with balloon_page_isolate:
>> balloon_page_isolate can remove page from list between
>> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
>> both of them lock page using trylock_page so race is tight but it is
>> not impossible.
> Plausible to happen if stress testing compaction simultaneously with
> freezing/unloading the balloon driver. As you noted, it's quite tight
> despite not impossible. Nice catch.
>
>
>> Probably it's really easier to rewrite it than to fix bugs one by one =/
> I'm not against a rewrite, but I don't think that rewriting the code to get rid
> of such bugs changes the fact we still have to address them in the actual placed
> code as we go on finding them. That's why I thought your inital changeset fine,
> with patches for stable going first and code overhaul for next following them up.
>
> For this race you spotted, I think a simple change like the following
> might be enough (not-tested)
This locking scheme is too fragile and uncommon.
What about this:
* special page->_mapcount marks ballooned pages
* page->private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page->private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
>
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index 6e45a50..fd3a497 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> balloon_dev_info *b_dev_info)
> * to be released by the balloon driver.
> */
> if (trylock_page(page)) {
> + /*
> + * Skip dequeue attempt for this page to a later round
> + * if balloon_page_isolate() has sucessfully isolated
> + * it just before we got the page lock here.
> + */
> + if (page_count(page) != 1) {
> + unlock_page(page);
> + continue
> + }
> +
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> /*
> * Raise the page refcount here to prevent any
> * wrong
>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-22 20:06 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 20:06 UTC (permalink / raw)
To: Rafael Aquini
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
> On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> <akpm@linux-foundation.org> wrote:
>> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >
>> >> >
>> >> > So I'm going to send "fix for
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> > separately, but it has no changelog at all.
>> >>
>> >> Probably it would be better if you drop everything except actually
>> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >
>> > I think it's best if I drop everything:
>> >
>> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> > mm-balloon_compaction-use-common-page-ballooning.patch
>> > mm-balloon_compaction-general-cleanup.patch
>> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >
>> > Please go through it and send out a new version?
>> >
>> >
>>
>> I've found yet another bug in this code. It seems here is a nest.
>> balloon_page_dequeue can race with balloon_page_isolate:
>> balloon_page_isolate can remove page from list between
>> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
>> both of them lock page using trylock_page so race is tight but it is
>> not impossible.
> Plausible to happen if stress testing compaction simultaneously with
> freezing/unloading the balloon driver. As you noted, it's quite tight
> despite not impossible. Nice catch.
>
>
>> Probably it's really easier to rewrite it than to fix bugs one by one =/
> I'm not against a rewrite, but I don't think that rewriting the code to get rid
> of such bugs changes the fact we still have to address them in the actual placed
> code as we go on finding them. That's why I thought your inital changeset fine,
> with patches for stable going first and code overhaul for next following them up.
>
> For this race you spotted, I think a simple change like the following
> might be enough (not-tested)
This locking scheme is too fragile and uncommon.
What about this:
* special page->_mapcount marks ballooned pages
* page->private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page->private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
>
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index 6e45a50..fd3a497 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> balloon_dev_info *b_dev_info)
> * to be released by the balloon driver.
> */
> if (trylock_page(page)) {
> + /*
> + * Skip dequeue attempt for this page to a later round
> + * if balloon_page_isolate() has sucessfully isolated
> + * it just before we got the page lock here.
> + */
> + if (page_count(page) != 1) {
> + unlock_page(page);
> + continue
> + }
> +
> spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> /*
> * Raise the page refcount here to prevent any
> * wrong
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-22 20:06 ` Konstantin Khlebnikov
@ 2014-09-22 20:22 ` Rafael Aquini
-1 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-22 20:22 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> >> <akpm@linux-foundation.org> wrote:
> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >> >
> >> >> >
> >> >> > So I'm going to send "fix for
> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> >> > separately, but it has no changelog at all.
> >> >>
> >> >> Probably it would be better if you drop everything except actually
> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >> >
> >> > I think it's best if I drop everything:
> >> >
> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
> >> > mm-balloon_compaction-use-common-page-ballooning.patch
> >> > mm-balloon_compaction-general-cleanup.patch
> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >> >
> >> > Please go through it and send out a new version?
> >> >
> >> >
> >>
> >> I've found yet another bug in this code. It seems here is a nest.
> >> balloon_page_dequeue can race with balloon_page_isolate:
> >> balloon_page_isolate can remove page from list between
> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> >> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
> >> both of them lock page using trylock_page so race is tight but it is
> >> not impossible.
> > Plausible to happen if stress testing compaction simultaneously with
> > freezing/unloading the balloon driver. As you noted, it's quite tight
> > despite not impossible. Nice catch.
> >
> >
> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
> > I'm not against a rewrite, but I don't think that rewriting the code to get rid
> > of such bugs changes the fact we still have to address them in the actual placed
> > code as we go on finding them. That's why I thought your inital changeset fine,
> > with patches for stable going first and code overhaul for next following them up.
> >
> > For this race you spotted, I think a simple change like the following
> > might be enough (not-tested)
>
> This locking scheme is too fragile and uncommon.
>
page_lock and refcounting was what I had at my disposal to sort these
races out since I haven't thoutgh on a special page->_mapcount when
designing this feature. It's the way other page races are sorted out.
Not arguing it's not fragile, but it's the way code is layed out since
some time, so we must check the feasibility of a total overhaul for
stable branches.
> What about this:
>
> * special page->_mapcount marks ballooned pages
> * page->private points to balloon (directly, without intermediate mapping)
> * flag PagePrivate means page currently in balloon page list (i.e. not
> isolated, like PageLRU for normal pages)
> * lock_page protects all of them
>
> balloon_page_dequeue() will delete page from balloon list only if it's
> not isolated, also it always clears page->private and balloon mark.
> put-back rechecks mark after locking the page and releases it as
> normal page if mark is gone.
>
I have already agreed with you here, since the changes above are mostly from
your original overhaul proposal. It's a much better approach for that
balloon code, no doubts. Thanks for doing it. Only thing we need to take
care here is about its requirement on changing the semantics for those
interfaces might turn the changes unfeasible for old stable branches. If we
can ignore this mentioned fact entirely, I don't see why not going with
your idea all branches across, otherwise I think we should overhaul the
code for -next, and send pontual fixes for stable.
Cheers,
-- Rafael
> >
> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> > index 6e45a50..fd3a497 100644
> > --- a/mm/balloon_compaction.c
> > +++ b/mm/balloon_compaction.c
> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> > balloon_dev_info *b_dev_info)
> > * to be released by the balloon driver.
> > */
> > if (trylock_page(page)) {
> > + /*
> > + * Skip dequeue attempt for this page to a later round
> > + * if balloon_page_isolate() has sucessfully isolated
> > + * it just before we got the page lock here.
> > + */
> > + if (page_count(page) != 1) {
> > + unlock_page(page);
> > + continue
> > + }
> > +
> > spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> > /*
> > * Raise the page refcount here to prevent any
> > * wrong
> >
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-22 20:22 ` Rafael Aquini
0 siblings, 0 replies; 63+ messages in thread
From: Rafael Aquini @ 2014-09-22 20:22 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> >> <akpm@linux-foundation.org> wrote:
> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
> >> >
> >> >> >
> >> >> > So I'm going to send "fix for
> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> >> > separately, but it has no changelog at all.
> >> >>
> >> >> Probably it would be better if you drop everything except actually
> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >> >
> >> > I think it's best if I drop everything:
> >> >
> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
> >> > mm-balloon_compaction-use-common-page-ballooning.patch
> >> > mm-balloon_compaction-general-cleanup.patch
> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >> >
> >> > Please go through it and send out a new version?
> >> >
> >> >
> >>
> >> I've found yet another bug in this code. It seems here is a nest.
> >> balloon_page_dequeue can race with balloon_page_isolate:
> >> balloon_page_isolate can remove page from list between
> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> >> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
> >> both of them lock page using trylock_page so race is tight but it is
> >> not impossible.
> > Plausible to happen if stress testing compaction simultaneously with
> > freezing/unloading the balloon driver. As you noted, it's quite tight
> > despite not impossible. Nice catch.
> >
> >
> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
> > I'm not against a rewrite, but I don't think that rewriting the code to get rid
> > of such bugs changes the fact we still have to address them in the actual placed
> > code as we go on finding them. That's why I thought your inital changeset fine,
> > with patches for stable going first and code overhaul for next following them up.
> >
> > For this race you spotted, I think a simple change like the following
> > might be enough (not-tested)
>
> This locking scheme is too fragile and uncommon.
>
page_lock and refcounting was what I had at my disposal to sort these
races out since I haven't thoutgh on a special page->_mapcount when
designing this feature. It's the way other page races are sorted out.
Not arguing it's not fragile, but it's the way code is layed out since
some time, so we must check the feasibility of a total overhaul for
stable branches.
> What about this:
>
> * special page->_mapcount marks ballooned pages
> * page->private points to balloon (directly, without intermediate mapping)
> * flag PagePrivate means page currently in balloon page list (i.e. not
> isolated, like PageLRU for normal pages)
> * lock_page protects all of them
>
> balloon_page_dequeue() will delete page from balloon list only if it's
> not isolated, also it always clears page->private and balloon mark.
> put-back rechecks mark after locking the page and releases it as
> normal page if mark is gone.
>
I have already agreed with you here, since the changes above are mostly from
your original overhaul proposal. It's a much better approach for that
balloon code, no doubts. Thanks for doing it. Only thing we need to take
care here is about its requirement on changing the semantics for those
interfaces might turn the changes unfeasible for old stable branches. If we
can ignore this mentioned fact entirely, I don't see why not going with
your idea all branches across, otherwise I think we should overhaul the
code for -next, and send pontual fixes for stable.
Cheers,
-- Rafael
> >
> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> > index 6e45a50..fd3a497 100644
> > --- a/mm/balloon_compaction.c
> > +++ b/mm/balloon_compaction.c
> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> > balloon_dev_info *b_dev_info)
> > * to be released by the balloon driver.
> > */
> > if (trylock_page(page)) {
> > + /*
> > + * Skip dequeue attempt for this page to a later round
> > + * if balloon_page_isolate() has sucessfully isolated
> > + * it just before we got the page lock here.
> > + */
> > + if (page_count(page) != 1) {
> > + unlock_page(page);
> > + continue
> > + }
> > +
> > spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> > /*
> > * Raise the page refcount here to prevent any
> > * wrong
> >
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
2014-09-22 20:22 ` Rafael Aquini
@ 2014-09-22 20:46 ` Konstantin Khlebnikov
-1 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 20:46 UTC (permalink / raw)
To: Rafael Aquini
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Tue, Sep 23, 2014 at 12:22 AM, Rafael Aquini <aquini@redhat.com> wrote:
> On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
>> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
>> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> >> <akpm@linux-foundation.org> wrote:
>> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >> >
>> >> >> >
>> >> >> > So I'm going to send "fix for
>> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> >> > separately, but it has no changelog at all.
>> >> >>
>> >> >> Probably it would be better if you drop everything except actually
>> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >> >
>> >> > I think it's best if I drop everything:
>> >> >
>> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning.patch
>> >> > mm-balloon_compaction-general-cleanup.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >> >
>> >> > Please go through it and send out a new version?
>> >> >
>> >> >
>> >>
>> >> I've found yet another bug in this code. It seems here is a nest.
>> >> balloon_page_dequeue can race with balloon_page_isolate:
>> >> balloon_page_isolate can remove page from list between
>> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> >> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
>> >> both of them lock page using trylock_page so race is tight but it is
>> >> not impossible.
>> > Plausible to happen if stress testing compaction simultaneously with
>> > freezing/unloading the balloon driver. As you noted, it's quite tight
>> > despite not impossible. Nice catch.
>> >
>> >
>> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
>> > I'm not against a rewrite, but I don't think that rewriting the code to get rid
>> > of such bugs changes the fact we still have to address them in the actual placed
>> > code as we go on finding them. That's why I thought your inital changeset fine,
>> > with patches for stable going first and code overhaul for next following them up.
>> >
>> > For this race you spotted, I think a simple change like the following
>> > might be enough (not-tested)
>>
>> This locking scheme is too fragile and uncommon.
>>
>
> page_lock and refcounting was what I had at my disposal to sort these
> races out since I haven't thoutgh on a special page->_mapcount when
> designing this feature. It's the way other page races are sorted out.
> Not arguing it's not fragile, but it's the way code is layed out since
> some time, so we must check the feasibility of a total overhaul for
> stable branches.
>
>
>> What about this:
>>
>> * special page->_mapcount marks ballooned pages
>> * page->private points to balloon (directly, without intermediate mapping)
>> * flag PagePrivate means page currently in balloon page list (i.e. not
>> isolated, like PageLRU for normal pages)
>> * lock_page protects all of them
>>
>> balloon_page_dequeue() will delete page from balloon list only if it's
>> not isolated, also it always clears page->private and balloon mark.
>> put-back rechecks mark after locking the page and releases it as
>> normal page if mark is gone.
>>
>
> I have already agreed with you here, since the changes above are mostly from
> your original overhaul proposal. It's a much better approach for that
> balloon code, no doubts. Thanks for doing it. Only thing we need to take
> care here is about its requirement on changing the semantics for those
> interfaces might turn the changes unfeasible for old stable branches. If we
> can ignore this mentioned fact entirely, I don't see why not going with
> your idea all branches across, otherwise I think we should overhaul the
> code for -next, and send pontual fixes for stable.
>
Ok, I'll try to implement this approach in suitable for stable branches way.
Keeping several different versions is overkill for this code.
>
> Cheers,
> -- Rafael
>
>> >
>> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
>> > index 6e45a50..fd3a497 100644
>> > --- a/mm/balloon_compaction.c
>> > +++ b/mm/balloon_compaction.c
>> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
>> > balloon_dev_info *b_dev_info)
>> > * to be released by the balloon driver.
>> > */
>> > if (trylock_page(page)) {
>> > + /*
>> > + * Skip dequeue attempt for this page to a later round
>> > + * if balloon_page_isolate() has sucessfully isolated
>> > + * it just before we got the page lock here.
>> > + */
>> > + if (page_count(page) != 1) {
>> > + unlock_page(page);
>> > + continue
>> > + }
>> > +
>> > spin_lock_irqsave(&b_dev_info->pages_lock, flags);
>> > /*
>> > * Raise the page refcount here to prevent any
>> > * wrong
>> >
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
@ 2014-09-22 20:46 ` Konstantin Khlebnikov
0 siblings, 0 replies; 63+ messages in thread
From: Konstantin Khlebnikov @ 2014-09-22 20:46 UTC (permalink / raw)
To: Rafael Aquini
Cc: Andrew Morton, Konstantin Khlebnikov, Linux Kernel Mailing List,
linux-mm, Andrey Ryabinin, Sasha Levin
On Tue, Sep 23, 2014 at 12:22 AM, Rafael Aquini <aquini@redhat.com> wrote:
> On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
>> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini <aquini@redhat.com> wrote:
>> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> >> <akpm@linux-foundation.org> wrote:
>> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <koct9i@gmail.com> wrote:
>> >> >
>> >> >> >
>> >> >> > So I'm going to send "fix for
>> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> >> > separately, but it has no changelog at all.
>> >> >>
>> >> >> Probably it would be better if you drop everything except actually
>> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >> >
>> >> > I think it's best if I drop everything:
>> >> >
>> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning.patch
>> >> > mm-balloon_compaction-general-cleanup.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >> >
>> >> > Please go through it and send out a new version?
>> >> >
>> >> >
>> >>
>> >> I've found yet another bug in this code. It seems here is a nest.
>> >> balloon_page_dequeue can race with balloon_page_isolate:
>> >> balloon_page_isolate can remove page from list between
>> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> >> balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock);
>> >> both of them lock page using trylock_page so race is tight but it is
>> >> not impossible.
>> > Plausible to happen if stress testing compaction simultaneously with
>> > freezing/unloading the balloon driver. As you noted, it's quite tight
>> > despite not impossible. Nice catch.
>> >
>> >
>> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
>> > I'm not against a rewrite, but I don't think that rewriting the code to get rid
>> > of such bugs changes the fact we still have to address them in the actual placed
>> > code as we go on finding them. That's why I thought your inital changeset fine,
>> > with patches for stable going first and code overhaul for next following them up.
>> >
>> > For this race you spotted, I think a simple change like the following
>> > might be enough (not-tested)
>>
>> This locking scheme is too fragile and uncommon.
>>
>
> page_lock and refcounting was what I had at my disposal to sort these
> races out since I haven't thoutgh on a special page->_mapcount when
> designing this feature. It's the way other page races are sorted out.
> Not arguing it's not fragile, but it's the way code is layed out since
> some time, so we must check the feasibility of a total overhaul for
> stable branches.
>
>
>> What about this:
>>
>> * special page->_mapcount marks ballooned pages
>> * page->private points to balloon (directly, without intermediate mapping)
>> * flag PagePrivate means page currently in balloon page list (i.e. not
>> isolated, like PageLRU for normal pages)
>> * lock_page protects all of them
>>
>> balloon_page_dequeue() will delete page from balloon list only if it's
>> not isolated, also it always clears page->private and balloon mark.
>> put-back rechecks mark after locking the page and releases it as
>> normal page if mark is gone.
>>
>
> I have already agreed with you here, since the changes above are mostly from
> your original overhaul proposal. It's a much better approach for that
> balloon code, no doubts. Thanks for doing it. Only thing we need to take
> care here is about its requirement on changing the semantics for those
> interfaces might turn the changes unfeasible for old stable branches. If we
> can ignore this mentioned fact entirely, I don't see why not going with
> your idea all branches across, otherwise I think we should overhaul the
> code for -next, and send pontual fixes for stable.
>
Ok, I'll try to implement this approach in suitable for stable branches way.
Keeping several different versions is overkill for this code.
>
> Cheers,
> -- Rafael
>
>> >
>> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
>> > index 6e45a50..fd3a497 100644
>> > --- a/mm/balloon_compaction.c
>> > +++ b/mm/balloon_compaction.c
>> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
>> > balloon_dev_info *b_dev_info)
>> > * to be released by the balloon driver.
>> > */
>> > if (trylock_page(page)) {
>> > + /*
>> > + * Skip dequeue attempt for this page to a later round
>> > + * if balloon_page_isolate() has sucessfully isolated
>> > + * it just before we got the page lock here.
>> > + */
>> > + if (page_count(page) != 1) {
>> > + unlock_page(page);
>> > + continue
>> > + }
>> > +
>> > spin_lock_irqsave(&b_dev_info->pages_lock, flags);
>> > /*
>> > * Raise the page refcount here to prevent any
>> > * wrong
>> >
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 63+ messages in thread
end of thread, other threads:[~2014-09-22 20:46 UTC | newest]
Thread overview: 63+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-30 16:41 [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-08-30 16:41 ` [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 12:29 ` Rafael Aquini
2014-09-02 12:29 ` Rafael Aquini
2014-08-30 16:41 ` [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 12:31 ` Rafael Aquini
2014-09-02 12:31 ` Rafael Aquini
2014-08-30 16:41 ` [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 12:32 ` Rafael Aquini
2014-09-02 12:32 ` Rafael Aquini
2014-08-30 16:41 ` [PATCH v2 4/6] mm: introduce common page state for ballooned memory Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 12:53 ` Rafael Aquini
2014-09-02 12:53 ` Rafael Aquini
2014-09-12 23:51 ` Andrew Morton
2014-09-12 23:51 ` Andrew Morton
2014-09-13 5:26 ` Konstantin Khlebnikov
2014-09-13 5:26 ` Konstantin Khlebnikov
2014-09-13 5:42 ` Andrew Morton
2014-09-13 5:42 ` Andrew Morton
2014-09-13 8:22 ` Konstantin Khlebnikov
2014-09-19 21:35 ` Andrew Morton
2014-09-19 21:35 ` Andrew Morton
2014-09-20 5:25 ` Konstantin Khlebnikov
2014-09-20 5:25 ` Konstantin Khlebnikov
2014-09-20 6:23 ` Andrew Morton
2014-09-20 6:23 ` Andrew Morton
2014-09-22 18:40 ` Konstantin Khlebnikov
2014-09-22 18:40 ` Konstantin Khlebnikov
2014-09-22 19:22 ` Rafael Aquini
2014-09-22 19:22 ` Rafael Aquini
2014-09-22 20:06 ` Konstantin Khlebnikov
2014-09-22 20:06 ` Konstantin Khlebnikov
2014-09-22 20:22 ` Rafael Aquini
2014-09-22 20:22 ` Rafael Aquini
2014-09-22 20:46 ` Konstantin Khlebnikov
2014-09-22 20:46 ` Konstantin Khlebnikov
2014-09-13 14:03 ` Sasha Levin
2014-09-13 14:03 ` Sasha Levin
2014-08-30 16:41 ` [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 12:57 ` Rafael Aquini
2014-09-02 12:57 ` Rafael Aquini
2014-09-12 23:57 ` Andrew Morton
2014-09-12 23:57 ` Andrew Morton
2014-08-30 16:41 ` [PATCH v2 6/6] mm/balloon_compaction: general cleanup Konstantin Khlebnikov
2014-08-30 16:41 ` Konstantin Khlebnikov
2014-09-02 13:09 ` Rafael Aquini
2014-09-02 13:09 ` Rafael Aquini
2014-09-13 0:04 ` Andrew Morton
2014-09-13 0:04 ` Andrew Morton
2014-09-13 0:06 ` Andrew Morton
2014-09-13 0:06 ` Andrew Morton
2014-09-13 5:43 ` Konstantin Khlebnikov
2014-09-13 5:43 ` Konstantin Khlebnikov
2014-09-13 0:09 ` [PATCH v2 0/6] mm/balloon_compaction: fixes and cleanups Andrew Morton
2014-09-13 0:09 ` Andrew Morton
2014-09-13 5:01 ` Konstantin Khlebnikov
2014-09-13 5:01 ` Konstantin Khlebnikov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.