[RFC][PATCH v3 0/7] introduce automatic pool compaction

[RFC][PATCH v3 0/7] introduce automatic pool compaction

[Sergey Senozhatsky]

Hello,

This patch set tweaks compaction and makes it possible to trigger
pool compaction automatically when system is getting low on memory.

zsmalloc in some cases can suffer from a notable fragmentation and
compaction can release some considerable amount of memory. The problem
here is that currently we fully rely on user space to perform compaction
when needed. However, performing zsmalloc compaction is not always an
obvious thing to do. For example, suppose we have a `idle' fragmented
(compaction was never performed) zram device and system is getting low
on memory due to some 3rd party user processes (gcc LTO, or firefox, etc.).
It's quite unlikely that user space will issue zpool compaction in this
case. Besides, user space cannot tell for sure how badly pool is
fragmented; however, this info is known to zsmalloc and, hence, to a
shrinker.

v3:
-- drop almost_empty waterline adjustment patch (Minchan)
-- do not hold class->lock for the entire compaction period (Minchan)

v2:
-- use a slab shrinker instead of triggering compaction from zs_free (Minchan)

Sergey Senozhatsky (7):
  zsmalloc: drop unused variable `nr_to_migrate'
  zsmalloc: partial page ordering within a fullness_list
  zsmalloc: always keep per-class stats
  zsmalloc: introduce zs_can_compact() function
  zsmalloc: cosmetic compaction code adjustments
  zsmalloc/zram: move `num_migrated' to zs_pool
  zsmalloc: register a shrinker to trigger auto-compaction

 drivers/block/zram/zram_drv.c |  12 +--
 drivers/block/zram/zram_drv.h |   1 -
 include/linux/zsmalloc.h      |   1 +
 mm/zsmalloc.c                 | 220 +++++++++++++++++++++++++++---------------
 4 files changed, 150 insertions(+), 84 deletions(-)

-- 
2.4.4

[RFC][PATCH v3 0/7] introduce automatic pool compaction

[Sergey Senozhatsky]

Hello,

This patch set tweaks compaction and makes it possible to trigger
pool compaction automatically when system is getting low on memory.

zsmalloc in some cases can suffer from a notable fragmentation and
compaction can release some considerable amount of memory. The problem
here is that currently we fully rely on user space to perform compaction
when needed. However, performing zsmalloc compaction is not always an
obvious thing to do. For example, suppose we have a `idle' fragmented
(compaction was never performed) zram device and system is getting low
on memory due to some 3rd party user processes (gcc LTO, or firefox, etc.).
It's quite unlikely that user space will issue zpool compaction in this
case. Besides, user space cannot tell for sure how badly pool is
fragmented; however, this info is known to zsmalloc and, hence, to a
shrinker.

v3:
-- drop almost_empty waterline adjustment patch (Minchan)
-- do not hold class->lock for the entire compaction period (Minchan)

v2:
-- use a slab shrinker instead of triggering compaction from zs_free (Minchan)

Sergey Senozhatsky (7):
  zsmalloc: drop unused variable `nr_to_migrate'
  zsmalloc: partial page ordering within a fullness_list
  zsmalloc: always keep per-class stats
  zsmalloc: introduce zs_can_compact() function
  zsmalloc: cosmetic compaction code adjustments
  zsmalloc/zram: move `num_migrated' to zs_pool
  zsmalloc: register a shrinker to trigger auto-compaction

 drivers/block/zram/zram_drv.c |  12 +--
 drivers/block/zram/zram_drv.h |   1 -
 include/linux/zsmalloc.h      |   1 +
 mm/zsmalloc.c                 | 220 +++++++++++++++++++++++++++---------------
 4 files changed, 150 insertions(+), 84 deletions(-)

-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 1/7] zsmalloc: drop unused variable `nr_to_migrate'

[Sergey Senozhatsky]

__zs_compact() does not use `nr_to_migrate', drop it.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 0a7f81a..7d816c2 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1703,7 +1703,6 @@ static struct page *isolate_source_page(struct size_class *class)
 static unsigned long __zs_compact(struct zs_pool *pool,
 				struct size_class *class)
 {
-	int nr_to_migrate;
 	struct zs_compact_control cc;
 	struct page *src_page;
 	struct page *dst_page = NULL;
@@ -1714,8 +1713,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 		BUG_ON(!is_first_page(src_page));
 
-		/* The goal is to migrate all live objects in source page */
-		nr_to_migrate = src_page->inuse;
 		cc.index = 0;
 		cc.s_page = src_page;
 
@@ -1730,7 +1727,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 			putback_zspage(pool, class, dst_page);
 			nr_total_migrated += cc.nr_migrated;
-			nr_to_migrate -= cc.nr_migrated;
 		}
 
 		/* Stop if we couldn't find slot */
-- 
2.4.4

[RFC][PATCHv3 1/7] zsmalloc: drop unused variable `nr_to_migrate'

[Sergey Senozhatsky]

__zs_compact() does not use `nr_to_migrate', drop it.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 0a7f81a..7d816c2 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1703,7 +1703,6 @@ static struct page *isolate_source_page(struct size_class *class)
 static unsigned long __zs_compact(struct zs_pool *pool,
 				struct size_class *class)
 {
-	int nr_to_migrate;
 	struct zs_compact_control cc;
 	struct page *src_page;
 	struct page *dst_page = NULL;
@@ -1714,8 +1713,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 		BUG_ON(!is_first_page(src_page));
 
-		/* The goal is to migrate all live objects in source page */
-		nr_to_migrate = src_page->inuse;
 		cc.index = 0;
 		cc.s_page = src_page;
 
@@ -1730,7 +1727,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 			putback_zspage(pool, class, dst_page);
 			nr_total_migrated += cc.nr_migrated;
-			nr_to_migrate -= cc.nr_migrated;
 		}
 
 		/* Stop if we couldn't find slot */
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
pages. Put a page with higher ->inuse count first within its
->fullness_list, which will give us better chances to fill up this
page with new objects (find_get_zspage() return ->fullness_list head
for new object allocation), so some zspages will become
ZS_ALMOST_FULL/ZS_FULL quicker.

It performs a trivial and cheap ->inuse compare which does not slow
down zsmalloc, and in the worst case it keeps the list pages not in
any particular order, just like we do it now.

A more expensive solution could sort fullness_list by ->inuse count.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7d816c2..6e2ebb6 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
 		return;
 
 	head = &class->fullness_list[fullness];
-	if (*head)
-		list_add_tail(&page->lru, &(*head)->lru);
+	if (*head) {
+		/*
+		 * We want to see more ZS_FULL pages and less almost
+		 * empty/full. Put pages with higher ->inuse first.
+		 */
+		if (page->inuse < (*head)->inuse)
+			list_add_tail(&page->lru, &(*head)->lru);
+		else
+			list_add(&page->lru, &(*head)->lru);
+	}
 
 	*head = page;
 	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
-- 
2.4.4

[RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
pages. Put a page with higher ->inuse count first within its
->fullness_list, which will give us better chances to fill up this
page with new objects (find_get_zspage() return ->fullness_list head
for new object allocation), so some zspages will become
ZS_ALMOST_FULL/ZS_FULL quicker.

It performs a trivial and cheap ->inuse compare which does not slow
down zsmalloc, and in the worst case it keeps the list pages not in
any particular order, just like we do it now.

A more expensive solution could sort fullness_list by ->inuse count.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7d816c2..6e2ebb6 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
 		return;
 
 	head = &class->fullness_list[fullness];
-	if (*head)
-		list_add_tail(&page->lru, &(*head)->lru);
+	if (*head) {
+		/*
+		 * We want to see more ZS_FULL pages and less almost
+		 * empty/full. Put pages with higher ->inuse first.
+		 */
+		if (page->inuse < (*head)->inuse)
+			list_add_tail(&page->lru, &(*head)->lru);
+		else
+			list_add(&page->lru, &(*head)->lru);
+	}
 
 	*head = page;
 	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Sergey Senozhatsky]

Always account per-class `zs_size_stat' stats. This data will
help us make better decisions during compaction. We are especially
interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
class compaction will result in any memory gain.

For instance, we know the number of allocated objects in the class,
the number of objects being used (so we also know how many objects
are not used) and the number of objects per-page. So we can ensure
if we have enough unused objects to form at least one ZS_EMPTY
zspage during compaction.

We calculate this value on per-class basis so we can calculate a
total number of zspages that can be released. Which is exactly what
a shrinker wants to know.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 48 ++++++++++++------------------------------------
 1 file changed, 12 insertions(+), 36 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 6e2ebb6..4b6f12e 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -169,14 +169,12 @@ enum zs_stat_type {
 	NR_ZS_STAT_TYPE,
 };
 
-#ifdef CONFIG_ZSMALLOC_STAT
-
-static struct dentry *zs_stat_root;
-
 struct zs_size_stat {
 	unsigned long objs[NR_ZS_STAT_TYPE];
 };
 
+#ifdef CONFIG_ZSMALLOC_STAT
+static struct dentry *zs_stat_root;
 #endif
 
 /*
@@ -201,25 +199,21 @@ static int zs_size_classes;
 static const int fullness_threshold_frac = 4;
 
 struct size_class {
+	spinlock_t		lock;
+	struct page		*fullness_list[_ZS_NR_FULLNESS_GROUPS];
 	/*
 	 * Size of objects stored in this class. Must be multiple
 	 * of ZS_ALIGN.
 	 */
-	int size;
-	unsigned int index;
+	int			size;
+	unsigned int		index;
 
 	/* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */
-	int pages_per_zspage;
-	/* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
-	bool huge;
-
-#ifdef CONFIG_ZSMALLOC_STAT
-	struct zs_size_stat stats;
-#endif
-
-	spinlock_t lock;
+	int			pages_per_zspage;
+	struct zs_size_stat	stats;
 
-	struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
+	/* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
+	bool			huge;
 };
 
 /*
@@ -441,8 +435,6 @@ static int get_size_class_index(int size)
 	return min(zs_size_classes - 1, idx);
 }
 
-#ifdef CONFIG_ZSMALLOC_STAT
-
 static inline void zs_stat_inc(struct size_class *class,
 				enum zs_stat_type type, unsigned long cnt)
 {
@@ -461,6 +453,8 @@ static inline unsigned long zs_stat_get(struct size_class *class,
 	return class->stats.objs[type];
 }
 
+#ifdef CONFIG_ZSMALLOC_STAT
+
 static int __init zs_stat_init(void)
 {
 	if (!debugfs_initialized())
@@ -576,23 +570,6 @@ static void zs_pool_stat_destroy(struct zs_pool *pool)
 }
 
 #else /* CONFIG_ZSMALLOC_STAT */
-
-static inline void zs_stat_inc(struct size_class *class,
-				enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline void zs_stat_dec(struct size_class *class,
-				enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline unsigned long zs_stat_get(struct size_class *class,
-				enum zs_stat_type type)
-{
-	return 0;
-}
-
 static int __init zs_stat_init(void)
 {
 	return 0;
@@ -610,7 +587,6 @@ static inline int zs_pool_stat_create(char *name, struct zs_pool *pool)
 static inline void zs_pool_stat_destroy(struct zs_pool *pool)
 {
 }
-
 #endif
 
 
-- 
2.4.4

[RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Sergey Senozhatsky]

Always account per-class `zs_size_stat' stats. This data will
help us make better decisions during compaction. We are especially
interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
class compaction will result in any memory gain.

For instance, we know the number of allocated objects in the class,
the number of objects being used (so we also know how many objects
are not used) and the number of objects per-page. So we can ensure
if we have enough unused objects to form at least one ZS_EMPTY
zspage during compaction.

We calculate this value on per-class basis so we can calculate a
total number of zspages that can be released. Which is exactly what
a shrinker wants to know.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 48 ++++++++++++------------------------------------
 1 file changed, 12 insertions(+), 36 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 6e2ebb6..4b6f12e 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -169,14 +169,12 @@ enum zs_stat_type {
 	NR_ZS_STAT_TYPE,
 };
 
-#ifdef CONFIG_ZSMALLOC_STAT
-
-static struct dentry *zs_stat_root;
-
 struct zs_size_stat {
 	unsigned long objs[NR_ZS_STAT_TYPE];
 };
 
+#ifdef CONFIG_ZSMALLOC_STAT
+static struct dentry *zs_stat_root;
 #endif
 
 /*
@@ -201,25 +199,21 @@ static int zs_size_classes;
 static const int fullness_threshold_frac = 4;
 
 struct size_class {
+	spinlock_t		lock;
+	struct page		*fullness_list[_ZS_NR_FULLNESS_GROUPS];
 	/*
 	 * Size of objects stored in this class. Must be multiple
 	 * of ZS_ALIGN.
 	 */
-	int size;
-	unsigned int index;
+	int			size;
+	unsigned int		index;
 
 	/* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */
-	int pages_per_zspage;
-	/* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
-	bool huge;
-
-#ifdef CONFIG_ZSMALLOC_STAT
-	struct zs_size_stat stats;
-#endif
-
-	spinlock_t lock;
+	int			pages_per_zspage;
+	struct zs_size_stat	stats;
 
-	struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
+	/* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
+	bool			huge;
 };
 
 /*
@@ -441,8 +435,6 @@ static int get_size_class_index(int size)
 	return min(zs_size_classes - 1, idx);
 }
 
-#ifdef CONFIG_ZSMALLOC_STAT
-
 static inline void zs_stat_inc(struct size_class *class,
 				enum zs_stat_type type, unsigned long cnt)
 {
@@ -461,6 +453,8 @@ static inline unsigned long zs_stat_get(struct size_class *class,
 	return class->stats.objs[type];
 }
 
+#ifdef CONFIG_ZSMALLOC_STAT
+
 static int __init zs_stat_init(void)
 {
 	if (!debugfs_initialized())
@@ -576,23 +570,6 @@ static void zs_pool_stat_destroy(struct zs_pool *pool)
 }
 
 #else /* CONFIG_ZSMALLOC_STAT */
-
-static inline void zs_stat_inc(struct size_class *class,
-				enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline void zs_stat_dec(struct size_class *class,
-				enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline unsigned long zs_stat_get(struct size_class *class,
-				enum zs_stat_type type)
-{
-	return 0;
-}
-
 static int __init zs_stat_init(void)
 {
 	return 0;
@@ -610,7 +587,6 @@ static inline int zs_pool_stat_create(char *name, struct zs_pool *pool)
 static inline void zs_pool_stat_destroy(struct zs_pool *pool)
 {
 }
-
 #endif
 
 
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Sergey Senozhatsky]

This function checks if class compaction will free any pages.
Rephrasing -- do we have enough unused objects to form at least
one ZS_EMPTY page and free it. It aborts compaction if class
compaction will not result in any (further) savings.

EXAMPLE (this debug output is not part of this patch set):

-- class size
-- number of allocated objects
-- number of used objects
-- max objects per zspage
-- pages per zspage
-- estimated number of pages that will be freed

[..]
class-512 objs:544 inuse:540 maxobj-per-zspage:8  pages-per-zspage:1 zspages-to-free:0
 ... class-512 compaction is useless. break
class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
 ... class-496 compaction is useless. break
class-448 objs:657 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:4
class-448 objs:648 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:3
class-448 objs:639 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:2
class-448 objs:630 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:1
class-448 objs:621 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:0
 ... class-448 compaction is useless. break
class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
 ... class-432 compaction is useless. break
class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
 ... class-416 compaction is useless. break
class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
 ... class-400 compaction is useless. break
class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
 ... class-384 compaction is useless. break
[..]

Every "compaction is useless" indicates that we saved CPU cycles.

class-512 has
	544	object allocated
	540	objects used
	8	objects per-page

Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
migrate all of its objects and free this zspage; so compaction will not
make a lot of sense, it's better to just leave it as is.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 4b6f12e..18e10a4 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
 	return page;
 }
 
+/*
+ * Make sure that we actually can compact this class,
+ * IOW if migration will release at least one szpage.
+ *
+ * Should be called under class->lock
+ */
+static unsigned long zs_can_compact(struct size_class *class)
+{
+	/*
+	 * Calculate how many unused allocated objects we
+	 * have and see if we can free any zspages. Otherwise,
+	 * compaction can just move objects back and forth w/o
+	 * any memory gain.
+	 */
+	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
+		zs_stat_get(class, OBJ_USED);
+
+	obj_wasted /= get_maxobj_per_zspage(class->size,
+			class->pages_per_zspage);
+	return obj_wasted;
+}
+
 static unsigned long __zs_compact(struct zs_pool *pool,
 				struct size_class *class)
 {
@@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 		BUG_ON(!is_first_page(src_page));
 
+		if (!zs_can_compact(class))
+			break;
+
 		cc.index = 0;
 		cc.s_page = src_page;
 
-- 
2.4.4

[RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Sergey Senozhatsky]

This function checks if class compaction will free any pages.
Rephrasing -- do we have enough unused objects to form at least
one ZS_EMPTY page and free it. It aborts compaction if class
compaction will not result in any (further) savings.

EXAMPLE (this debug output is not part of this patch set):

-- class size
-- number of allocated objects
-- number of used objects
-- max objects per zspage
-- pages per zspage
-- estimated number of pages that will be freed

[..]
class-512 objs:544 inuse:540 maxobj-per-zspage:8  pages-per-zspage:1 zspages-to-free:0
 ... class-512 compaction is useless. break
class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
 ... class-496 compaction is useless. break
class-448 objs:657 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:4
class-448 objs:648 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:3
class-448 objs:639 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:2
class-448 objs:630 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:1
class-448 objs:621 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:0
 ... class-448 compaction is useless. break
class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
 ... class-432 compaction is useless. break
class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
 ... class-416 compaction is useless. break
class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
 ... class-400 compaction is useless. break
class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
 ... class-384 compaction is useless. break
[..]

Every "compaction is useless" indicates that we saved CPU cycles.

class-512 has
	544	object allocated
	540	objects used
	8	objects per-page

Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
migrate all of its objects and free this zspage; so compaction will not
make a lot of sense, it's better to just leave it as is.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 4b6f12e..18e10a4 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
 	return page;
 }
 
+/*
+ * Make sure that we actually can compact this class,
+ * IOW if migration will release at least one szpage.
+ *
+ * Should be called under class->lock
+ */
+static unsigned long zs_can_compact(struct size_class *class)
+{
+	/*
+	 * Calculate how many unused allocated objects we
+	 * have and see if we can free any zspages. Otherwise,
+	 * compaction can just move objects back and forth w/o
+	 * any memory gain.
+	 */
+	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
+		zs_stat_get(class, OBJ_USED);
+
+	obj_wasted /= get_maxobj_per_zspage(class->size,
+			class->pages_per_zspage);
+	return obj_wasted;
+}
+
 static unsigned long __zs_compact(struct zs_pool *pool,
 				struct size_class *class)
 {
@@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 
 		BUG_ON(!is_first_page(src_page));
 
+		if (!zs_can_compact(class))
+			break;
+
 		cc.index = 0;
 		cc.s_page = src_page;
 
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 5/7] zsmalloc: cosmetic compaction code adjustments

[Sergey Senozhatsky]

Change zs_object_copy() argument order to be (DST, SRC) rather
than (SRC, DST). copy/move functions usually have (to, from)
arguments order.

Rename alloc_target_page() to isolate_target_page(). This
function doesn't allocate anything, it isolates target page,
pretty much like isolate_source_page().

Tweak __zs_compact() comment.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 18e10a4..5fa96ae 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1479,7 +1479,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
 }
 EXPORT_SYMBOL_GPL(zs_free);
 
-static void zs_object_copy(unsigned long src, unsigned long dst,
+static void zs_object_copy(unsigned long dst, unsigned long src,
 				struct size_class *class)
 {
 	struct page *s_page, *d_page;
@@ -1620,7 +1620,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 
 		used_obj = handle_to_obj(handle);
 		free_obj = obj_malloc(d_page, class, handle);
-		zs_object_copy(used_obj, free_obj, class);
+		zs_object_copy(free_obj, used_obj, class);
 		index++;
 		record_obj(handle, free_obj);
 		unpin_tag(handle);
@@ -1636,7 +1636,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 	return ret;
 }
 
-static struct page *alloc_target_page(struct size_class *class)
+static struct page *isolate_target_page(struct size_class *class)
 {
 	int i;
 	struct page *page;
@@ -1725,11 +1725,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		cc.index = 0;
 		cc.s_page = src_page;
 
-		while ((dst_page = alloc_target_page(class))) {
+		while ((dst_page = isolate_target_page(class))) {
 			cc.d_page = dst_page;
 			/*
-			 * If there is no more space in dst_page, try to
-			 * allocate another zspage.
+			 * If there is no more space in dst_page, resched
+			 * and see if anyone had allocated another zspage.
 			 */
 			if (!migrate_zspage(pool, class, &cc))
 				break;
-- 
2.4.4

[RFC][PATCHv3 5/7] zsmalloc: cosmetic compaction code adjustments

[Sergey Senozhatsky]

Change zs_object_copy() argument order to be (DST, SRC) rather
than (SRC, DST). copy/move functions usually have (to, from)
arguments order.

Rename alloc_target_page() to isolate_target_page(). This
function doesn't allocate anything, it isolates target page,
pretty much like isolate_source_page().

Tweak __zs_compact() comment.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 18e10a4..5fa96ae 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1479,7 +1479,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
 }
 EXPORT_SYMBOL_GPL(zs_free);
 
-static void zs_object_copy(unsigned long src, unsigned long dst,
+static void zs_object_copy(unsigned long dst, unsigned long src,
 				struct size_class *class)
 {
 	struct page *s_page, *d_page;
@@ -1620,7 +1620,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 
 		used_obj = handle_to_obj(handle);
 		free_obj = obj_malloc(d_page, class, handle);
-		zs_object_copy(used_obj, free_obj, class);
+		zs_object_copy(free_obj, used_obj, class);
 		index++;
 		record_obj(handle, free_obj);
 		unpin_tag(handle);
@@ -1636,7 +1636,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 	return ret;
 }
 
-static struct page *alloc_target_page(struct size_class *class)
+static struct page *isolate_target_page(struct size_class *class)
 {
 	int i;
 	struct page *page;
@@ -1725,11 +1725,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		cc.index = 0;
 		cc.s_page = src_page;
 
-		while ((dst_page = alloc_target_page(class))) {
+		while ((dst_page = isolate_target_page(class))) {
 			cc.d_page = dst_page;
 			/*
-			 * If there is no more space in dst_page, try to
-			 * allocate another zspage.
+			 * If there is no more space in dst_page, resched
+			 * and see if anyone had allocated another zspage.
 			 */
 			if (!migrate_zspage(pool, class, &cc))
 				break;
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 6/7] zsmalloc/zram: move `num_migrated' to zs_pool

[Sergey Senozhatsky]

Remove the number of migrated objects from `zs_compact_control'
and move it to `zs_pool'. `zs_compact_control' has a limited
lifespan, we lose it when zs_compaction() returns back to zram.

To keep track of objects migrated during auto-compaction (issued
by the shrinker) we need to store this number in zs_pool.

Introduce zs_get_num_migrated() to export zs_pool's ->num_migrated
counter and use it in zram, so we can also drop a zram's copy of
`num_migrated'.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 drivers/block/zram/zram_drv.c | 12 ++++++------
 drivers/block/zram/zram_drv.h |  1 -
 include/linux/zsmalloc.h      |  1 +
 mm/zsmalloc.c                 | 41 +++++++++++++++++++----------------------
 4 files changed, 26 insertions(+), 29 deletions(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index fb655e8..28ad3f8 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -388,7 +388,6 @@ static ssize_t comp_algorithm_store(struct device *dev,
 static ssize_t compact_store(struct device *dev,
 		struct device_attribute *attr, const char *buf, size_t len)
 {
-	unsigned long nr_migrated;
 	struct zram *zram = dev_to_zram(dev);
 	struct zram_meta *meta;
 
@@ -399,8 +398,7 @@ static ssize_t compact_store(struct device *dev,
 	}
 
 	meta = zram->meta;
-	nr_migrated = zs_compact(meta->mem_pool);
-	atomic64_add(nr_migrated, &zram->stats.num_migrated);
+	zs_compact(meta->mem_pool);
 	up_read(&zram->init_lock);
 
 	return len;
@@ -428,13 +426,15 @@ static ssize_t mm_stat_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
 	struct zram *zram = dev_to_zram(dev);
-	u64 orig_size, mem_used = 0;
+	u64 orig_size, mem_used = 0, num_migrated = 0;
 	long max_used;
 	ssize_t ret;
 
 	down_read(&zram->init_lock);
-	if (init_done(zram))
+	if (init_done(zram)) {
 		mem_used = zs_get_total_pages(zram->meta->mem_pool);
+		num_migrated = zs_get_num_migrated(zram->meta->mem_pool);
+	}
 
 	orig_size = atomic64_read(&zram->stats.pages_stored);
 	max_used = atomic_long_read(&zram->stats.max_used_pages);
@@ -447,7 +447,7 @@ static ssize_t mm_stat_show(struct device *dev,
 			zram->limit_pages << PAGE_SHIFT,
 			max_used << PAGE_SHIFT,
 			(u64)atomic64_read(&zram->stats.zero_pages),
-			(u64)atomic64_read(&zram->stats.num_migrated));
+			num_migrated);
 	up_read(&zram->init_lock);
 
 	return ret;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 6dbe2df..8e92339 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -78,7 +78,6 @@ struct zram_stats {
 	atomic64_t compr_data_size;	/* compressed size of pages stored */
 	atomic64_t num_reads;	/* failed + successful */
 	atomic64_t num_writes;	/* --do-- */
-	atomic64_t num_migrated;	/* no. of migrated object */
 	atomic64_t failed_reads;	/* can happen when memory is too low */
 	atomic64_t failed_writes;	/* can happen when memory is too low */
 	atomic64_t invalid_io;	/* non-page-aligned I/O requests */
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 1338190..e878875 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -47,6 +47,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle,
 void zs_unmap_object(struct zs_pool *pool, unsigned long handle);
 
 unsigned long zs_get_total_pages(struct zs_pool *pool);
+unsigned long zs_get_num_migrated(struct zs_pool *pool);
 unsigned long zs_compact(struct zs_pool *pool);
 
 #endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 5fa96ae..c9aea0a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -237,16 +237,19 @@ struct link_free {
 };
 
 struct zs_pool {
-	char *name;
+	char			*name;
 
-	struct size_class **size_class;
-	struct kmem_cache *handle_cachep;
+	struct size_class	**size_class;
+	struct kmem_cache	*handle_cachep;
 
-	gfp_t flags;	/* allocation flags used when growing pool */
-	atomic_long_t pages_allocated;
+	/* Allocation flags used when growing pool */
+	gfp_t			flags;
+	atomic_long_t		pages_allocated;
+	/* How many objects were migrated */
+	unsigned long		num_migrated;
 
 #ifdef CONFIG_ZSMALLOC_STAT
-	struct dentry *stat_dentry;
+	struct dentry		*stat_dentry;
 #endif
 };
 
@@ -1220,6 +1223,12 @@ unsigned long zs_get_total_pages(struct zs_pool *pool)
 }
 EXPORT_SYMBOL_GPL(zs_get_total_pages);
 
+unsigned long zs_get_num_migrated(struct zs_pool *pool)
+{
+	return pool->num_migrated;
+}
+EXPORT_SYMBOL_GPL(zs_get_num_migrated);
+
 /**
  * zs_map_object - get address of allocated object from handle.
  * @pool: pool from which the object was allocated
@@ -1586,8 +1595,6 @@ struct zs_compact_control {
 	 /* Starting object index within @s_page which used for live object
 	  * in the subpage. */
 	int index;
-	/* how many of objects are migrated */
-	int nr_migrated;
 };
 
 static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
@@ -1598,7 +1605,6 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 	struct page *s_page = cc->s_page;
 	struct page *d_page = cc->d_page;
 	unsigned long index = cc->index;
-	int nr_migrated = 0;
 	int ret = 0;
 
 	while (1) {
@@ -1625,13 +1631,12 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 		record_obj(handle, free_obj);
 		unpin_tag(handle);
 		obj_free(pool, class, used_obj);
-		nr_migrated++;
+		pool->num_migrated++;
 	}
 
 	/* Remember last position in this iteration */
 	cc->s_page = s_page;
 	cc->index = index;
-	cc->nr_migrated = nr_migrated;
 
 	return ret;
 }
@@ -1706,13 +1711,11 @@ static unsigned long zs_can_compact(struct size_class *class)
 	return obj_wasted;
 }
 
-static unsigned long __zs_compact(struct zs_pool *pool,
-				struct size_class *class)
+static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 {
 	struct zs_compact_control cc;
 	struct page *src_page;
 	struct page *dst_page = NULL;
-	unsigned long nr_total_migrated = 0;
 
 	spin_lock(&class->lock);
 	while ((src_page = isolate_source_page(class))) {
@@ -1735,7 +1738,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 				break;
 
 			putback_zspage(pool, class, dst_page);
-			nr_total_migrated += cc.nr_migrated;
 		}
 
 		/* Stop if we couldn't find slot */
@@ -1745,7 +1747,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		putback_zspage(pool, class, dst_page);
 		putback_zspage(pool, class, src_page);
 		spin_unlock(&class->lock);
-		nr_total_migrated += cc.nr_migrated;
 		cond_resched();
 		spin_lock(&class->lock);
 	}
@@ -1754,14 +1755,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		putback_zspage(pool, class, src_page);
 
 	spin_unlock(&class->lock);
-
-	return nr_total_migrated;
 }
 
 unsigned long zs_compact(struct zs_pool *pool)
 {
 	int i;
-	unsigned long nr_migrated = 0;
 	struct size_class *class;
 
 	for (i = zs_size_classes - 1; i >= 0; i--) {
@@ -1770,10 +1768,9 @@ unsigned long zs_compact(struct zs_pool *pool)
 			continue;
 		if (class->index != i)
 			continue;
-		nr_migrated += __zs_compact(pool, class);
+		__zs_compact(pool, class);
 	}
-
-	return nr_migrated;
+	return pool->num_migrated;
 }
 EXPORT_SYMBOL_GPL(zs_compact);
 
-- 
2.4.4

[RFC][PATCHv3 6/7] zsmalloc/zram: move `num_migrated' to zs_pool

[Sergey Senozhatsky]

Remove the number of migrated objects from `zs_compact_control'
and move it to `zs_pool'. `zs_compact_control' has a limited
lifespan, we lose it when zs_compaction() returns back to zram.

To keep track of objects migrated during auto-compaction (issued
by the shrinker) we need to store this number in zs_pool.

Introduce zs_get_num_migrated() to export zs_pool's ->num_migrated
counter and use it in zram, so we can also drop a zram's copy of
`num_migrated'.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 drivers/block/zram/zram_drv.c | 12 ++++++------
 drivers/block/zram/zram_drv.h |  1 -
 include/linux/zsmalloc.h      |  1 +
 mm/zsmalloc.c                 | 41 +++++++++++++++++++----------------------
 4 files changed, 26 insertions(+), 29 deletions(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index fb655e8..28ad3f8 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -388,7 +388,6 @@ static ssize_t comp_algorithm_store(struct device *dev,
 static ssize_t compact_store(struct device *dev,
 		struct device_attribute *attr, const char *buf, size_t len)
 {
-	unsigned long nr_migrated;
 	struct zram *zram = dev_to_zram(dev);
 	struct zram_meta *meta;
 
@@ -399,8 +398,7 @@ static ssize_t compact_store(struct device *dev,
 	}
 
 	meta = zram->meta;
-	nr_migrated = zs_compact(meta->mem_pool);
-	atomic64_add(nr_migrated, &zram->stats.num_migrated);
+	zs_compact(meta->mem_pool);
 	up_read(&zram->init_lock);
 
 	return len;
@@ -428,13 +426,15 @@ static ssize_t mm_stat_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
 	struct zram *zram = dev_to_zram(dev);
-	u64 orig_size, mem_used = 0;
+	u64 orig_size, mem_used = 0, num_migrated = 0;
 	long max_used;
 	ssize_t ret;
 
 	down_read(&zram->init_lock);
-	if (init_done(zram))
+	if (init_done(zram)) {
 		mem_used = zs_get_total_pages(zram->meta->mem_pool);
+		num_migrated = zs_get_num_migrated(zram->meta->mem_pool);
+	}
 
 	orig_size = atomic64_read(&zram->stats.pages_stored);
 	max_used = atomic_long_read(&zram->stats.max_used_pages);
@@ -447,7 +447,7 @@ static ssize_t mm_stat_show(struct device *dev,
 			zram->limit_pages << PAGE_SHIFT,
 			max_used << PAGE_SHIFT,
 			(u64)atomic64_read(&zram->stats.zero_pages),
-			(u64)atomic64_read(&zram->stats.num_migrated));
+			num_migrated);
 	up_read(&zram->init_lock);
 
 	return ret;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 6dbe2df..8e92339 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -78,7 +78,6 @@ struct zram_stats {
 	atomic64_t compr_data_size;	/* compressed size of pages stored */
 	atomic64_t num_reads;	/* failed + successful */
 	atomic64_t num_writes;	/* --do-- */
-	atomic64_t num_migrated;	/* no. of migrated object */
 	atomic64_t failed_reads;	/* can happen when memory is too low */
 	atomic64_t failed_writes;	/* can happen when memory is too low */
 	atomic64_t invalid_io;	/* non-page-aligned I/O requests */
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 1338190..e878875 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -47,6 +47,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle,
 void zs_unmap_object(struct zs_pool *pool, unsigned long handle);
 
 unsigned long zs_get_total_pages(struct zs_pool *pool);
+unsigned long zs_get_num_migrated(struct zs_pool *pool);
 unsigned long zs_compact(struct zs_pool *pool);
 
 #endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 5fa96ae..c9aea0a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -237,16 +237,19 @@ struct link_free {
 };
 
 struct zs_pool {
-	char *name;
+	char			*name;
 
-	struct size_class **size_class;
-	struct kmem_cache *handle_cachep;
+	struct size_class	**size_class;
+	struct kmem_cache	*handle_cachep;
 
-	gfp_t flags;	/* allocation flags used when growing pool */
-	atomic_long_t pages_allocated;
+	/* Allocation flags used when growing pool */
+	gfp_t			flags;
+	atomic_long_t		pages_allocated;
+	/* How many objects were migrated */
+	unsigned long		num_migrated;
 
 #ifdef CONFIG_ZSMALLOC_STAT
-	struct dentry *stat_dentry;
+	struct dentry		*stat_dentry;
 #endif
 };
 
@@ -1220,6 +1223,12 @@ unsigned long zs_get_total_pages(struct zs_pool *pool)
 }
 EXPORT_SYMBOL_GPL(zs_get_total_pages);
 
+unsigned long zs_get_num_migrated(struct zs_pool *pool)
+{
+	return pool->num_migrated;
+}
+EXPORT_SYMBOL_GPL(zs_get_num_migrated);
+
 /**
  * zs_map_object - get address of allocated object from handle.
  * @pool: pool from which the object was allocated
@@ -1586,8 +1595,6 @@ struct zs_compact_control {
 	 /* Starting object index within @s_page which used for live object
 	  * in the subpage. */
 	int index;
-	/* how many of objects are migrated */
-	int nr_migrated;
 };
 
 static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
@@ -1598,7 +1605,6 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 	struct page *s_page = cc->s_page;
 	struct page *d_page = cc->d_page;
 	unsigned long index = cc->index;
-	int nr_migrated = 0;
 	int ret = 0;
 
 	while (1) {
@@ -1625,13 +1631,12 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
 		record_obj(handle, free_obj);
 		unpin_tag(handle);
 		obj_free(pool, class, used_obj);
-		nr_migrated++;
+		pool->num_migrated++;
 	}
 
 	/* Remember last position in this iteration */
 	cc->s_page = s_page;
 	cc->index = index;
-	cc->nr_migrated = nr_migrated;
 
 	return ret;
 }
@@ -1706,13 +1711,11 @@ static unsigned long zs_can_compact(struct size_class *class)
 	return obj_wasted;
 }
 
-static unsigned long __zs_compact(struct zs_pool *pool,
-				struct size_class *class)
+static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 {
 	struct zs_compact_control cc;
 	struct page *src_page;
 	struct page *dst_page = NULL;
-	unsigned long nr_total_migrated = 0;
 
 	spin_lock(&class->lock);
 	while ((src_page = isolate_source_page(class))) {
@@ -1735,7 +1738,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 				break;
 
 			putback_zspage(pool, class, dst_page);
-			nr_total_migrated += cc.nr_migrated;
 		}
 
 		/* Stop if we couldn't find slot */
@@ -1745,7 +1747,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		putback_zspage(pool, class, dst_page);
 		putback_zspage(pool, class, src_page);
 		spin_unlock(&class->lock);
-		nr_total_migrated += cc.nr_migrated;
 		cond_resched();
 		spin_lock(&class->lock);
 	}
@@ -1754,14 +1755,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
 		putback_zspage(pool, class, src_page);
 
 	spin_unlock(&class->lock);
-
-	return nr_total_migrated;
 }
 
 unsigned long zs_compact(struct zs_pool *pool)
 {
 	int i;
-	unsigned long nr_migrated = 0;
 	struct size_class *class;
 
 	for (i = zs_size_classes - 1; i >= 0; i--) {
@@ -1770,10 +1768,9 @@ unsigned long zs_compact(struct zs_pool *pool)
 			continue;
 		if (class->index != i)
 			continue;
-		nr_migrated += __zs_compact(pool, class);
+		__zs_compact(pool, class);
 	}
-
-	return nr_migrated;
+	return pool->num_migrated;
 }
 EXPORT_SYMBOL_GPL(zs_compact);
 
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

Perform automatic pool compaction by a shrinker when system
is getting tight on memory.

User-space has a very little knowledge regarding zsmalloc fragmentation
and basically has no mechanism to tell whether compaction will result
in any memory gain. Another issue is that user space is not always
aware of the fact that system is getting tight on memory. Which leads
to very uncomfortable scenarios when user space may start issuing
compaction 'randomly' or from crontab (for example). Fragmentation
is not always necessarily bad, allocated and unused objects, after all,
may be filled with the data later, w/o the need of allocating a new
zspage. On the other hand, we obviously don't want to waste memory
when systems needs it.

Compaction now has a relatively quick pool scan so we are able to
estimate the number of pages that will be freed easily, which makes it
possible to call this function from a shrinker->count_objects() callback.
We also abort compaction as soon as we detect that we can't free any
pages any more, preventing wasteful objects migrations.

Minchan Kim proposed to use the shrinker (the original patch was too
aggressive and was attempting to perform compaction for every
ALMOST_EMPTY zspage).

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Suggested-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 71 insertions(+), 7 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index c9aea0a..692b7dc 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -247,7 +247,9 @@ struct zs_pool {
 	atomic_long_t		pages_allocated;
 	/* How many objects were migrated */
 	unsigned long		num_migrated;
-
+	/* Compact classes */
+	struct shrinker		shrinker;
+	bool			shrinker_enabled;
 #ifdef CONFIG_ZSMALLOC_STAT
 	struct dentry		*stat_dentry;
 #endif
@@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 
 		while ((dst_page = isolate_target_page(class))) {
 			cc.d_page = dst_page;
-			/*
-			 * If there is no more space in dst_page, resched
-			 * and see if anyone had allocated another zspage.
-			 */
+
 			if (!migrate_zspage(pool, class, &cc))
-				break;
+				goto out;
 
 			putback_zspage(pool, class, dst_page);
 		}
@@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 		cond_resched();
 		spin_lock(&class->lock);
 	}
-
+out:
+	if (dst_page)
+		putback_zspage(pool, class, dst_page);
 	if (src_page)
 		putback_zspage(pool, class, src_page);
 
@@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
 }
 EXPORT_SYMBOL_GPL(zs_compact);
 
+static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
+		struct shrink_control *sc)
+{
+	unsigned long freed;
+	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+			shrinker);
+
+	freed = pool->num_migrated;
+	/* Compact classes and calculate compaction delta */
+	freed = zs_compact(pool) - freed;
+
+	return freed ? freed : SHRINK_STOP;
+}
+
+static unsigned long zs_shrinker_count(struct shrinker *shrinker,
+		struct shrink_control *sc)
+{
+	int i;
+	struct size_class *class;
+	unsigned long to_free = 0;
+	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+			shrinker);
+
+	if (!pool->shrinker_enabled)
+		return 0;
+
+	for (i = zs_size_classes - 1; i >= 0; i--) {
+		class = pool->size_class[i];
+		if (!class)
+			continue;
+		if (class->index != i)
+			continue;
+
+		spin_lock(&class->lock);
+		to_free += zs_can_compact(class);
+		spin_unlock(&class->lock);
+	}
+
+	return to_free;
+}
+
+static void zs_unregister_shrinker(struct zs_pool *pool)
+{
+	if (pool->shrinker_enabled) {
+		unregister_shrinker(&pool->shrinker);
+		pool->shrinker_enabled = false;
+	}
+}
+
+static int zs_register_shrinker(struct zs_pool *pool)
+{
+	pool->shrinker.scan_objects = zs_shrinker_scan;
+	pool->shrinker.count_objects = zs_shrinker_count;
+	pool->shrinker.batch = 0;
+	pool->shrinker.seeks = DEFAULT_SEEKS;
+
+	return register_shrinker(&pool->shrinker);
+}
+
 /**
  * zs_create_pool - Creates an allocation pool to work from.
  * @flags: allocation flags used to allocate pool metadata
@@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
 	if (zs_pool_stat_create(name, pool))
 		goto err;
 
+	/* Not critical, we still can use the pool */
+	if (zs_register_shrinker(pool) == 0)
+		pool->shrinker_enabled = true;
 	return pool;
 
 err:
@@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
 {
 	int i;
 
+	zs_unregister_shrinker(pool);
 	zs_pool_stat_destroy(pool);
 
 	for (i = 0; i < zs_size_classes; i++) {
-- 
2.4.4

[RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

Perform automatic pool compaction by a shrinker when system
is getting tight on memory.

User-space has a very little knowledge regarding zsmalloc fragmentation
and basically has no mechanism to tell whether compaction will result
in any memory gain. Another issue is that user space is not always
aware of the fact that system is getting tight on memory. Which leads
to very uncomfortable scenarios when user space may start issuing
compaction 'randomly' or from crontab (for example). Fragmentation
is not always necessarily bad, allocated and unused objects, after all,
may be filled with the data later, w/o the need of allocating a new
zspage. On the other hand, we obviously don't want to waste memory
when systems needs it.

Compaction now has a relatively quick pool scan so we are able to
estimate the number of pages that will be freed easily, which makes it
possible to call this function from a shrinker->count_objects() callback.
We also abort compaction as soon as we detect that we can't free any
pages any more, preventing wasteful objects migrations.

Minchan Kim proposed to use the shrinker (the original patch was too
aggressive and was attempting to perform compaction for every
ALMOST_EMPTY zspage).

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Suggested-by: Minchan Kim <minchan@kernel.org>
---
 mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 71 insertions(+), 7 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index c9aea0a..692b7dc 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -247,7 +247,9 @@ struct zs_pool {
 	atomic_long_t		pages_allocated;
 	/* How many objects were migrated */
 	unsigned long		num_migrated;
-
+	/* Compact classes */
+	struct shrinker		shrinker;
+	bool			shrinker_enabled;
 #ifdef CONFIG_ZSMALLOC_STAT
 	struct dentry		*stat_dentry;
 #endif
@@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 
 		while ((dst_page = isolate_target_page(class))) {
 			cc.d_page = dst_page;
-			/*
-			 * If there is no more space in dst_page, resched
-			 * and see if anyone had allocated another zspage.
-			 */
+
 			if (!migrate_zspage(pool, class, &cc))
-				break;
+				goto out;
 
 			putback_zspage(pool, class, dst_page);
 		}
@@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 		cond_resched();
 		spin_lock(&class->lock);
 	}
-
+out:
+	if (dst_page)
+		putback_zspage(pool, class, dst_page);
 	if (src_page)
 		putback_zspage(pool, class, src_page);
 
@@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
 }
 EXPORT_SYMBOL_GPL(zs_compact);
 
+static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
+		struct shrink_control *sc)
+{
+	unsigned long freed;
+	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+			shrinker);
+
+	freed = pool->num_migrated;
+	/* Compact classes and calculate compaction delta */
+	freed = zs_compact(pool) - freed;
+
+	return freed ? freed : SHRINK_STOP;
+}
+
+static unsigned long zs_shrinker_count(struct shrinker *shrinker,
+		struct shrink_control *sc)
+{
+	int i;
+	struct size_class *class;
+	unsigned long to_free = 0;
+	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+			shrinker);
+
+	if (!pool->shrinker_enabled)
+		return 0;
+
+	for (i = zs_size_classes - 1; i >= 0; i--) {
+		class = pool->size_class[i];
+		if (!class)
+			continue;
+		if (class->index != i)
+			continue;
+
+		spin_lock(&class->lock);
+		to_free += zs_can_compact(class);
+		spin_unlock(&class->lock);
+	}
+
+	return to_free;
+}
+
+static void zs_unregister_shrinker(struct zs_pool *pool)
+{
+	if (pool->shrinker_enabled) {
+		unregister_shrinker(&pool->shrinker);
+		pool->shrinker_enabled = false;
+	}
+}
+
+static int zs_register_shrinker(struct zs_pool *pool)
+{
+	pool->shrinker.scan_objects = zs_shrinker_scan;
+	pool->shrinker.count_objects = zs_shrinker_count;
+	pool->shrinker.batch = 0;
+	pool->shrinker.seeks = DEFAULT_SEEKS;
+
+	return register_shrinker(&pool->shrinker);
+}
+
 /**
  * zs_create_pool - Creates an allocation pool to work from.
  * @flags: allocation flags used to allocate pool metadata
@@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
 	if (zs_pool_stat_create(name, pool))
 		goto err;
 
+	/* Not critical, we still can use the pool */
+	if (zs_register_shrinker(pool) == 0)
+		pool->shrinker_enabled = true;
 	return pool;
 
 err:
@@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
 {
 	int i;
 
+	zs_unregister_shrinker(pool);
 	zs_pool_stat_destroy(pool);
 
 	for (i = 0; i < zs_size_classes; i++) {
-- 
2.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

Hello,

Minchan, I didn't publish this patch separately yet, mostly to keep
the discussion in on thread. If we decide that this patch is good
enough, I'll resubmit it separately.

I did some synthetic testing. And (not surprising at all) its not so
clear. Any

I used a modified zsmalloc debug stats (to also account and report ZS_FULL
zspages). Automatic compaction was disabled.

the results are:

              almost_full         full almost_empty obj_allocated   obj_used pages_used
Base
 Total                 3          163           25          2265       1691        302
 Total                 2          161           26          2297       1688        298
 Total                 2          145           27          2396       1701        311
 Total                 3          152           26          2364       1696        312
 Total                 3          162           25          2243       1701        302

Patched
 Total                 3          155           22          2259       1691        293
 Total                 4          153           20          2177       1697        292
 Total                 2          157           23          2229       1696        298
 Total                 2          164           24          2242       1694        301
 Total                 2          159           24          2286       1696        301


Sooo... I don't know. The numbers are weird. On my x86_64 I saw somewhat
lowered 'almost_empty', 'obj_allocated', 'obj_used', 'pages_used'. But
it's a bit suspicious.

The patch was not expected to dramatically improve things anyway. It's
rather a theoretical improvement -- we sometimes keep busiest zspages first
and, at the same time, we can re-use recently used zspages.


I think it makes sense to also consider 'fullness_group fullness' in
insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.

IOW, something like this

---

 mm/zsmalloc.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 692b7dc..d576397 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
 		 * We want to see more ZS_FULL pages and less almost
 		 * empty/full. Put pages with higher ->inuse first.
 		 */
-		if (page->inuse < (*head)->inuse)
-			list_add_tail(&page->lru, &(*head)->lru);
-		else
+		if (fullness == ZS_ALMOST_FULL ||
+				(page->inuse >= (*head)->inuse))
 			list_add(&page->lru, &(*head)->lru);
+		else
+			list_add_tail(&page->lru, &(*head)->lru);
 	}
 
 	*head = page;

---

test script

modprobe zram
echo 4 > /sys/block/zram0/max_comp_streams
echo lzo > /sys/block/zram0/comp_algorithm
echo 3g > /sys/block/zram0/disksize
mkfs.ext4 /dev/zram0
mount -o relatime,defaults /dev/zram0 /zram

cd /zram/
sync

for i in {1..8192}; do
        dd if=/media/dump/down/zero_file of=/zram/$i iflag=direct bs=4K count=20 > /dev/null 2>&1
done

sync

head -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
tail -n 1 /sys/kernel/debug/zsmalloc/zram0/classes

cd /
umount /zram
rmmod zram


	-ss

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

Hello,

Minchan, I didn't publish this patch separately yet, mostly to keep
the discussion in on thread. If we decide that this patch is good
enough, I'll resubmit it separately.

I did some synthetic testing. And (not surprising at all) its not so
clear. Any

I used a modified zsmalloc debug stats (to also account and report ZS_FULL
zspages). Automatic compaction was disabled.

the results are:

              almost_full         full almost_empty obj_allocated   obj_used pages_used
Base
 Total                 3          163           25          2265       1691        302
 Total                 2          161           26          2297       1688        298
 Total                 2          145           27          2396       1701        311
 Total                 3          152           26          2364       1696        312
 Total                 3          162           25          2243       1701        302

Patched
 Total                 3          155           22          2259       1691        293
 Total                 4          153           20          2177       1697        292
 Total                 2          157           23          2229       1696        298
 Total                 2          164           24          2242       1694        301
 Total                 2          159           24          2286       1696        301


Sooo... I don't know. The numbers are weird. On my x86_64 I saw somewhat
lowered 'almost_empty', 'obj_allocated', 'obj_used', 'pages_used'. But
it's a bit suspicious.

The patch was not expected to dramatically improve things anyway. It's
rather a theoretical improvement -- we sometimes keep busiest zspages first
and, at the same time, we can re-use recently used zspages.


I think it makes sense to also consider 'fullness_group fullness' in
insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.

IOW, something like this

---

 mm/zsmalloc.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 692b7dc..d576397 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
 		 * We want to see more ZS_FULL pages and less almost
 		 * empty/full. Put pages with higher ->inuse first.
 		 */
-		if (page->inuse < (*head)->inuse)
-			list_add_tail(&page->lru, &(*head)->lru);
-		else
+		if (fullness == ZS_ALMOST_FULL ||
+				(page->inuse >= (*head)->inuse))
 			list_add(&page->lru, &(*head)->lru);
+		else
+			list_add_tail(&page->lru, &(*head)->lru);
 	}
 
 	*head = page;

---

test script

modprobe zram
echo 4 > /sys/block/zram0/max_comp_streams
echo lzo > /sys/block/zram0/comp_algorithm
echo 3g > /sys/block/zram0/disksize
mkfs.ext4 /dev/zram0
mount -o relatime,defaults /dev/zram0 /zram

cd /zram/
sync

for i in {1..8192}; do
        dd if=/media/dump/down/zero_file of=/zram/$i iflag=direct bs=4K count=20 > /dev/null 2>&1
done

sync

head -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
tail -n 1 /sys/kernel/debug/zsmalloc/zram0/classes

cd /
umount /zram
rmmod zram


	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCH v3 0/7] introduce automatic pool compaction

[Sergey Senozhatsky]

Minchan, FYI,

I re-done the test (untouched almost_empty waterline).


		Kernel log
test started [  788.064886]
test ended   [ 4025.914190]

test (doing `cp' in parallel):
(a) for i in {1..X}; do cp -R ~/git .; sync; rm -fr git/; done

# compiled kernel, with object files. 2.2G
(b) for i in {1..X}; do cp -R ~/linux/ .; sync; rm -fr linux/; done

(c) for i in {1..X}; do cp -R ~/glibc/ .; sync; rm -fr glibc/; done


Minimal si_meminfo(&si)->si.freeram observed on the system was: 6390


cat /sys/block/zram0/stat
   31253        0   250024      316 12281689        0 98253512   141263        0   141590   141763

cat /sys/block/zram0/mm_stat
3183374336 2105262569 2140762112        0 2821394432     1864   358173


The results are:

compaction nr:2335 (full:953 part:60956)

ratio: 0.01539   (~1.53% of classes were fully compacted)


More or less same numbers.


	-ss

Re: [RFC][PATCH v3 0/7] introduce automatic pool compaction

[Sergey Senozhatsky]

Minchan, FYI,

I re-done the test (untouched almost_empty waterline).


		Kernel log
test started [  788.064886]
test ended   [ 4025.914190]

test (doing `cp' in parallel):
(a) for i in {1..X}; do cp -R ~/git .; sync; rm -fr git/; done

# compiled kernel, with object files. 2.2G
(b) for i in {1..X}; do cp -R ~/linux/ .; sync; rm -fr linux/; done

(c) for i in {1..X}; do cp -R ~/glibc/ .; sync; rm -fr glibc/; done


Minimal si_meminfo(&si)->si.freeram observed on the system was: 6390


cat /sys/block/zram0/stat
   31253        0   250024      316 12281689        0 98253512   141263        0   141590   141763

cat /sys/block/zram0/mm_stat
3183374336 2105262569 2140762112        0 2821394432     1864   358173


The results are:

compaction nr:2335 (full:953 part:60956)

ratio: 0.01539   (~1.53% of classes were fully compacted)


More or less same numbers.


	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I think it makes sense to also consider 'fullness_group fullness' in
> insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
> head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
> 
> IOW, something like this
> 
> ---
> 
>  mm/zsmalloc.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 692b7dc..d576397 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		 * We want to see more ZS_FULL pages and less almost
>  		 * empty/full. Put pages with higher ->inuse first.
>  		 */
> -		if (page->inuse < (*head)->inuse)
> -			list_add_tail(&page->lru, &(*head)->lru);
> -		else
> +		if (fullness == ZS_ALMOST_FULL ||
> +				(page->inuse >= (*head)->inuse))
>  			list_add(&page->lru, &(*head)->lru);
> +		else
> +			list_add_tail(&page->lru, &(*head)->lru);
>  	}
>  
>  	*head = page;
> 


             almost_full         full almost_empty obj_allocated   obj_used pages_used

Base
 Total                 3          168           26          2324       1822        307
 Total                 3          167           29          2391       1815        314
 Total                 5          172           25          2392       1827        313

Patched
 Total                 4          180           27          2425       1835        327
 Total                 4          169           27          2405       1825        312
 Total                 2          176           28          2452       1825        315


... no chance the test is right.

	-ss

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I think it makes sense to also consider 'fullness_group fullness' in
> insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
> head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
> 
> IOW, something like this
> 
> ---
> 
>  mm/zsmalloc.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 692b7dc..d576397 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		 * We want to see more ZS_FULL pages and less almost
>  		 * empty/full. Put pages with higher ->inuse first.
>  		 */
> -		if (page->inuse < (*head)->inuse)
> -			list_add_tail(&page->lru, &(*head)->lru);
> -		else
> +		if (fullness == ZS_ALMOST_FULL ||
> +				(page->inuse >= (*head)->inuse))
>  			list_add(&page->lru, &(*head)->lru);
> +		else
> +			list_add_tail(&page->lru, &(*head)->lru);
>  	}
>  
>  	*head = page;
> 


             almost_full         full almost_empty obj_allocated   obj_used pages_used

Base
 Total                 3          168           26          2324       1822        307
 Total                 3          167           29          2391       1815        314
 Total                 5          172           25          2392       1827        313

Patched
 Total                 4          180           27          2425       1835        327
 Total                 4          169           27          2405       1825        312
 Total                 2          176           28          2452       1825        315


... no chance the test is right.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I used a modified zsmalloc debug stats (to also account and report ZS_FULL
> zspages).

just in case. account and report per-class ZS_FULL numbers.

----

 mm/zsmalloc.c | 29 ++++++++++++++++++++---------
 1 file changed, 20 insertions(+), 9 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e0d28a2..97ca25d 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -166,6 +166,7 @@ enum zs_stat_type {
 	OBJ_USED,
 	CLASS_ALMOST_FULL,
 	CLASS_ALMOST_EMPTY,
+	CLASS_FULL,
 	NR_ZS_STAT_TYPE,
 };
 
@@ -483,13 +484,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 	struct zs_pool *pool = s->private;
 	struct size_class *class;
 	int objs_per_zspage;
-	unsigned long class_almost_full, class_almost_empty;
+	unsigned long class_almost_full, class_full, class_almost_empty;
 	unsigned long obj_allocated, obj_used, pages_used;
-	unsigned long total_class_almost_full = 0, total_class_almost_empty = 0;
+	unsigned long total_class_almost_full = 0, total_class_full = 0, total_class_almost_empty = 0;
 	unsigned long total_objs = 0, total_used_objs = 0, total_pages = 0;
 
-	seq_printf(s, " %5s %5s %11s %12s %13s %10s %10s %16s\n",
-			"class", "size", "almost_full", "almost_empty",
+	seq_printf(s, " %5s %5s %11s %12s %12s %13s %10s %10s %16s\n",
+			"class", "size", "almost_full", "full", "almost_empty",
 			"obj_allocated", "obj_used", "pages_used",
 			"pages_per_zspage");
 
@@ -501,6 +502,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 
 		spin_lock(&class->lock);
 		class_almost_full = zs_stat_get(class, CLASS_ALMOST_FULL);
+		class_full = zs_stat_get(class, CLASS_FULL);
 		class_almost_empty = zs_stat_get(class, CLASS_ALMOST_EMPTY);
 		obj_allocated = zs_stat_get(class, OBJ_ALLOCATED);
 		obj_used = zs_stat_get(class, OBJ_USED);
@@ -511,12 +513,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 		pages_used = obj_allocated / objs_per_zspage *
 				class->pages_per_zspage;
 
-		seq_printf(s, " %5u %5u %11lu %12lu %13lu %10lu %10lu %16d\n",
-			i, class->size, class_almost_full, class_almost_empty,
+		seq_printf(s, " %5u %5u %11lu %12lu %12lu %13lu %10lu %10lu %16d\n",
+			i, class->size, class_almost_full, class_full, class_almost_empty,
 			obj_allocated, obj_used, pages_used,
 			class->pages_per_zspage);
 
 		total_class_almost_full += class_almost_full;
+		total_class_full += class_full;
 		total_class_almost_empty += class_almost_empty;
 		total_objs += obj_allocated;
 		total_used_objs += obj_used;
@@ -524,8 +527,9 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 	}
 
 	seq_puts(s, "\n");
-	seq_printf(s, " %5s %5s %11lu %12lu %13lu %10lu %10lu\n",
+	seq_printf(s, " %5s %5s %11lu %12lu %12lu %13lu %10lu %10lu\n",
 			"Total", "", total_class_almost_full,
+			total_class_full,
 			total_class_almost_empty, total_objs,
 			total_used_objs, total_pages);
 
@@ -652,7 +656,10 @@ static void insert_zspage(struct page *page, struct size_class *class,
 	}
 
 	*head = page;
-	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+	if (fullness == ZS_FULL)
+		zs_stat_inc(class, CLASS_FULL, 1);
+	else
+		zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
 			CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
 }
 
@@ -679,7 +686,10 @@ static void remove_zspage(struct page *page, struct size_class *class,
 					struct page, lru);
 
 	list_del_init(&page->lru);
-	zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
+	if (fullness == ZS_FULL)
+		zs_stat_dec(class, CLASS_FULL, 1);
+	else
+		zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
 			CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
 }
 
@@ -1410,6 +1420,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size)
 		spin_lock(&class->lock);
 		zs_stat_inc(class, OBJ_ALLOCATED, get_maxobj_per_zspage(
 				class->size, class->pages_per_zspage));
+		zs_stat_inc(class, CLASS_FULL, 1);
 	}
 
 	obj = obj_malloc(first_page, class, handle);

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I used a modified zsmalloc debug stats (to also account and report ZS_FULL
> zspages).

just in case. account and report per-class ZS_FULL numbers.

----

 mm/zsmalloc.c | 29 ++++++++++++++++++++---------
 1 file changed, 20 insertions(+), 9 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e0d28a2..97ca25d 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -166,6 +166,7 @@ enum zs_stat_type {
 	OBJ_USED,
 	CLASS_ALMOST_FULL,
 	CLASS_ALMOST_EMPTY,
+	CLASS_FULL,
 	NR_ZS_STAT_TYPE,
 };
 
@@ -483,13 +484,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 	struct zs_pool *pool = s->private;
 	struct size_class *class;
 	int objs_per_zspage;
-	unsigned long class_almost_full, class_almost_empty;
+	unsigned long class_almost_full, class_full, class_almost_empty;
 	unsigned long obj_allocated, obj_used, pages_used;
-	unsigned long total_class_almost_full = 0, total_class_almost_empty = 0;
+	unsigned long total_class_almost_full = 0, total_class_full = 0, total_class_almost_empty = 0;
 	unsigned long total_objs = 0, total_used_objs = 0, total_pages = 0;
 
-	seq_printf(s, " %5s %5s %11s %12s %13s %10s %10s %16s\n",
-			"class", "size", "almost_full", "almost_empty",
+	seq_printf(s, " %5s %5s %11s %12s %12s %13s %10s %10s %16s\n",
+			"class", "size", "almost_full", "full", "almost_empty",
 			"obj_allocated", "obj_used", "pages_used",
 			"pages_per_zspage");
 
@@ -501,6 +502,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 
 		spin_lock(&class->lock);
 		class_almost_full = zs_stat_get(class, CLASS_ALMOST_FULL);
+		class_full = zs_stat_get(class, CLASS_FULL);
 		class_almost_empty = zs_stat_get(class, CLASS_ALMOST_EMPTY);
 		obj_allocated = zs_stat_get(class, OBJ_ALLOCATED);
 		obj_used = zs_stat_get(class, OBJ_USED);
@@ -511,12 +513,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 		pages_used = obj_allocated / objs_per_zspage *
 				class->pages_per_zspage;
 
-		seq_printf(s, " %5u %5u %11lu %12lu %13lu %10lu %10lu %16d\n",
-			i, class->size, class_almost_full, class_almost_empty,
+		seq_printf(s, " %5u %5u %11lu %12lu %12lu %13lu %10lu %10lu %16d\n",
+			i, class->size, class_almost_full, class_full, class_almost_empty,
 			obj_allocated, obj_used, pages_used,
 			class->pages_per_zspage);
 
 		total_class_almost_full += class_almost_full;
+		total_class_full += class_full;
 		total_class_almost_empty += class_almost_empty;
 		total_objs += obj_allocated;
 		total_used_objs += obj_used;
@@ -524,8 +527,9 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
 	}
 
 	seq_puts(s, "\n");
-	seq_printf(s, " %5s %5s %11lu %12lu %13lu %10lu %10lu\n",
+	seq_printf(s, " %5s %5s %11lu %12lu %12lu %13lu %10lu %10lu\n",
 			"Total", "", total_class_almost_full,
+			total_class_full,
 			total_class_almost_empty, total_objs,
 			total_used_objs, total_pages);
 
@@ -652,7 +656,10 @@ static void insert_zspage(struct page *page, struct size_class *class,
 	}
 
 	*head = page;
-	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+	if (fullness == ZS_FULL)
+		zs_stat_inc(class, CLASS_FULL, 1);
+	else
+		zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
 			CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
 }
 
@@ -679,7 +686,10 @@ static void remove_zspage(struct page *page, struct size_class *class,
 					struct page, lru);
 
 	list_del_init(&page->lru);
-	zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
+	if (fullness == ZS_FULL)
+		zs_stat_dec(class, CLASS_FULL, 1);
+	else
+		zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
 			CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
 }
 
@@ -1410,6 +1420,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size)
 		spin_lock(&class->lock);
 		zs_stat_inc(class, OBJ_ALLOCATED, get_maxobj_per_zspage(
 				class->size, class->pages_per_zspage));
+		zs_stat_inc(class, CLASS_FULL, 1);
 	}
 
 	obj = obj_malloc(first_page, class, handle);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		 * We want to see more ZS_FULL pages and less almost
>  		 * empty/full. Put pages with higher ->inuse first.
>  		 */
> -		if (page->inuse < (*head)->inuse)
> -			list_add_tail(&page->lru, &(*head)->lru);
> -		else
> +		if (fullness == ZS_ALMOST_FULL ||
> +				(page->inuse >= (*head)->inuse))
>  			list_add(&page->lru, &(*head)->lru);
> +		else
> +			list_add_tail(&page->lru, &(*head)->lru);
>  	}
>  
>  	*head = page;

oh, dear. what I was thinking of. this is just stupid. please ignore
this part.

	-ss

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

On (06/18/15 21:13), Sergey Senozhatsky wrote:
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		 * We want to see more ZS_FULL pages and less almost
>  		 * empty/full. Put pages with higher ->inuse first.
>  		 */
> -		if (page->inuse < (*head)->inuse)
> -			list_add_tail(&page->lru, &(*head)->lru);
> -		else
> +		if (fullness == ZS_ALMOST_FULL ||
> +				(page->inuse >= (*head)->inuse))
>  			list_add(&page->lru, &(*head)->lru);
> +		else
> +			list_add_tail(&page->lru, &(*head)->lru);
>  	}
>  
>  	*head = page;

oh, dear. what I was thinking of. this is just stupid. please ignore
this part.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> Always account per-class `zs_size_stat' stats. This data will
> help us make better decisions during compaction. We are especially
> interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> class compaction will result in any memory gain.
> 
> For instance, we know the number of allocated objects in the class,
> the number of objects being used (so we also know how many objects
> are not used) and the number of objects per-page. So we can ensure
> if we have enough unused objects to form at least one ZS_EMPTY
> zspage during compaction.
> 
> We calculate this value on per-class basis so we can calculate a
> total number of zspages that can be released. Which is exactly what
> a shrinker wants to know.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>

Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> Always account per-class `zs_size_stat' stats. This data will
> help us make better decisions during compaction. We are especially
> interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> class compaction will result in any memory gain.
> 
> For instance, we know the number of allocated objects in the class,
> the number of objects being used (so we also know how many objects
> are not used) and the number of objects per-page. So we can ensure
> if we have enough unused objects to form at least one ZS_EMPTY
> zspage during compaction.
> 
> We calculate this value on per-class basis so we can calculate a
> total number of zspages that can be released. Which is exactly what
> a shrinker wants to know.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:41PM +0900, Sergey Senozhatsky wrote:
> This function checks if class compaction will free any pages.
> Rephrasing -- do we have enough unused objects to form at least
> one ZS_EMPTY page and free it. It aborts compaction if class
> compaction will not result in any (further) savings.
> 
> EXAMPLE (this debug output is not part of this patch set):
> 
> -- class size
> -- number of allocated objects
> -- number of used objects
> -- max objects per zspage
> -- pages per zspage
> -- estimated number of pages that will be freed
> 
> [..]
> class-512 objs:544 inuse:540 maxobj-per-zspage:8  pages-per-zspage:1 zspages-to-free:0
>  ... class-512 compaction is useless. break
> class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
> class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
> class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
>  ... class-496 compaction is useless. break
> class-448 objs:657 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:4
> class-448 objs:648 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:3
> class-448 objs:639 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:2
> class-448 objs:630 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:1
> class-448 objs:621 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:0
>  ... class-448 compaction is useless. break
> class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
> class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
>  ... class-432 compaction is useless. break
> class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
> class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
> class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
>  ... class-416 compaction is useless. break
> class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
> class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
>  ... class-400 compaction is useless. break
> class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
>  ... class-384 compaction is useless. break
> [..]
> 
> Every "compaction is useless" indicates that we saved CPU cycles.
> 
> class-512 has
> 	544	object allocated
> 	540	objects used
> 	8	objects per-page
> 
> Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
> migrate all of its objects and free this zspage; so compaction will not
> make a lot of sense, it's better to just leave it as is.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 4b6f12e..18e10a4 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
>  	return page;
>  }
>  
> +/*
> + * Make sure that we actually can compact this class,
> + * IOW if migration will release at least one szpage.

                                                 zspage,

> + *
> + * Should be called under class->lock

Please comment about return.

> + */
> +static unsigned long zs_can_compact(struct size_class *class)
> +{
> +	/*
> +	 * Calculate how many unused allocated objects we
> +	 * have and see if we can free any zspages. Otherwise,
> +	 * compaction can just move objects back and forth w/o
> +	 * any memory gain.
> +	 */
> +	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> +		zs_stat_get(class, OBJ_USED);
> +

I want to check one more thing.

We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
In this implementation, compaction cannot have a source so
it would better to bail out.
IOW,

      if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
              return 0;

> +	obj_wasted /= get_maxobj_per_zspage(class->size,
> +			class->pages_per_zspage);
> +	return obj_wasted;
> +}
> +
>  static unsigned long __zs_compact(struct zs_pool *pool,
>  				struct size_class *class)
>  {
> @@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
>  
>  		BUG_ON(!is_first_page(src_page));
>  
> +		if (!zs_can_compact(class))
> +			break;
> +
>  		cc.index = 0;
>  		cc.s_page = src_page;
>  
> -- 
> 2.4.4
> 

Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:41PM +0900, Sergey Senozhatsky wrote:
> This function checks if class compaction will free any pages.
> Rephrasing -- do we have enough unused objects to form at least
> one ZS_EMPTY page and free it. It aborts compaction if class
> compaction will not result in any (further) savings.
> 
> EXAMPLE (this debug output is not part of this patch set):
> 
> -- class size
> -- number of allocated objects
> -- number of used objects
> -- max objects per zspage
> -- pages per zspage
> -- estimated number of pages that will be freed
> 
> [..]
> class-512 objs:544 inuse:540 maxobj-per-zspage:8  pages-per-zspage:1 zspages-to-free:0
>  ... class-512 compaction is useless. break
> class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
> class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
> class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
>  ... class-496 compaction is useless. break
> class-448 objs:657 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:4
> class-448 objs:648 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:3
> class-448 objs:639 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:2
> class-448 objs:630 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:1
> class-448 objs:621 inuse:617 maxobj-per-zspage:9  pages-per-zspage:1 zspages-to-free:0
>  ... class-448 compaction is useless. break
> class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
> class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
>  ... class-432 compaction is useless. break
> class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
> class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
> class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
>  ... class-416 compaction is useless. break
> class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
> class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
>  ... class-400 compaction is useless. break
> class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
>  ... class-384 compaction is useless. break
> [..]
> 
> Every "compaction is useless" indicates that we saved CPU cycles.
> 
> class-512 has
> 	544	object allocated
> 	540	objects used
> 	8	objects per-page
> 
> Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
> migrate all of its objects and free this zspage; so compaction will not
> make a lot of sense, it's better to just leave it as is.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 4b6f12e..18e10a4 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
>  	return page;
>  }
>  
> +/*
> + * Make sure that we actually can compact this class,
> + * IOW if migration will release at least one szpage.

                                                 zspage,

> + *
> + * Should be called under class->lock

Please comment about return.

> + */
> +static unsigned long zs_can_compact(struct size_class *class)
> +{
> +	/*
> +	 * Calculate how many unused allocated objects we
> +	 * have and see if we can free any zspages. Otherwise,
> +	 * compaction can just move objects back and forth w/o
> +	 * any memory gain.
> +	 */
> +	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> +		zs_stat_get(class, OBJ_USED);
> +

I want to check one more thing.

We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
In this implementation, compaction cannot have a source so
it would better to bail out.
IOW,

      if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
              return 0;

> +	obj_wasted /= get_maxobj_per_zspage(class->size,
> +			class->pages_per_zspage);
> +	return obj_wasted;
> +}
> +
>  static unsigned long __zs_compact(struct zs_pool *pool,
>  				struct size_class *class)
>  {
> @@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
>  
>  		BUG_ON(!is_first_page(src_page));
>  
> +		if (!zs_can_compact(class))
> +			break;
> +
>  		cc.index = 0;
>  		cc.s_page = src_page;
>  
> -- 
> 2.4.4
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:39PM +0900, Sergey Senozhatsky wrote:
> We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
> pages. Put a page with higher ->inuse count first within its
> ->fullness_list, which will give us better chances to fill up this
> page with new objects (find_get_zspage() return ->fullness_list head
> for new object allocation), so some zspages will become
> ZS_ALMOST_FULL/ZS_FULL quicker.
> 
> It performs a trivial and cheap ->inuse compare which does not slow
> down zsmalloc, and in the worst case it keeps the list pages not in
> any particular order, just like we do it now.
> 
> A more expensive solution could sort fullness_list by ->inuse count.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 7d816c2..6e2ebb6 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		return;
>  
>  	head = &class->fullness_list[fullness];
> -	if (*head)
> -		list_add_tail(&page->lru, &(*head)->lru);
> +	if (*head) {
> +		/*
> +		 * We want to see more ZS_FULL pages and less almost
> +		 * empty/full. Put pages with higher ->inuse first.
> +		 */
> +		if (page->inuse < (*head)->inuse)
> +			list_add_tail(&page->lru, &(*head)->lru);
> +		else
> +			list_add(&page->lru, &(*head)->lru);
> +	}

>  
>  	*head = page;

Why do you want to always put @page in the head?
How about this?

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e8cb31c..1c5fde9 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
        if (fullness >= _ZS_NR_FULLNESS_GROUPS)
                return;

+       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+
        head = &class->fullness_list[fullness];
-       if (*head) {
-               /*
-                * We want to see more ZS_FULL pages and less almost
-                * empty/full. Put pages with higher ->inuse first.
-                */
-               if (page->inuse < (*head)->inuse)
-                       list_add_tail(&page->lru, &(*head)->lru);
-               else
-                       list_add(&page->lru, &(*head)->lru);
+       if (!*head) {
+               *head = page;
+               return;
        }

-       *head = page;
-       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
-                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+       /*
+        * We want to see more ZS_FULL pages and less almost
+        * empty/full. Put pages with higher ->inuse first.
+        */
+       list_add_tail(&page->lru, &(*head)->lru);
+       if (page->inuse >= (*head)->inuse)
+               *head = page;
 }

 /*
-- 
1.7.9.5




>  	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> -- 
> 2.4.4
> 

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:39PM +0900, Sergey Senozhatsky wrote:
> We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
> pages. Put a page with higher ->inuse count first within its
> ->fullness_list, which will give us better chances to fill up this
> page with new objects (find_get_zspage() return ->fullness_list head
> for new object allocation), so some zspages will become
> ZS_ALMOST_FULL/ZS_FULL quicker.
> 
> It performs a trivial and cheap ->inuse compare which does not slow
> down zsmalloc, and in the worst case it keeps the list pages not in
> any particular order, just like we do it now.
> 
> A more expensive solution could sort fullness_list by ->inuse count.
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 7d816c2..6e2ebb6 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
>  		return;
>  
>  	head = &class->fullness_list[fullness];
> -	if (*head)
> -		list_add_tail(&page->lru, &(*head)->lru);
> +	if (*head) {
> +		/*
> +		 * We want to see more ZS_FULL pages and less almost
> +		 * empty/full. Put pages with higher ->inuse first.
> +		 */
> +		if (page->inuse < (*head)->inuse)
> +			list_add_tail(&page->lru, &(*head)->lru);
> +		else
> +			list_add(&page->lru, &(*head)->lru);
> +	}

>  
>  	*head = page;

Why do you want to always put @page in the head?
How about this?

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e8cb31c..1c5fde9 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
        if (fullness >= _ZS_NR_FULLNESS_GROUPS)
                return;

+       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+
        head = &class->fullness_list[fullness];
-       if (*head) {
-               /*
-                * We want to see more ZS_FULL pages and less almost
-                * empty/full. Put pages with higher ->inuse first.
-                */
-               if (page->inuse < (*head)->inuse)
-                       list_add_tail(&page->lru, &(*head)->lru);
-               else
-                       list_add(&page->lru, &(*head)->lru);
+       if (!*head) {
+               *head = page;
+               return;
        }

-       *head = page;
-       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
-                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+       /*
+        * We want to see more ZS_FULL pages and less almost
+        * empty/full. Put pages with higher ->inuse first.
+        */
+       list_add_tail(&page->lru, &(*head)->lru);
+       if (page->inuse >= (*head)->inuse)
+               *head = page;
 }

 /*
-- 
1.7.9.5




>  	zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> -- 
> 2.4.4
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:44PM +0900, Sergey Senozhatsky wrote:
> Perform automatic pool compaction by a shrinker when system
> is getting tight on memory.
> 
> User-space has a very little knowledge regarding zsmalloc fragmentation
> and basically has no mechanism to tell whether compaction will result
> in any memory gain. Another issue is that user space is not always
> aware of the fact that system is getting tight on memory. Which leads
> to very uncomfortable scenarios when user space may start issuing
> compaction 'randomly' or from crontab (for example). Fragmentation
> is not always necessarily bad, allocated and unused objects, after all,
> may be filled with the data later, w/o the need of allocating a new
> zspage. On the other hand, we obviously don't want to waste memory
> when systems needs it.
> 
> Compaction now has a relatively quick pool scan so we are able to
> estimate the number of pages that will be freed easily, which makes it
> possible to call this function from a shrinker->count_objects() callback.
> We also abort compaction as soon as we detect that we can't free any
> pages any more, preventing wasteful objects migrations.
> 
> Minchan Kim proposed to use the shrinker (the original patch was too
> aggressive and was attempting to perform compaction for every
> ALMOST_EMPTY zspage).
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Suggested-by: Minchan Kim <minchan@kernel.org>
> ---
>  mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 71 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index c9aea0a..692b7dc 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -247,7 +247,9 @@ struct zs_pool {
>  	atomic_long_t		pages_allocated;
>  	/* How many objects were migrated */
>  	unsigned long		num_migrated;
> -
> +	/* Compact classes */
> +	struct shrinker		shrinker;
> +	bool			shrinker_enabled;
>  #ifdef CONFIG_ZSMALLOC_STAT
>  	struct dentry		*stat_dentry;
>  #endif
> @@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>  
>  		while ((dst_page = isolate_target_page(class))) {
>  			cc.d_page = dst_page;
> -			/*
> -			 * If there is no more space in dst_page, resched
> -			 * and see if anyone had allocated another zspage.
> -			 */
> +
>  			if (!migrate_zspage(pool, class, &cc))
> -				break;
> +				goto out;

It should retry with another target_page instead of going out.

>  
>  			putback_zspage(pool, class, dst_page);
>  		}
> @@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>  		cond_resched();
>  		spin_lock(&class->lock);
>  	}
> -
> +out:
> +	if (dst_page)
> +		putback_zspage(pool, class, dst_page);
>  	if (src_page)
>  		putback_zspage(pool, class, src_page);
>  
> @@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
>  }
>  EXPORT_SYMBOL_GPL(zs_compact);
>  
> +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> +		struct shrink_control *sc)
> +{
> +	unsigned long freed;
> +	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> +			shrinker);
> +
> +	freed = pool->num_migrated;
> +	/* Compact classes and calculate compaction delta */
> +	freed = zs_compact(pool) - freed;

Returns migrated object count.

> +
> +	return freed ? freed : SHRINK_STOP;
> +}
> +
> +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> +		struct shrink_control *sc)
> +{
> +	int i;
> +	struct size_class *class;
> +	unsigned long to_free = 0;
> +	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> +			shrinker);
> +
> +	if (!pool->shrinker_enabled)
> +		return 0;
> +
> +	for (i = zs_size_classes - 1; i >= 0; i--) {
> +		class = pool->size_class[i];
> +		if (!class)
> +			continue;
> +		if (class->index != i)
> +			continue;
> +
> +		spin_lock(&class->lock);
> +		to_free += zs_can_compact(class);

But it returns wasted_obj / max_obj_per_zspage?

> +		spin_unlock(&class->lock);
> +	}
> +
> +	return to_free;
> +}
> +
> +static void zs_unregister_shrinker(struct zs_pool *pool)
> +{
> +	if (pool->shrinker_enabled) {
> +		unregister_shrinker(&pool->shrinker);
> +		pool->shrinker_enabled = false;
> +	}
> +}
> +
> +static int zs_register_shrinker(struct zs_pool *pool)
> +{
> +	pool->shrinker.scan_objects = zs_shrinker_scan;
> +	pool->shrinker.count_objects = zs_shrinker_count;
> +	pool->shrinker.batch = 0;
> +	pool->shrinker.seeks = DEFAULT_SEEKS;
> +
> +	return register_shrinker(&pool->shrinker);
> +}
> +
>  /**
>   * zs_create_pool - Creates an allocation pool to work from.
>   * @flags: allocation flags used to allocate pool metadata
> @@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
>  	if (zs_pool_stat_create(name, pool))
>  		goto err;
>  
> +	/* Not critical, we still can use the pool */
> +	if (zs_register_shrinker(pool) == 0)
> +		pool->shrinker_enabled = true;
>  	return pool;
>  
>  err:
> @@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
>  {
>  	int i;
>  
> +	zs_unregister_shrinker(pool);
>  	zs_pool_stat_destroy(pool);
>  
>  	for (i = 0; i < zs_size_classes; i++) {
> -- 
> 2.4.4
> 

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Minchan Kim]

On Thu, Jun 18, 2015 at 08:46:44PM +0900, Sergey Senozhatsky wrote:
> Perform automatic pool compaction by a shrinker when system
> is getting tight on memory.
> 
> User-space has a very little knowledge regarding zsmalloc fragmentation
> and basically has no mechanism to tell whether compaction will result
> in any memory gain. Another issue is that user space is not always
> aware of the fact that system is getting tight on memory. Which leads
> to very uncomfortable scenarios when user space may start issuing
> compaction 'randomly' or from crontab (for example). Fragmentation
> is not always necessarily bad, allocated and unused objects, after all,
> may be filled with the data later, w/o the need of allocating a new
> zspage. On the other hand, we obviously don't want to waste memory
> when systems needs it.
> 
> Compaction now has a relatively quick pool scan so we are able to
> estimate the number of pages that will be freed easily, which makes it
> possible to call this function from a shrinker->count_objects() callback.
> We also abort compaction as soon as we detect that we can't free any
> pages any more, preventing wasteful objects migrations.
> 
> Minchan Kim proposed to use the shrinker (the original patch was too
> aggressive and was attempting to perform compaction for every
> ALMOST_EMPTY zspage).
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Suggested-by: Minchan Kim <minchan@kernel.org>
> ---
>  mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 71 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index c9aea0a..692b7dc 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -247,7 +247,9 @@ struct zs_pool {
>  	atomic_long_t		pages_allocated;
>  	/* How many objects were migrated */
>  	unsigned long		num_migrated;
> -
> +	/* Compact classes */
> +	struct shrinker		shrinker;
> +	bool			shrinker_enabled;
>  #ifdef CONFIG_ZSMALLOC_STAT
>  	struct dentry		*stat_dentry;
>  #endif
> @@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>  
>  		while ((dst_page = isolate_target_page(class))) {
>  			cc.d_page = dst_page;
> -			/*
> -			 * If there is no more space in dst_page, resched
> -			 * and see if anyone had allocated another zspage.
> -			 */
> +
>  			if (!migrate_zspage(pool, class, &cc))
> -				break;
> +				goto out;

It should retry with another target_page instead of going out.

>  
>  			putback_zspage(pool, class, dst_page);
>  		}
> @@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>  		cond_resched();
>  		spin_lock(&class->lock);
>  	}
> -
> +out:
> +	if (dst_page)
> +		putback_zspage(pool, class, dst_page);
>  	if (src_page)
>  		putback_zspage(pool, class, src_page);
>  
> @@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
>  }
>  EXPORT_SYMBOL_GPL(zs_compact);
>  
> +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> +		struct shrink_control *sc)
> +{
> +	unsigned long freed;
> +	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> +			shrinker);
> +
> +	freed = pool->num_migrated;
> +	/* Compact classes and calculate compaction delta */
> +	freed = zs_compact(pool) - freed;

Returns migrated object count.

> +
> +	return freed ? freed : SHRINK_STOP;
> +}
> +
> +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> +		struct shrink_control *sc)
> +{
> +	int i;
> +	struct size_class *class;
> +	unsigned long to_free = 0;
> +	struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> +			shrinker);
> +
> +	if (!pool->shrinker_enabled)
> +		return 0;
> +
> +	for (i = zs_size_classes - 1; i >= 0; i--) {
> +		class = pool->size_class[i];
> +		if (!class)
> +			continue;
> +		if (class->index != i)
> +			continue;
> +
> +		spin_lock(&class->lock);
> +		to_free += zs_can_compact(class);

But it returns wasted_obj / max_obj_per_zspage?

> +		spin_unlock(&class->lock);
> +	}
> +
> +	return to_free;
> +}
> +
> +static void zs_unregister_shrinker(struct zs_pool *pool)
> +{
> +	if (pool->shrinker_enabled) {
> +		unregister_shrinker(&pool->shrinker);
> +		pool->shrinker_enabled = false;
> +	}
> +}
> +
> +static int zs_register_shrinker(struct zs_pool *pool)
> +{
> +	pool->shrinker.scan_objects = zs_shrinker_scan;
> +	pool->shrinker.count_objects = zs_shrinker_count;
> +	pool->shrinker.batch = 0;
> +	pool->shrinker.seeks = DEFAULT_SEEKS;
> +
> +	return register_shrinker(&pool->shrinker);
> +}
> +
>  /**
>   * zs_create_pool - Creates an allocation pool to work from.
>   * @flags: allocation flags used to allocate pool metadata
> @@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
>  	if (zs_pool_stat_create(name, pool))
>  		goto err;
>  
> +	/* Not critical, we still can use the pool */
> +	if (zs_register_shrinker(pool) == 0)
> +		pool->shrinker_enabled = true;
>  	return pool;
>  
>  err:
> @@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
>  {
>  	int i;
>  
> +	zs_unregister_shrinker(pool);
>  	zs_pool_stat_destroy(pool);
>  
>  	for (i = 0; i < zs_size_classes; i++) {
> -- 
> 2.4.4
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

Hello,

thanks for review.

On (06/29/15 16:07), Minchan Kim wrote:
[..]
> >  			if (!migrate_zspage(pool, class, &cc))
> > -				break;
> > +				goto out;
> 
> It should retry with another target_page instead of going out.

yep.

[..]
> > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > +		struct shrink_control *sc)
> > +{
[..]
> 
> Returns migrated object count.
> 
[..]
> > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > +		struct shrink_control *sc)
> > +{
[..]
> 
> But it returns wasted_obj / max_obj_per_zspage?
> 

Good catch.
So,  __zs_compact() and zs_shrinker_count() are ok. Returning
"wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
sense there. The only place is zs_shrinker_scan()->zs_compact().

Hm, I can think of:

(a) We can change zs_compact() to return the total number of
freed zspages. That will not really change a user visible
interface. We export (fwiw) the number of compacted objects
in mm_stat. Basically, this is internal zsmalloc() counter and
no user space program can ever do anything with that data. From
that prospective we will just replace one senseless number with
another (equally meaningless) one.


(b) replace zs_compact() call in zs_shrinker_scan() with a class loop

1764         int i;
1765         unsigned long nr_migrated = 0;
1766         struct size_class *class;
1767
1768         for (i = zs_size_classes - 1; i >= 0; i--) {
1769                 class = pool->size_class[i];
1770                 if (!class)
1771                         continue;
1772                 if (class->index != i)
1773                         continue;
1774                 nr_migrated += __zs_compact(pool, class);
1775         }
1776
1777         return nr_migrated;

But on every iteration increment nr_migrated with
		"nr_migrated += just_migrated / max_obj_per_zspage"

(which will be unnecessary if zs_compact() will return the number of freed
zspages).

So, (b) is mostly fine, except that we already have several pool->size_class
loops, with same `if (!class)' and `if (class->index...)' checks; and it
asks for some sort of refactoring or... a tricky for_each_class() define.


In both cases, however, we don't tell anything valuable to user space.
Thus,

(c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
And change compaction to operate in terms of pages (PAGE_SIZE). At
least mm_stat::compacted will turn into something useful for user
space.

	-ss

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

Hello,

thanks for review.

On (06/29/15 16:07), Minchan Kim wrote:
[..]
> >  			if (!migrate_zspage(pool, class, &cc))
> > -				break;
> > +				goto out;
> 
> It should retry with another target_page instead of going out.

yep.

[..]
> > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > +		struct shrink_control *sc)
> > +{
[..]
> 
> Returns migrated object count.
> 
[..]
> > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > +		struct shrink_control *sc)
> > +{
[..]
> 
> But it returns wasted_obj / max_obj_per_zspage?
> 

Good catch.
So,  __zs_compact() and zs_shrinker_count() are ok. Returning
"wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
sense there. The only place is zs_shrinker_scan()->zs_compact().

Hm, I can think of:

(a) We can change zs_compact() to return the total number of
freed zspages. That will not really change a user visible
interface. We export (fwiw) the number of compacted objects
in mm_stat. Basically, this is internal zsmalloc() counter and
no user space program can ever do anything with that data. From
that prospective we will just replace one senseless number with
another (equally meaningless) one.


(b) replace zs_compact() call in zs_shrinker_scan() with a class loop

1764         int i;
1765         unsigned long nr_migrated = 0;
1766         struct size_class *class;
1767
1768         for (i = zs_size_classes - 1; i >= 0; i--) {
1769                 class = pool->size_class[i];
1770                 if (!class)
1771                         continue;
1772                 if (class->index != i)
1773                         continue;
1774                 nr_migrated += __zs_compact(pool, class);
1775         }
1776
1777         return nr_migrated;

But on every iteration increment nr_migrated with
		"nr_migrated += just_migrated / max_obj_per_zspage"

(which will be unnecessary if zs_compact() will return the number of freed
zspages).

So, (b) is mostly fine, except that we already have several pool->size_class
loops, with same `if (!class)' and `if (class->index...)' checks; and it
asks for some sort of refactoring or... a tricky for_each_class() define.


In both cases, however, we don't tell anything valuable to user space.
Thus,

(c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
And change compaction to operate in terms of pages (PAGE_SIZE). At
least mm_stat::compacted will turn into something useful for user
space.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Sergey Senozhatsky]

On (06/29/15 15:45), Minchan Kim wrote:
[..]
> > +/*
> > + * Make sure that we actually can compact this class,
> > + * IOW if migration will release at least one szpage.
> 
>                                                  zspage,

ok.

> > + *
> > + * Should be called under class->lock
> 
> Please comment about return.

ok.

> > + */
> > +static unsigned long zs_can_compact(struct size_class *class)
> > +{
> > +	/*
> > +	 * Calculate how many unused allocated objects we
> > +	 * have and see if we can free any zspages. Otherwise,
> > +	 * compaction can just move objects back and forth w/o
> > +	 * any memory gain.
> > +	 */
> > +	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> > +		zs_stat_get(class, OBJ_USED);
> > +
> 
> I want to check one more thing.
> 
> We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
> In this implementation, compaction cannot have a source so
> it would better to bail out.
> IOW,
> 
>       if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
>               return 0;

ok.

	-ss

Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function

[Sergey Senozhatsky]

On (06/29/15 15:45), Minchan Kim wrote:
[..]
> > +/*
> > + * Make sure that we actually can compact this class,
> > + * IOW if migration will release at least one szpage.
> 
>                                                  zspage,

ok.

> > + *
> > + * Should be called under class->lock
> 
> Please comment about return.

ok.

> > + */
> > +static unsigned long zs_can_compact(struct size_class *class)
> > +{
> > +	/*
> > +	 * Calculate how many unused allocated objects we
> > +	 * have and see if we can free any zspages. Otherwise,
> > +	 * compaction can just move objects back and forth w/o
> > +	 * any memory gain.
> > +	 */
> > +	unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> > +		zs_stat_get(class, OBJ_USED);
> > +
> 
> I want to check one more thing.
> 
> We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
> In this implementation, compaction cannot have a source so
> it would better to bail out.
> IOW,
> 
>       if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
>               return 0;

ok.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Sergey Senozhatsky]

On (06/29/15 15:40), Minchan Kim wrote:
> On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> > Always account per-class `zs_size_stat' stats. This data will
> > help us make better decisions during compaction. We are especially
> > interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> > class compaction will result in any memory gain.
> > 
> > For instance, we know the number of allocated objects in the class,
> > the number of objects being used (so we also know how many objects
> > are not used) and the number of objects per-page. So we can ensure
> > if we have enough unused objects to form at least one ZS_EMPTY
> > zspage during compaction.
> > 
> > We calculate this value on per-class basis so we can calculate a
> > total number of zspages that can be released. Which is exactly what
> > a shrinker wants to know.
> > 
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Acked-by: Minchan Kim <minchan@kernel.org>
> 

thanks.

	-ss

Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats

[Sergey Senozhatsky]

On (06/29/15 15:40), Minchan Kim wrote:
> On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> > Always account per-class `zs_size_stat' stats. This data will
> > help us make better decisions during compaction. We are especially
> > interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> > class compaction will result in any memory gain.
> > 
> > For instance, we know the number of allocated objects in the class,
> > the number of objects being used (so we also know how many objects
> > are not used) and the number of objects per-page. So we can ensure
> > if we have enough unused objects to form at least one ZS_EMPTY
> > zspage during compaction.
> > 
> > We calculate this value on per-class basis so we can calculate a
> > total number of zspages that can be released. Which is exactly what
> > a shrinker wants to know.
> > 
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Acked-by: Minchan Kim <minchan@kernel.org>
> 

thanks.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Minchan Kim]

Hi Sergey,

Sorry for too late reply.

On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> Hello,
> 
> thanks for review.
> 
> On (06/29/15 16:07), Minchan Kim wrote:
> [..]
> > >  			if (!migrate_zspage(pool, class, &cc))
> > > -				break;
> > > +				goto out;
> > 
> > It should retry with another target_page instead of going out.
> 
> yep.
> 
> [..]
> > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > +		struct shrink_control *sc)
> > > +{
> [..]
> > 
> > Returns migrated object count.
> > 
> [..]
> > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > +		struct shrink_control *sc)
> > > +{
> [..]
> > 
> > But it returns wasted_obj / max_obj_per_zspage?
> > 
> 
> Good catch.
> So,  __zs_compact() and zs_shrinker_count() are ok. Returning
> "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> sense there. The only place is zs_shrinker_scan()->zs_compact().

I want to make zs_can_compact return freeable page unit.

ie,

        return obj_wasted * class->pages_per_zspage;
  
and let's make __zs_compact returns the number of freed pages.

IOW, I like your (c).

> 
> Hm, I can think of:
> 
> (a) We can change zs_compact() to return the total number of
> freed zspages. That will not really change a user visible
> interface. We export (fwiw) the number of compacted objects
> in mm_stat. Basically, this is internal zsmalloc() counter and
> no user space program can ever do anything with that data. From
> that prospective we will just replace one senseless number with
> another (equally meaningless) one.
> 
> 
> (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> 
> 1764         int i;
> 1765         unsigned long nr_migrated = 0;
> 1766         struct size_class *class;
> 1767
> 1768         for (i = zs_size_classes - 1; i >= 0; i--) {
> 1769                 class = pool->size_class[i];
> 1770                 if (!class)
> 1771                         continue;
> 1772                 if (class->index != i)
> 1773                         continue;
> 1774                 nr_migrated += __zs_compact(pool, class);
> 1775         }
> 1776
> 1777         return nr_migrated;
> 
> But on every iteration increment nr_migrated with
> 		"nr_migrated += just_migrated / max_obj_per_zspage"
> 
> (which will be unnecessary if zs_compact() will return the number of freed
> zspages).
> 
> So, (b) is mostly fine, except that we already have several pool->size_class
> loops, with same `if (!class)' and `if (class->index...)' checks; and it
> asks for some sort of refactoring or... a tricky for_each_class() define.
> 
> 
> In both cases, however, we don't tell anything valuable to user space.
> Thus,
> 
> (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> And change compaction to operate in terms of pages (PAGE_SIZE). At
> least mm_stat::compacted will turn into something useful for user
> space.

Yes.

Thanks.

> 
> 	-ss

-- 
Kind regards,
Minchan Kim

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Minchan Kim]

Hi Sergey,

Sorry for too late reply.

On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> Hello,
> 
> thanks for review.
> 
> On (06/29/15 16:07), Minchan Kim wrote:
> [..]
> > >  			if (!migrate_zspage(pool, class, &cc))
> > > -				break;
> > > +				goto out;
> > 
> > It should retry with another target_page instead of going out.
> 
> yep.
> 
> [..]
> > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > +		struct shrink_control *sc)
> > > +{
> [..]
> > 
> > Returns migrated object count.
> > 
> [..]
> > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > +		struct shrink_control *sc)
> > > +{
> [..]
> > 
> > But it returns wasted_obj / max_obj_per_zspage?
> > 
> 
> Good catch.
> So,  __zs_compact() and zs_shrinker_count() are ok. Returning
> "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> sense there. The only place is zs_shrinker_scan()->zs_compact().

I want to make zs_can_compact return freeable page unit.

ie,

        return obj_wasted * class->pages_per_zspage;
  
and let's make __zs_compact returns the number of freed pages.

IOW, I like your (c).

> 
> Hm, I can think of:
> 
> (a) We can change zs_compact() to return the total number of
> freed zspages. That will not really change a user visible
> interface. We export (fwiw) the number of compacted objects
> in mm_stat. Basically, this is internal zsmalloc() counter and
> no user space program can ever do anything with that data. From
> that prospective we will just replace one senseless number with
> another (equally meaningless) one.
> 
> 
> (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> 
> 1764         int i;
> 1765         unsigned long nr_migrated = 0;
> 1766         struct size_class *class;
> 1767
> 1768         for (i = zs_size_classes - 1; i >= 0; i--) {
> 1769                 class = pool->size_class[i];
> 1770                 if (!class)
> 1771                         continue;
> 1772                 if (class->index != i)
> 1773                         continue;
> 1774                 nr_migrated += __zs_compact(pool, class);
> 1775         }
> 1776
> 1777         return nr_migrated;
> 
> But on every iteration increment nr_migrated with
> 		"nr_migrated += just_migrated / max_obj_per_zspage"
> 
> (which will be unnecessary if zs_compact() will return the number of freed
> zspages).
> 
> So, (b) is mostly fine, except that we already have several pool->size_class
> loops, with same `if (!class)' and `if (class->index...)' checks; and it
> asks for some sort of refactoring or... a tricky for_each_class() define.
> 
> 
> In both cases, however, we don't tell anything valuable to user space.
> Thus,
> 
> (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> And change compaction to operate in terms of pages (PAGE_SIZE). At
> least mm_stat::compacted will turn into something useful for user
> space.

Yes.

Thanks.

> 
> 	-ss

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

On (06/29/15 22:39), Minchan Kim wrote:
> Hi Sergey,
> 
> Sorry for too late reply.
> 

Hello Minchan,

Sure, no problem.


Will send out a new patchset later today. Thanks.

	-ss

> On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> > Hello,
> > 
> > thanks for review.
> > 
> > On (06/29/15 16:07), Minchan Kim wrote:
> > [..]
> > > >  			if (!migrate_zspage(pool, class, &cc))
> > > > -				break;
> > > > +				goto out;
> > > 
> > > It should retry with another target_page instead of going out.
> > 
> > yep.
> > 
> > [..]
> > > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > > +		struct shrink_control *sc)
> > > > +{
> > [..]
> > > 
> > > Returns migrated object count.
> > > 
> > [..]
> > > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > > +		struct shrink_control *sc)
> > > > +{
> > [..]
> > > 
> > > But it returns wasted_obj / max_obj_per_zspage?
> > > 
> > 
> > Good catch.
> > So,  __zs_compact() and zs_shrinker_count() are ok. Returning
> > "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> > sense there. The only place is zs_shrinker_scan()->zs_compact().
> 
> I want to make zs_can_compact return freeable page unit.
> 
> ie,
> 
>         return obj_wasted * class->pages_per_zspage;
>   
> and let's make __zs_compact returns the number of freed pages.
> 
> IOW, I like your (c).
> 
> > 
> > Hm, I can think of:
> > 
> > (a) We can change zs_compact() to return the total number of
> > freed zspages. That will not really change a user visible
> > interface. We export (fwiw) the number of compacted objects
> > in mm_stat. Basically, this is internal zsmalloc() counter and
> > no user space program can ever do anything with that data. From
> > that prospective we will just replace one senseless number with
> > another (equally meaningless) one.
> > 
> > 
> > (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> > 
> > 1764         int i;
> > 1765         unsigned long nr_migrated = 0;
> > 1766         struct size_class *class;
> > 1767
> > 1768         for (i = zs_size_classes - 1; i >= 0; i--) {
> > 1769                 class = pool->size_class[i];
> > 1770                 if (!class)
> > 1771                         continue;
> > 1772                 if (class->index != i)
> > 1773                         continue;
> > 1774                 nr_migrated += __zs_compact(pool, class);
> > 1775         }
> > 1776
> > 1777         return nr_migrated;
> > 
> > But on every iteration increment nr_migrated with
> > 		"nr_migrated += just_migrated / max_obj_per_zspage"
> > 
> > (which will be unnecessary if zs_compact() will return the number of freed
> > zspages).
> > 
> > So, (b) is mostly fine, except that we already have several pool->size_class
> > loops, with same `if (!class)' and `if (class->index...)' checks; and it
> > asks for some sort of refactoring or... a tricky for_each_class() define.
> > 
> > 
> > In both cases, however, we don't tell anything valuable to user space.
> > Thus,
> > 
> > (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> > And change compaction to operate in terms of pages (PAGE_SIZE). At
> > least mm_stat::compacted will turn into something useful for user
> > space.
> 
> Yes.
> 
> Thanks.
> 
> > 
> > 	-ss
> 
> -- 
> Kind regards,
> Minchan Kim
> 

Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction

[Sergey Senozhatsky]

On (06/29/15 22:39), Minchan Kim wrote:
> Hi Sergey,
> 
> Sorry for too late reply.
> 

Hello Minchan,

Sure, no problem.


Will send out a new patchset later today. Thanks.

	-ss

> On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> > Hello,
> > 
> > thanks for review.
> > 
> > On (06/29/15 16:07), Minchan Kim wrote:
> > [..]
> > > >  			if (!migrate_zspage(pool, class, &cc))
> > > > -				break;
> > > > +				goto out;
> > > 
> > > It should retry with another target_page instead of going out.
> > 
> > yep.
> > 
> > [..]
> > > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > > +		struct shrink_control *sc)
> > > > +{
> > [..]
> > > 
> > > Returns migrated object count.
> > > 
> > [..]
> > > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > > +		struct shrink_control *sc)
> > > > +{
> > [..]
> > > 
> > > But it returns wasted_obj / max_obj_per_zspage?
> > > 
> > 
> > Good catch.
> > So,  __zs_compact() and zs_shrinker_count() are ok. Returning
> > "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> > sense there. The only place is zs_shrinker_scan()->zs_compact().
> 
> I want to make zs_can_compact return freeable page unit.
> 
> ie,
> 
>         return obj_wasted * class->pages_per_zspage;
>   
> and let's make __zs_compact returns the number of freed pages.
> 
> IOW, I like your (c).
> 
> > 
> > Hm, I can think of:
> > 
> > (a) We can change zs_compact() to return the total number of
> > freed zspages. That will not really change a user visible
> > interface. We export (fwiw) the number of compacted objects
> > in mm_stat. Basically, this is internal zsmalloc() counter and
> > no user space program can ever do anything with that data. From
> > that prospective we will just replace one senseless number with
> > another (equally meaningless) one.
> > 
> > 
> > (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> > 
> > 1764         int i;
> > 1765         unsigned long nr_migrated = 0;
> > 1766         struct size_class *class;
> > 1767
> > 1768         for (i = zs_size_classes - 1; i >= 0; i--) {
> > 1769                 class = pool->size_class[i];
> > 1770                 if (!class)
> > 1771                         continue;
> > 1772                 if (class->index != i)
> > 1773                         continue;
> > 1774                 nr_migrated += __zs_compact(pool, class);
> > 1775         }
> > 1776
> > 1777         return nr_migrated;
> > 
> > But on every iteration increment nr_migrated with
> > 		"nr_migrated += just_migrated / max_obj_per_zspage"
> > 
> > (which will be unnecessary if zs_compact() will return the number of freed
> > zspages).
> > 
> > So, (b) is mostly fine, except that we already have several pool->size_class
> > loops, with same `if (!class)' and `if (class->index...)' checks; and it
> > asks for some sort of refactoring or... a tricky for_each_class() define.
> > 
> > 
> > In both cases, however, we don't tell anything valuable to user space.
> > Thus,
> > 
> > (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> > And change compaction to operate in terms of pages (PAGE_SIZE). At
> > least mm_stat::compacted will turn into something useful for user
> > space.
> 
> Yes.
> 
> Thanks.
> 
> > 
> > 	-ss
> 
> -- 
> Kind regards,
> Minchan Kim
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

Hello,

Sorry for long reply.

On (06/29/15 15:52), Minchan Kim wrote:
[..]
> >  	head = &class->fullness_list[fullness];
> > -	if (*head)
> > -		list_add_tail(&page->lru, &(*head)->lru);
> > +	if (*head) {
> > +		/*
> > +		 * We want to see more ZS_FULL pages and less almost
> > +		 * empty/full. Put pages with higher ->inuse first.
> > +		 */
> > +		if (page->inuse < (*head)->inuse)
> > +			list_add_tail(&page->lru, &(*head)->lru);
> > +		else
> > +			list_add(&page->lru, &(*head)->lru);
> > +	}
> 
> >  
> >  	*head = page;
> 
> Why do you want to always put @page in the head?
> How about this?


Yeah, right. Looks OK to me. How do we want to handle it? Do you want
to submit it with Suggested-by: ss (I'm fine) or you want me to submit
it later today with Suggested-by: Minchan Kim?

	-ss

> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index e8cb31c..1c5fde9 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
>         if (fullness >= _ZS_NR_FULLNESS_GROUPS)
>                 return;
> 
> +       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> +                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +
>         head = &class->fullness_list[fullness];
> -       if (*head) {
> -               /*
> -                * We want to see more ZS_FULL pages and less almost
> -                * empty/full. Put pages with higher ->inuse first.
> -                */
> -               if (page->inuse < (*head)->inuse)
> -                       list_add_tail(&page->lru, &(*head)->lru);
> -               else
> -                       list_add(&page->lru, &(*head)->lru);
> +       if (!*head) {
> +               *head = page;
> +               return;
>         }
> 
> -       *head = page;
> -       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> -                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +       /*
> +        * We want to see more ZS_FULL pages and less almost
> +        * empty/full. Put pages with higher ->inuse first.
> +        */
> +       list_add_tail(&page->lru, &(*head)->lru);
> +       if (page->inuse >= (*head)->inuse)
> +               *head = page;
>  }

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Sergey Senozhatsky]

Hello,

Sorry for long reply.

On (06/29/15 15:52), Minchan Kim wrote:
[..]
> >  	head = &class->fullness_list[fullness];
> > -	if (*head)
> > -		list_add_tail(&page->lru, &(*head)->lru);
> > +	if (*head) {
> > +		/*
> > +		 * We want to see more ZS_FULL pages and less almost
> > +		 * empty/full. Put pages with higher ->inuse first.
> > +		 */
> > +		if (page->inuse < (*head)->inuse)
> > +			list_add_tail(&page->lru, &(*head)->lru);
> > +		else
> > +			list_add(&page->lru, &(*head)->lru);
> > +	}
> 
> >  
> >  	*head = page;
> 
> Why do you want to always put @page in the head?
> How about this?


Yeah, right. Looks OK to me. How do we want to handle it? Do you want
to submit it with Suggested-by: ss (I'm fine) or you want me to submit
it later today with Suggested-by: Minchan Kim?

	-ss

> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index e8cb31c..1c5fde9 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
>         if (fullness >= _ZS_NR_FULLNESS_GROUPS)
>                 return;
> 
> +       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> +                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +
>         head = &class->fullness_list[fullness];
> -       if (*head) {
> -               /*
> -                * We want to see more ZS_FULL pages and less almost
> -                * empty/full. Put pages with higher ->inuse first.
> -                */
> -               if (page->inuse < (*head)->inuse)
> -                       list_add_tail(&page->lru, &(*head)->lru);
> -               else
> -                       list_add(&page->lru, &(*head)->lru);
> +       if (!*head) {
> +               *head = page;
> +               return;
>         }
> 
> -       *head = page;
> -       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> -                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +       /*
> +        * We want to see more ZS_FULL pages and less almost
> +        * empty/full. Put pages with higher ->inuse first.
> +        */
> +       list_add_tail(&page->lru, &(*head)->lru);
> +       if (page->inuse >= (*head)->inuse)
> +               *head = page;
>  }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Minchan Kim]

On Tue, Jun 30, 2015 at 08:41:24AM +0900, Sergey Senozhatsky wrote:
> Hello,
> 
> Sorry for long reply.
> 
> On (06/29/15 15:52), Minchan Kim wrote:
> [..]
> > >  	head = &class->fullness_list[fullness];
> > > -	if (*head)
> > > -		list_add_tail(&page->lru, &(*head)->lru);
> > > +	if (*head) {
> > > +		/*
> > > +		 * We want to see more ZS_FULL pages and less almost
> > > +		 * empty/full. Put pages with higher ->inuse first.
> > > +		 */
> > > +		if (page->inuse < (*head)->inuse)
> > > +			list_add_tail(&page->lru, &(*head)->lru);
> > > +		else
> > > +			list_add(&page->lru, &(*head)->lru);
> > > +	}
> > 
> > >  
> > >  	*head = page;
> > 
> > Why do you want to always put @page in the head?
> > How about this?
> 
> 
> Yeah, right. Looks OK to me. How do we want to handle it? Do you want
> to submit it with Suggested-by: ss (I'm fine) or you want me to submit
> it later today with Suggested-by: Minchan Kim?

Hi Sergey,

Thanks for considering the credit.
However, it was just review so no need my sign. :)

I really appreciate your good works.
Thanks.


> 
> 	-ss
> 
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index e8cb31c..1c5fde9 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> >         if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> >                 return;
> > 
> > +       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > +                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +
> >         head = &class->fullness_list[fullness];
> > -       if (*head) {
> > -               /*
> > -                * We want to see more ZS_FULL pages and less almost
> > -                * empty/full. Put pages with higher ->inuse first.
> > -                */
> > -               if (page->inuse < (*head)->inuse)
> > -                       list_add_tail(&page->lru, &(*head)->lru);
> > -               else
> > -                       list_add(&page->lru, &(*head)->lru);
> > +       if (!*head) {
> > +               *head = page;
> > +               return;
> >         }
> > 
> > -       *head = page;
> > -       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > -                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +       /*
> > +        * We want to see more ZS_FULL pages and less almost
> > +        * empty/full. Put pages with higher ->inuse first.
> > +        */
> > +       list_add_tail(&page->lru, &(*head)->lru);
> > +       if (page->inuse >= (*head)->inuse)
> > +               *head = page;
> >  }

-- 
Kind regards,
Minchan Kim

Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list

[Minchan Kim]

On Tue, Jun 30, 2015 at 08:41:24AM +0900, Sergey Senozhatsky wrote:
> Hello,
> 
> Sorry for long reply.
> 
> On (06/29/15 15:52), Minchan Kim wrote:
> [..]
> > >  	head = &class->fullness_list[fullness];
> > > -	if (*head)
> > > -		list_add_tail(&page->lru, &(*head)->lru);
> > > +	if (*head) {
> > > +		/*
> > > +		 * We want to see more ZS_FULL pages and less almost
> > > +		 * empty/full. Put pages with higher ->inuse first.
> > > +		 */
> > > +		if (page->inuse < (*head)->inuse)
> > > +			list_add_tail(&page->lru, &(*head)->lru);
> > > +		else
> > > +			list_add(&page->lru, &(*head)->lru);
> > > +	}
> > 
> > >  
> > >  	*head = page;
> > 
> > Why do you want to always put @page in the head?
> > How about this?
> 
> 
> Yeah, right. Looks OK to me. How do we want to handle it? Do you want
> to submit it with Suggested-by: ss (I'm fine) or you want me to submit
> it later today with Suggested-by: Minchan Kim?

Hi Sergey,

Thanks for considering the credit.
However, it was just review so no need my sign. :)

I really appreciate your good works.
Thanks.


> 
> 	-ss
> 
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index e8cb31c..1c5fde9 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> >         if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> >                 return;
> > 
> > +       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > +                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +
> >         head = &class->fullness_list[fullness];
> > -       if (*head) {
> > -               /*
> > -                * We want to see more ZS_FULL pages and less almost
> > -                * empty/full. Put pages with higher ->inuse first.
> > -                */
> > -               if (page->inuse < (*head)->inuse)
> > -                       list_add_tail(&page->lru, &(*head)->lru);
> > -               else
> > -                       list_add(&page->lru, &(*head)->lru);
> > +       if (!*head) {
> > +               *head = page;
> > +               return;
> >         }
> > 
> > -       *head = page;
> > -       zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > -                       CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +       /*
> > +        * We want to see more ZS_FULL pages and less almost
> > +        * empty/full. Put pages with higher ->inuse first.
> > +        */
> > +       list_add_tail(&page->lru, &(*head)->lru);
> > +       if (page->inuse >= (*head)->inuse)
> > +               *head = page;
> >  }

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>