* [RFC][PATCH v3 0/7] introduce automatic pool compaction
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Hello,
This patch set tweaks compaction and makes it possible to trigger
pool compaction automatically when system is getting low on memory.
zsmalloc in some cases can suffer from a notable fragmentation and
compaction can release some considerable amount of memory. The problem
here is that currently we fully rely on user space to perform compaction
when needed. However, performing zsmalloc compaction is not always an
obvious thing to do. For example, suppose we have a `idle' fragmented
(compaction was never performed) zram device and system is getting low
on memory due to some 3rd party user processes (gcc LTO, or firefox, etc.).
It's quite unlikely that user space will issue zpool compaction in this
case. Besides, user space cannot tell for sure how badly pool is
fragmented; however, this info is known to zsmalloc and, hence, to a
shrinker.
v3:
-- drop almost_empty waterline adjustment patch (Minchan)
-- do not hold class->lock for the entire compaction period (Minchan)
v2:
-- use a slab shrinker instead of triggering compaction from zs_free (Minchan)
Sergey Senozhatsky (7):
zsmalloc: drop unused variable `nr_to_migrate'
zsmalloc: partial page ordering within a fullness_list
zsmalloc: always keep per-class stats
zsmalloc: introduce zs_can_compact() function
zsmalloc: cosmetic compaction code adjustments
zsmalloc/zram: move `num_migrated' to zs_pool
zsmalloc: register a shrinker to trigger auto-compaction
drivers/block/zram/zram_drv.c | 12 +--
drivers/block/zram/zram_drv.h | 1 -
include/linux/zsmalloc.h | 1 +
mm/zsmalloc.c | 220 +++++++++++++++++++++++++++---------------
4 files changed, 150 insertions(+), 84 deletions(-)
--
2.4.4
^ permalink raw reply [flat|nested] 48+ messages in thread
* [RFC][PATCH v3 0/7] introduce automatic pool compaction
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Hello,
This patch set tweaks compaction and makes it possible to trigger
pool compaction automatically when system is getting low on memory.
zsmalloc in some cases can suffer from a notable fragmentation and
compaction can release some considerable amount of memory. The problem
here is that currently we fully rely on user space to perform compaction
when needed. However, performing zsmalloc compaction is not always an
obvious thing to do. For example, suppose we have a `idle' fragmented
(compaction was never performed) zram device and system is getting low
on memory due to some 3rd party user processes (gcc LTO, or firefox, etc.).
It's quite unlikely that user space will issue zpool compaction in this
case. Besides, user space cannot tell for sure how badly pool is
fragmented; however, this info is known to zsmalloc and, hence, to a
shrinker.
v3:
-- drop almost_empty waterline adjustment patch (Minchan)
-- do not hold class->lock for the entire compaction period (Minchan)
v2:
-- use a slab shrinker instead of triggering compaction from zs_free (Minchan)
Sergey Senozhatsky (7):
zsmalloc: drop unused variable `nr_to_migrate'
zsmalloc: partial page ordering within a fullness_list
zsmalloc: always keep per-class stats
zsmalloc: introduce zs_can_compact() function
zsmalloc: cosmetic compaction code adjustments
zsmalloc/zram: move `num_migrated' to zs_pool
zsmalloc: register a shrinker to trigger auto-compaction
drivers/block/zram/zram_drv.c | 12 +--
drivers/block/zram/zram_drv.h | 1 -
include/linux/zsmalloc.h | 1 +
mm/zsmalloc.c | 220 +++++++++++++++++++++++++++---------------
4 files changed, 150 insertions(+), 84 deletions(-)
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 1/7] zsmalloc: drop unused variable `nr_to_migrate'
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
__zs_compact() does not use `nr_to_migrate', drop it.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 0a7f81a..7d816c2 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1703,7 +1703,6 @@ static struct page *isolate_source_page(struct size_class *class)
static unsigned long __zs_compact(struct zs_pool *pool,
struct size_class *class)
{
- int nr_to_migrate;
struct zs_compact_control cc;
struct page *src_page;
struct page *dst_page = NULL;
@@ -1714,8 +1713,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
BUG_ON(!is_first_page(src_page));
- /* The goal is to migrate all live objects in source page */
- nr_to_migrate = src_page->inuse;
cc.index = 0;
cc.s_page = src_page;
@@ -1730,7 +1727,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, dst_page);
nr_total_migrated += cc.nr_migrated;
- nr_to_migrate -= cc.nr_migrated;
}
/* Stop if we couldn't find slot */
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 1/7] zsmalloc: drop unused variable `nr_to_migrate'
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
__zs_compact() does not use `nr_to_migrate', drop it.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 0a7f81a..7d816c2 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1703,7 +1703,6 @@ static struct page *isolate_source_page(struct size_class *class)
static unsigned long __zs_compact(struct zs_pool *pool,
struct size_class *class)
{
- int nr_to_migrate;
struct zs_compact_control cc;
struct page *src_page;
struct page *dst_page = NULL;
@@ -1714,8 +1713,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
BUG_ON(!is_first_page(src_page));
- /* The goal is to migrate all live objects in source page */
- nr_to_migrate = src_page->inuse;
cc.index = 0;
cc.s_page = src_page;
@@ -1730,7 +1727,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, dst_page);
nr_total_migrated += cc.nr_migrated;
- nr_to_migrate -= cc.nr_migrated;
}
/* Stop if we couldn't find slot */
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
pages. Put a page with higher ->inuse count first within its
->fullness_list, which will give us better chances to fill up this
page with new objects (find_get_zspage() return ->fullness_list head
for new object allocation), so some zspages will become
ZS_ALMOST_FULL/ZS_FULL quicker.
It performs a trivial and cheap ->inuse compare which does not slow
down zsmalloc, and in the worst case it keeps the list pages not in
any particular order, just like we do it now.
A more expensive solution could sort fullness_list by ->inuse count.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7d816c2..6e2ebb6 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
return;
head = &class->fullness_list[fullness];
- if (*head)
- list_add_tail(&page->lru, &(*head)->lru);
+ if (*head) {
+ /*
+ * We want to see more ZS_FULL pages and less almost
+ * empty/full. Put pages with higher ->inuse first.
+ */
+ if (page->inuse < (*head)->inuse)
+ list_add_tail(&page->lru, &(*head)->lru);
+ else
+ list_add(&page->lru, &(*head)->lru);
+ }
*head = page;
zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
pages. Put a page with higher ->inuse count first within its
->fullness_list, which will give us better chances to fill up this
page with new objects (find_get_zspage() return ->fullness_list head
for new object allocation), so some zspages will become
ZS_ALMOST_FULL/ZS_FULL quicker.
It performs a trivial and cheap ->inuse compare which does not slow
down zsmalloc, and in the worst case it keeps the list pages not in
any particular order, just like we do it now.
A more expensive solution could sort fullness_list by ->inuse count.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7d816c2..6e2ebb6 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
return;
head = &class->fullness_list[fullness];
- if (*head)
- list_add_tail(&page->lru, &(*head)->lru);
+ if (*head) {
+ /*
+ * We want to see more ZS_FULL pages and less almost
+ * empty/full. Put pages with higher ->inuse first.
+ */
+ if (page->inuse < (*head)->inuse)
+ list_add_tail(&page->lru, &(*head)->lru);
+ else
+ list_add(&page->lru, &(*head)->lru);
+ }
*head = page;
zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Always account per-class `zs_size_stat' stats. This data will
help us make better decisions during compaction. We are especially
interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
class compaction will result in any memory gain.
For instance, we know the number of allocated objects in the class,
the number of objects being used (so we also know how many objects
are not used) and the number of objects per-page. So we can ensure
if we have enough unused objects to form at least one ZS_EMPTY
zspage during compaction.
We calculate this value on per-class basis so we can calculate a
total number of zspages that can be released. Which is exactly what
a shrinker wants to know.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 48 ++++++++++++------------------------------------
1 file changed, 12 insertions(+), 36 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 6e2ebb6..4b6f12e 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -169,14 +169,12 @@ enum zs_stat_type {
NR_ZS_STAT_TYPE,
};
-#ifdef CONFIG_ZSMALLOC_STAT
-
-static struct dentry *zs_stat_root;
-
struct zs_size_stat {
unsigned long objs[NR_ZS_STAT_TYPE];
};
+#ifdef CONFIG_ZSMALLOC_STAT
+static struct dentry *zs_stat_root;
#endif
/*
@@ -201,25 +199,21 @@ static int zs_size_classes;
static const int fullness_threshold_frac = 4;
struct size_class {
+ spinlock_t lock;
+ struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
/*
* Size of objects stored in this class. Must be multiple
* of ZS_ALIGN.
*/
- int size;
- unsigned int index;
+ int size;
+ unsigned int index;
/* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */
- int pages_per_zspage;
- /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
- bool huge;
-
-#ifdef CONFIG_ZSMALLOC_STAT
- struct zs_size_stat stats;
-#endif
-
- spinlock_t lock;
+ int pages_per_zspage;
+ struct zs_size_stat stats;
- struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
+ /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
+ bool huge;
};
/*
@@ -441,8 +435,6 @@ static int get_size_class_index(int size)
return min(zs_size_classes - 1, idx);
}
-#ifdef CONFIG_ZSMALLOC_STAT
-
static inline void zs_stat_inc(struct size_class *class,
enum zs_stat_type type, unsigned long cnt)
{
@@ -461,6 +453,8 @@ static inline unsigned long zs_stat_get(struct size_class *class,
return class->stats.objs[type];
}
+#ifdef CONFIG_ZSMALLOC_STAT
+
static int __init zs_stat_init(void)
{
if (!debugfs_initialized())
@@ -576,23 +570,6 @@ static void zs_pool_stat_destroy(struct zs_pool *pool)
}
#else /* CONFIG_ZSMALLOC_STAT */
-
-static inline void zs_stat_inc(struct size_class *class,
- enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline void zs_stat_dec(struct size_class *class,
- enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline unsigned long zs_stat_get(struct size_class *class,
- enum zs_stat_type type)
-{
- return 0;
-}
-
static int __init zs_stat_init(void)
{
return 0;
@@ -610,7 +587,6 @@ static inline int zs_pool_stat_create(char *name, struct zs_pool *pool)
static inline void zs_pool_stat_destroy(struct zs_pool *pool)
{
}
-
#endif
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Always account per-class `zs_size_stat' stats. This data will
help us make better decisions during compaction. We are especially
interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
class compaction will result in any memory gain.
For instance, we know the number of allocated objects in the class,
the number of objects being used (so we also know how many objects
are not used) and the number of objects per-page. So we can ensure
if we have enough unused objects to form at least one ZS_EMPTY
zspage during compaction.
We calculate this value on per-class basis so we can calculate a
total number of zspages that can be released. Which is exactly what
a shrinker wants to know.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 48 ++++++++++++------------------------------------
1 file changed, 12 insertions(+), 36 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 6e2ebb6..4b6f12e 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -169,14 +169,12 @@ enum zs_stat_type {
NR_ZS_STAT_TYPE,
};
-#ifdef CONFIG_ZSMALLOC_STAT
-
-static struct dentry *zs_stat_root;
-
struct zs_size_stat {
unsigned long objs[NR_ZS_STAT_TYPE];
};
+#ifdef CONFIG_ZSMALLOC_STAT
+static struct dentry *zs_stat_root;
#endif
/*
@@ -201,25 +199,21 @@ static int zs_size_classes;
static const int fullness_threshold_frac = 4;
struct size_class {
+ spinlock_t lock;
+ struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
/*
* Size of objects stored in this class. Must be multiple
* of ZS_ALIGN.
*/
- int size;
- unsigned int index;
+ int size;
+ unsigned int index;
/* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */
- int pages_per_zspage;
- /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
- bool huge;
-
-#ifdef CONFIG_ZSMALLOC_STAT
- struct zs_size_stat stats;
-#endif
-
- spinlock_t lock;
+ int pages_per_zspage;
+ struct zs_size_stat stats;
- struct page *fullness_list[_ZS_NR_FULLNESS_GROUPS];
+ /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */
+ bool huge;
};
/*
@@ -441,8 +435,6 @@ static int get_size_class_index(int size)
return min(zs_size_classes - 1, idx);
}
-#ifdef CONFIG_ZSMALLOC_STAT
-
static inline void zs_stat_inc(struct size_class *class,
enum zs_stat_type type, unsigned long cnt)
{
@@ -461,6 +453,8 @@ static inline unsigned long zs_stat_get(struct size_class *class,
return class->stats.objs[type];
}
+#ifdef CONFIG_ZSMALLOC_STAT
+
static int __init zs_stat_init(void)
{
if (!debugfs_initialized())
@@ -576,23 +570,6 @@ static void zs_pool_stat_destroy(struct zs_pool *pool)
}
#else /* CONFIG_ZSMALLOC_STAT */
-
-static inline void zs_stat_inc(struct size_class *class,
- enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline void zs_stat_dec(struct size_class *class,
- enum zs_stat_type type, unsigned long cnt)
-{
-}
-
-static inline unsigned long zs_stat_get(struct size_class *class,
- enum zs_stat_type type)
-{
- return 0;
-}
-
static int __init zs_stat_init(void)
{
return 0;
@@ -610,7 +587,6 @@ static inline int zs_pool_stat_create(char *name, struct zs_pool *pool)
static inline void zs_pool_stat_destroy(struct zs_pool *pool)
{
}
-
#endif
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
This function checks if class compaction will free any pages.
Rephrasing -- do we have enough unused objects to form at least
one ZS_EMPTY page and free it. It aborts compaction if class
compaction will not result in any (further) savings.
EXAMPLE (this debug output is not part of this patch set):
-- class size
-- number of allocated objects
-- number of used objects
-- max objects per zspage
-- pages per zspage
-- estimated number of pages that will be freed
[..]
class-512 objs:544 inuse:540 maxobj-per-zspage:8 pages-per-zspage:1 zspages-to-free:0
... class-512 compaction is useless. break
class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
... class-496 compaction is useless. break
class-448 objs:657 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:4
class-448 objs:648 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:3
class-448 objs:639 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:2
class-448 objs:630 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:1
class-448 objs:621 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:0
... class-448 compaction is useless. break
class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
... class-432 compaction is useless. break
class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
... class-416 compaction is useless. break
class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
... class-400 compaction is useless. break
class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
... class-384 compaction is useless. break
[..]
Every "compaction is useless" indicates that we saved CPU cycles.
class-512 has
544 object allocated
540 objects used
8 objects per-page
Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
migrate all of its objects and free this zspage; so compaction will not
make a lot of sense, it's better to just leave it as is.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 4b6f12e..18e10a4 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
return page;
}
+/*
+ * Make sure that we actually can compact this class,
+ * IOW if migration will release at least one szpage.
+ *
+ * Should be called under class->lock
+ */
+static unsigned long zs_can_compact(struct size_class *class)
+{
+ /*
+ * Calculate how many unused allocated objects we
+ * have and see if we can free any zspages. Otherwise,
+ * compaction can just move objects back and forth w/o
+ * any memory gain.
+ */
+ unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
+ zs_stat_get(class, OBJ_USED);
+
+ obj_wasted /= get_maxobj_per_zspage(class->size,
+ class->pages_per_zspage);
+ return obj_wasted;
+}
+
static unsigned long __zs_compact(struct zs_pool *pool,
struct size_class *class)
{
@@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
BUG_ON(!is_first_page(src_page));
+ if (!zs_can_compact(class))
+ break;
+
cc.index = 0;
cc.s_page = src_page;
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
This function checks if class compaction will free any pages.
Rephrasing -- do we have enough unused objects to form at least
one ZS_EMPTY page and free it. It aborts compaction if class
compaction will not result in any (further) savings.
EXAMPLE (this debug output is not part of this patch set):
-- class size
-- number of allocated objects
-- number of used objects
-- max objects per zspage
-- pages per zspage
-- estimated number of pages that will be freed
[..]
class-512 objs:544 inuse:540 maxobj-per-zspage:8 pages-per-zspage:1 zspages-to-free:0
... class-512 compaction is useless. break
class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
... class-496 compaction is useless. break
class-448 objs:657 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:4
class-448 objs:648 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:3
class-448 objs:639 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:2
class-448 objs:630 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:1
class-448 objs:621 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:0
... class-448 compaction is useless. break
class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
... class-432 compaction is useless. break
class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
... class-416 compaction is useless. break
class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
... class-400 compaction is useless. break
class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
... class-384 compaction is useless. break
[..]
Every "compaction is useless" indicates that we saved CPU cycles.
class-512 has
544 object allocated
540 objects used
8 objects per-page
Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
migrate all of its objects and free this zspage; so compaction will not
make a lot of sense, it's better to just leave it as is.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
mm/zsmalloc.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 4b6f12e..18e10a4 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
return page;
}
+/*
+ * Make sure that we actually can compact this class,
+ * IOW if migration will release at least one szpage.
+ *
+ * Should be called under class->lock
+ */
+static unsigned long zs_can_compact(struct size_class *class)
+{
+ /*
+ * Calculate how many unused allocated objects we
+ * have and see if we can free any zspages. Otherwise,
+ * compaction can just move objects back and forth w/o
+ * any memory gain.
+ */
+ unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
+ zs_stat_get(class, OBJ_USED);
+
+ obj_wasted /= get_maxobj_per_zspage(class->size,
+ class->pages_per_zspage);
+ return obj_wasted;
+}
+
static unsigned long __zs_compact(struct zs_pool *pool,
struct size_class *class)
{
@@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
BUG_ON(!is_first_page(src_page));
+ if (!zs_can_compact(class))
+ break;
+
cc.index = 0;
cc.s_page = src_page;
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 5/7] zsmalloc: cosmetic compaction code adjustments
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Change zs_object_copy() argument order to be (DST, SRC) rather
than (SRC, DST). copy/move functions usually have (to, from)
arguments order.
Rename alloc_target_page() to isolate_target_page(). This
function doesn't allocate anything, it isolates target page,
pretty much like isolate_source_page().
Tweak __zs_compact() comment.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 18e10a4..5fa96ae 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1479,7 +1479,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
}
EXPORT_SYMBOL_GPL(zs_free);
-static void zs_object_copy(unsigned long src, unsigned long dst,
+static void zs_object_copy(unsigned long dst, unsigned long src,
struct size_class *class)
{
struct page *s_page, *d_page;
@@ -1620,7 +1620,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
used_obj = handle_to_obj(handle);
free_obj = obj_malloc(d_page, class, handle);
- zs_object_copy(used_obj, free_obj, class);
+ zs_object_copy(free_obj, used_obj, class);
index++;
record_obj(handle, free_obj);
unpin_tag(handle);
@@ -1636,7 +1636,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
return ret;
}
-static struct page *alloc_target_page(struct size_class *class)
+static struct page *isolate_target_page(struct size_class *class)
{
int i;
struct page *page;
@@ -1725,11 +1725,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
cc.index = 0;
cc.s_page = src_page;
- while ((dst_page = alloc_target_page(class))) {
+ while ((dst_page = isolate_target_page(class))) {
cc.d_page = dst_page;
/*
- * If there is no more space in dst_page, try to
- * allocate another zspage.
+ * If there is no more space in dst_page, resched
+ * and see if anyone had allocated another zspage.
*/
if (!migrate_zspage(pool, class, &cc))
break;
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 5/7] zsmalloc: cosmetic compaction code adjustments
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Change zs_object_copy() argument order to be (DST, SRC) rather
than (SRC, DST). copy/move functions usually have (to, from)
arguments order.
Rename alloc_target_page() to isolate_target_page(). This
function doesn't allocate anything, it isolates target page,
pretty much like isolate_source_page().
Tweak __zs_compact() comment.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 18e10a4..5fa96ae 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1479,7 +1479,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
}
EXPORT_SYMBOL_GPL(zs_free);
-static void zs_object_copy(unsigned long src, unsigned long dst,
+static void zs_object_copy(unsigned long dst, unsigned long src,
struct size_class *class)
{
struct page *s_page, *d_page;
@@ -1620,7 +1620,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
used_obj = handle_to_obj(handle);
free_obj = obj_malloc(d_page, class, handle);
- zs_object_copy(used_obj, free_obj, class);
+ zs_object_copy(free_obj, used_obj, class);
index++;
record_obj(handle, free_obj);
unpin_tag(handle);
@@ -1636,7 +1636,7 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
return ret;
}
-static struct page *alloc_target_page(struct size_class *class)
+static struct page *isolate_target_page(struct size_class *class)
{
int i;
struct page *page;
@@ -1725,11 +1725,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
cc.index = 0;
cc.s_page = src_page;
- while ((dst_page = alloc_target_page(class))) {
+ while ((dst_page = isolate_target_page(class))) {
cc.d_page = dst_page;
/*
- * If there is no more space in dst_page, try to
- * allocate another zspage.
+ * If there is no more space in dst_page, resched
+ * and see if anyone had allocated another zspage.
*/
if (!migrate_zspage(pool, class, &cc))
break;
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 6/7] zsmalloc/zram: move `num_migrated' to zs_pool
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Remove the number of migrated objects from `zs_compact_control'
and move it to `zs_pool'. `zs_compact_control' has a limited
lifespan, we lose it when zs_compaction() returns back to zram.
To keep track of objects migrated during auto-compaction (issued
by the shrinker) we need to store this number in zs_pool.
Introduce zs_get_num_migrated() to export zs_pool's ->num_migrated
counter and use it in zram, so we can also drop a zram's copy of
`num_migrated'.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
drivers/block/zram/zram_drv.c | 12 ++++++------
drivers/block/zram/zram_drv.h | 1 -
include/linux/zsmalloc.h | 1 +
mm/zsmalloc.c | 41 +++++++++++++++++++----------------------
4 files changed, 26 insertions(+), 29 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index fb655e8..28ad3f8 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -388,7 +388,6 @@ static ssize_t comp_algorithm_store(struct device *dev,
static ssize_t compact_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- unsigned long nr_migrated;
struct zram *zram = dev_to_zram(dev);
struct zram_meta *meta;
@@ -399,8 +398,7 @@ static ssize_t compact_store(struct device *dev,
}
meta = zram->meta;
- nr_migrated = zs_compact(meta->mem_pool);
- atomic64_add(nr_migrated, &zram->stats.num_migrated);
+ zs_compact(meta->mem_pool);
up_read(&zram->init_lock);
return len;
@@ -428,13 +426,15 @@ static ssize_t mm_stat_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct zram *zram = dev_to_zram(dev);
- u64 orig_size, mem_used = 0;
+ u64 orig_size, mem_used = 0, num_migrated = 0;
long max_used;
ssize_t ret;
down_read(&zram->init_lock);
- if (init_done(zram))
+ if (init_done(zram)) {
mem_used = zs_get_total_pages(zram->meta->mem_pool);
+ num_migrated = zs_get_num_migrated(zram->meta->mem_pool);
+ }
orig_size = atomic64_read(&zram->stats.pages_stored);
max_used = atomic_long_read(&zram->stats.max_used_pages);
@@ -447,7 +447,7 @@ static ssize_t mm_stat_show(struct device *dev,
zram->limit_pages << PAGE_SHIFT,
max_used << PAGE_SHIFT,
(u64)atomic64_read(&zram->stats.zero_pages),
- (u64)atomic64_read(&zram->stats.num_migrated));
+ num_migrated);
up_read(&zram->init_lock);
return ret;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 6dbe2df..8e92339 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -78,7 +78,6 @@ struct zram_stats {
atomic64_t compr_data_size; /* compressed size of pages stored */
atomic64_t num_reads; /* failed + successful */
atomic64_t num_writes; /* --do-- */
- atomic64_t num_migrated; /* no. of migrated object */
atomic64_t failed_reads; /* can happen when memory is too low */
atomic64_t failed_writes; /* can happen when memory is too low */
atomic64_t invalid_io; /* non-page-aligned I/O requests */
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 1338190..e878875 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -47,6 +47,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle,
void zs_unmap_object(struct zs_pool *pool, unsigned long handle);
unsigned long zs_get_total_pages(struct zs_pool *pool);
+unsigned long zs_get_num_migrated(struct zs_pool *pool);
unsigned long zs_compact(struct zs_pool *pool);
#endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 5fa96ae..c9aea0a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -237,16 +237,19 @@ struct link_free {
};
struct zs_pool {
- char *name;
+ char *name;
- struct size_class **size_class;
- struct kmem_cache *handle_cachep;
+ struct size_class **size_class;
+ struct kmem_cache *handle_cachep;
- gfp_t flags; /* allocation flags used when growing pool */
- atomic_long_t pages_allocated;
+ /* Allocation flags used when growing pool */
+ gfp_t flags;
+ atomic_long_t pages_allocated;
+ /* How many objects were migrated */
+ unsigned long num_migrated;
#ifdef CONFIG_ZSMALLOC_STAT
- struct dentry *stat_dentry;
+ struct dentry *stat_dentry;
#endif
};
@@ -1220,6 +1223,12 @@ unsigned long zs_get_total_pages(struct zs_pool *pool)
}
EXPORT_SYMBOL_GPL(zs_get_total_pages);
+unsigned long zs_get_num_migrated(struct zs_pool *pool)
+{
+ return pool->num_migrated;
+}
+EXPORT_SYMBOL_GPL(zs_get_num_migrated);
+
/**
* zs_map_object - get address of allocated object from handle.
* @pool: pool from which the object was allocated
@@ -1586,8 +1595,6 @@ struct zs_compact_control {
/* Starting object index within @s_page which used for live object
* in the subpage. */
int index;
- /* how many of objects are migrated */
- int nr_migrated;
};
static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
@@ -1598,7 +1605,6 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
struct page *s_page = cc->s_page;
struct page *d_page = cc->d_page;
unsigned long index = cc->index;
- int nr_migrated = 0;
int ret = 0;
while (1) {
@@ -1625,13 +1631,12 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
record_obj(handle, free_obj);
unpin_tag(handle);
obj_free(pool, class, used_obj);
- nr_migrated++;
+ pool->num_migrated++;
}
/* Remember last position in this iteration */
cc->s_page = s_page;
cc->index = index;
- cc->nr_migrated = nr_migrated;
return ret;
}
@@ -1706,13 +1711,11 @@ static unsigned long zs_can_compact(struct size_class *class)
return obj_wasted;
}
-static unsigned long __zs_compact(struct zs_pool *pool,
- struct size_class *class)
+static void __zs_compact(struct zs_pool *pool, struct size_class *class)
{
struct zs_compact_control cc;
struct page *src_page;
struct page *dst_page = NULL;
- unsigned long nr_total_migrated = 0;
spin_lock(&class->lock);
while ((src_page = isolate_source_page(class))) {
@@ -1735,7 +1738,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
break;
putback_zspage(pool, class, dst_page);
- nr_total_migrated += cc.nr_migrated;
}
/* Stop if we couldn't find slot */
@@ -1745,7 +1747,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, dst_page);
putback_zspage(pool, class, src_page);
spin_unlock(&class->lock);
- nr_total_migrated += cc.nr_migrated;
cond_resched();
spin_lock(&class->lock);
}
@@ -1754,14 +1755,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, src_page);
spin_unlock(&class->lock);
-
- return nr_total_migrated;
}
unsigned long zs_compact(struct zs_pool *pool)
{
int i;
- unsigned long nr_migrated = 0;
struct size_class *class;
for (i = zs_size_classes - 1; i >= 0; i--) {
@@ -1770,10 +1768,9 @@ unsigned long zs_compact(struct zs_pool *pool)
continue;
if (class->index != i)
continue;
- nr_migrated += __zs_compact(pool, class);
+ __zs_compact(pool, class);
}
-
- return nr_migrated;
+ return pool->num_migrated;
}
EXPORT_SYMBOL_GPL(zs_compact);
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 6/7] zsmalloc/zram: move `num_migrated' to zs_pool
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Remove the number of migrated objects from `zs_compact_control'
and move it to `zs_pool'. `zs_compact_control' has a limited
lifespan, we lose it when zs_compaction() returns back to zram.
To keep track of objects migrated during auto-compaction (issued
by the shrinker) we need to store this number in zs_pool.
Introduce zs_get_num_migrated() to export zs_pool's ->num_migrated
counter and use it in zram, so we can also drop a zram's copy of
`num_migrated'.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
drivers/block/zram/zram_drv.c | 12 ++++++------
drivers/block/zram/zram_drv.h | 1 -
include/linux/zsmalloc.h | 1 +
mm/zsmalloc.c | 41 +++++++++++++++++++----------------------
4 files changed, 26 insertions(+), 29 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index fb655e8..28ad3f8 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -388,7 +388,6 @@ static ssize_t comp_algorithm_store(struct device *dev,
static ssize_t compact_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- unsigned long nr_migrated;
struct zram *zram = dev_to_zram(dev);
struct zram_meta *meta;
@@ -399,8 +398,7 @@ static ssize_t compact_store(struct device *dev,
}
meta = zram->meta;
- nr_migrated = zs_compact(meta->mem_pool);
- atomic64_add(nr_migrated, &zram->stats.num_migrated);
+ zs_compact(meta->mem_pool);
up_read(&zram->init_lock);
return len;
@@ -428,13 +426,15 @@ static ssize_t mm_stat_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct zram *zram = dev_to_zram(dev);
- u64 orig_size, mem_used = 0;
+ u64 orig_size, mem_used = 0, num_migrated = 0;
long max_used;
ssize_t ret;
down_read(&zram->init_lock);
- if (init_done(zram))
+ if (init_done(zram)) {
mem_used = zs_get_total_pages(zram->meta->mem_pool);
+ num_migrated = zs_get_num_migrated(zram->meta->mem_pool);
+ }
orig_size = atomic64_read(&zram->stats.pages_stored);
max_used = atomic_long_read(&zram->stats.max_used_pages);
@@ -447,7 +447,7 @@ static ssize_t mm_stat_show(struct device *dev,
zram->limit_pages << PAGE_SHIFT,
max_used << PAGE_SHIFT,
(u64)atomic64_read(&zram->stats.zero_pages),
- (u64)atomic64_read(&zram->stats.num_migrated));
+ num_migrated);
up_read(&zram->init_lock);
return ret;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 6dbe2df..8e92339 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -78,7 +78,6 @@ struct zram_stats {
atomic64_t compr_data_size; /* compressed size of pages stored */
atomic64_t num_reads; /* failed + successful */
atomic64_t num_writes; /* --do-- */
- atomic64_t num_migrated; /* no. of migrated object */
atomic64_t failed_reads; /* can happen when memory is too low */
atomic64_t failed_writes; /* can happen when memory is too low */
atomic64_t invalid_io; /* non-page-aligned I/O requests */
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 1338190..e878875 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -47,6 +47,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle,
void zs_unmap_object(struct zs_pool *pool, unsigned long handle);
unsigned long zs_get_total_pages(struct zs_pool *pool);
+unsigned long zs_get_num_migrated(struct zs_pool *pool);
unsigned long zs_compact(struct zs_pool *pool);
#endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 5fa96ae..c9aea0a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -237,16 +237,19 @@ struct link_free {
};
struct zs_pool {
- char *name;
+ char *name;
- struct size_class **size_class;
- struct kmem_cache *handle_cachep;
+ struct size_class **size_class;
+ struct kmem_cache *handle_cachep;
- gfp_t flags; /* allocation flags used when growing pool */
- atomic_long_t pages_allocated;
+ /* Allocation flags used when growing pool */
+ gfp_t flags;
+ atomic_long_t pages_allocated;
+ /* How many objects were migrated */
+ unsigned long num_migrated;
#ifdef CONFIG_ZSMALLOC_STAT
- struct dentry *stat_dentry;
+ struct dentry *stat_dentry;
#endif
};
@@ -1220,6 +1223,12 @@ unsigned long zs_get_total_pages(struct zs_pool *pool)
}
EXPORT_SYMBOL_GPL(zs_get_total_pages);
+unsigned long zs_get_num_migrated(struct zs_pool *pool)
+{
+ return pool->num_migrated;
+}
+EXPORT_SYMBOL_GPL(zs_get_num_migrated);
+
/**
* zs_map_object - get address of allocated object from handle.
* @pool: pool from which the object was allocated
@@ -1586,8 +1595,6 @@ struct zs_compact_control {
/* Starting object index within @s_page which used for live object
* in the subpage. */
int index;
- /* how many of objects are migrated */
- int nr_migrated;
};
static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
@@ -1598,7 +1605,6 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
struct page *s_page = cc->s_page;
struct page *d_page = cc->d_page;
unsigned long index = cc->index;
- int nr_migrated = 0;
int ret = 0;
while (1) {
@@ -1625,13 +1631,12 @@ static int migrate_zspage(struct zs_pool *pool, struct size_class *class,
record_obj(handle, free_obj);
unpin_tag(handle);
obj_free(pool, class, used_obj);
- nr_migrated++;
+ pool->num_migrated++;
}
/* Remember last position in this iteration */
cc->s_page = s_page;
cc->index = index;
- cc->nr_migrated = nr_migrated;
return ret;
}
@@ -1706,13 +1711,11 @@ static unsigned long zs_can_compact(struct size_class *class)
return obj_wasted;
}
-static unsigned long __zs_compact(struct zs_pool *pool,
- struct size_class *class)
+static void __zs_compact(struct zs_pool *pool, struct size_class *class)
{
struct zs_compact_control cc;
struct page *src_page;
struct page *dst_page = NULL;
- unsigned long nr_total_migrated = 0;
spin_lock(&class->lock);
while ((src_page = isolate_source_page(class))) {
@@ -1735,7 +1738,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
break;
putback_zspage(pool, class, dst_page);
- nr_total_migrated += cc.nr_migrated;
}
/* Stop if we couldn't find slot */
@@ -1745,7 +1747,6 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, dst_page);
putback_zspage(pool, class, src_page);
spin_unlock(&class->lock);
- nr_total_migrated += cc.nr_migrated;
cond_resched();
spin_lock(&class->lock);
}
@@ -1754,14 +1755,11 @@ static unsigned long __zs_compact(struct zs_pool *pool,
putback_zspage(pool, class, src_page);
spin_unlock(&class->lock);
-
- return nr_total_migrated;
}
unsigned long zs_compact(struct zs_pool *pool)
{
int i;
- unsigned long nr_migrated = 0;
struct size_class *class;
for (i = zs_size_classes - 1; i >= 0; i--) {
@@ -1770,10 +1768,9 @@ unsigned long zs_compact(struct zs_pool *pool)
continue;
if (class->index != i)
continue;
- nr_migrated += __zs_compact(pool, class);
+ __zs_compact(pool, class);
}
-
- return nr_migrated;
+ return pool->num_migrated;
}
EXPORT_SYMBOL_GPL(zs_compact);
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 11:46 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Perform automatic pool compaction by a shrinker when system
is getting tight on memory.
User-space has a very little knowledge regarding zsmalloc fragmentation
and basically has no mechanism to tell whether compaction will result
in any memory gain. Another issue is that user space is not always
aware of the fact that system is getting tight on memory. Which leads
to very uncomfortable scenarios when user space may start issuing
compaction 'randomly' or from crontab (for example). Fragmentation
is not always necessarily bad, allocated and unused objects, after all,
may be filled with the data later, w/o the need of allocating a new
zspage. On the other hand, we obviously don't want to waste memory
when systems needs it.
Compaction now has a relatively quick pool scan so we are able to
estimate the number of pages that will be freed easily, which makes it
possible to call this function from a shrinker->count_objects() callback.
We also abort compaction as soon as we detect that we can't free any
pages any more, preventing wasteful objects migrations.
Minchan Kim proposed to use the shrinker (the original patch was too
aggressive and was attempting to perform compaction for every
ALMOST_EMPTY zspage).
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Suggested-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 71 insertions(+), 7 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index c9aea0a..692b7dc 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -247,7 +247,9 @@ struct zs_pool {
atomic_long_t pages_allocated;
/* How many objects were migrated */
unsigned long num_migrated;
-
+ /* Compact classes */
+ struct shrinker shrinker;
+ bool shrinker_enabled;
#ifdef CONFIG_ZSMALLOC_STAT
struct dentry *stat_dentry;
#endif
@@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
while ((dst_page = isolate_target_page(class))) {
cc.d_page = dst_page;
- /*
- * If there is no more space in dst_page, resched
- * and see if anyone had allocated another zspage.
- */
+
if (!migrate_zspage(pool, class, &cc))
- break;
+ goto out;
putback_zspage(pool, class, dst_page);
}
@@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
cond_resched();
spin_lock(&class->lock);
}
-
+out:
+ if (dst_page)
+ putback_zspage(pool, class, dst_page);
if (src_page)
putback_zspage(pool, class, src_page);
@@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
}
EXPORT_SYMBOL_GPL(zs_compact);
+static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
+ struct shrink_control *sc)
+{
+ unsigned long freed;
+ struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+ shrinker);
+
+ freed = pool->num_migrated;
+ /* Compact classes and calculate compaction delta */
+ freed = zs_compact(pool) - freed;
+
+ return freed ? freed : SHRINK_STOP;
+}
+
+static unsigned long zs_shrinker_count(struct shrinker *shrinker,
+ struct shrink_control *sc)
+{
+ int i;
+ struct size_class *class;
+ unsigned long to_free = 0;
+ struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+ shrinker);
+
+ if (!pool->shrinker_enabled)
+ return 0;
+
+ for (i = zs_size_classes - 1; i >= 0; i--) {
+ class = pool->size_class[i];
+ if (!class)
+ continue;
+ if (class->index != i)
+ continue;
+
+ spin_lock(&class->lock);
+ to_free += zs_can_compact(class);
+ spin_unlock(&class->lock);
+ }
+
+ return to_free;
+}
+
+static void zs_unregister_shrinker(struct zs_pool *pool)
+{
+ if (pool->shrinker_enabled) {
+ unregister_shrinker(&pool->shrinker);
+ pool->shrinker_enabled = false;
+ }
+}
+
+static int zs_register_shrinker(struct zs_pool *pool)
+{
+ pool->shrinker.scan_objects = zs_shrinker_scan;
+ pool->shrinker.count_objects = zs_shrinker_count;
+ pool->shrinker.batch = 0;
+ pool->shrinker.seeks = DEFAULT_SEEKS;
+
+ return register_shrinker(&pool->shrinker);
+}
+
/**
* zs_create_pool - Creates an allocation pool to work from.
* @flags: allocation flags used to allocate pool metadata
@@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
if (zs_pool_stat_create(name, pool))
goto err;
+ /* Not critical, we still can use the pool */
+ if (zs_register_shrinker(pool) == 0)
+ pool->shrinker_enabled = true;
return pool;
err:
@@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
{
int i;
+ zs_unregister_shrinker(pool);
zs_pool_stat_destroy(pool);
for (i = 0; i < zs_size_classes; i++) {
--
2.4.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
@ 2015-06-18 11:46 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 11:46 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Perform automatic pool compaction by a shrinker when system
is getting tight on memory.
User-space has a very little knowledge regarding zsmalloc fragmentation
and basically has no mechanism to tell whether compaction will result
in any memory gain. Another issue is that user space is not always
aware of the fact that system is getting tight on memory. Which leads
to very uncomfortable scenarios when user space may start issuing
compaction 'randomly' or from crontab (for example). Fragmentation
is not always necessarily bad, allocated and unused objects, after all,
may be filled with the data later, w/o the need of allocating a new
zspage. On the other hand, we obviously don't want to waste memory
when systems needs it.
Compaction now has a relatively quick pool scan so we are able to
estimate the number of pages that will be freed easily, which makes it
possible to call this function from a shrinker->count_objects() callback.
We also abort compaction as soon as we detect that we can't free any
pages any more, preventing wasteful objects migrations.
Minchan Kim proposed to use the shrinker (the original patch was too
aggressive and was attempting to perform compaction for every
ALMOST_EMPTY zspage).
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Suggested-by: Minchan Kim <minchan@kernel.org>
---
mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 71 insertions(+), 7 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index c9aea0a..692b7dc 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -247,7 +247,9 @@ struct zs_pool {
atomic_long_t pages_allocated;
/* How many objects were migrated */
unsigned long num_migrated;
-
+ /* Compact classes */
+ struct shrinker shrinker;
+ bool shrinker_enabled;
#ifdef CONFIG_ZSMALLOC_STAT
struct dentry *stat_dentry;
#endif
@@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
while ((dst_page = isolate_target_page(class))) {
cc.d_page = dst_page;
- /*
- * If there is no more space in dst_page, resched
- * and see if anyone had allocated another zspage.
- */
+
if (!migrate_zspage(pool, class, &cc))
- break;
+ goto out;
putback_zspage(pool, class, dst_page);
}
@@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
cond_resched();
spin_lock(&class->lock);
}
-
+out:
+ if (dst_page)
+ putback_zspage(pool, class, dst_page);
if (src_page)
putback_zspage(pool, class, src_page);
@@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
}
EXPORT_SYMBOL_GPL(zs_compact);
+static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
+ struct shrink_control *sc)
+{
+ unsigned long freed;
+ struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+ shrinker);
+
+ freed = pool->num_migrated;
+ /* Compact classes and calculate compaction delta */
+ freed = zs_compact(pool) - freed;
+
+ return freed ? freed : SHRINK_STOP;
+}
+
+static unsigned long zs_shrinker_count(struct shrinker *shrinker,
+ struct shrink_control *sc)
+{
+ int i;
+ struct size_class *class;
+ unsigned long to_free = 0;
+ struct zs_pool *pool = container_of(shrinker, struct zs_pool,
+ shrinker);
+
+ if (!pool->shrinker_enabled)
+ return 0;
+
+ for (i = zs_size_classes - 1; i >= 0; i--) {
+ class = pool->size_class[i];
+ if (!class)
+ continue;
+ if (class->index != i)
+ continue;
+
+ spin_lock(&class->lock);
+ to_free += zs_can_compact(class);
+ spin_unlock(&class->lock);
+ }
+
+ return to_free;
+}
+
+static void zs_unregister_shrinker(struct zs_pool *pool)
+{
+ if (pool->shrinker_enabled) {
+ unregister_shrinker(&pool->shrinker);
+ pool->shrinker_enabled = false;
+ }
+}
+
+static int zs_register_shrinker(struct zs_pool *pool)
+{
+ pool->shrinker.scan_objects = zs_shrinker_scan;
+ pool->shrinker.count_objects = zs_shrinker_count;
+ pool->shrinker.batch = 0;
+ pool->shrinker.seeks = DEFAULT_SEEKS;
+
+ return register_shrinker(&pool->shrinker);
+}
+
/**
* zs_create_pool - Creates an allocation pool to work from.
* @flags: allocation flags used to allocate pool metadata
@@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
if (zs_pool_stat_create(name, pool))
goto err;
+ /* Not critical, we still can use the pool */
+ if (zs_register_shrinker(pool) == 0)
+ pool->shrinker_enabled = true;
return pool;
err:
@@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
{
int i;
+ zs_unregister_shrinker(pool);
zs_pool_stat_destroy(pool);
for (i = 0; i < zs_size_classes; i++) {
--
2.4.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 12:13 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:13 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Hello,
Minchan, I didn't publish this patch separately yet, mostly to keep
the discussion in on thread. If we decide that this patch is good
enough, I'll resubmit it separately.
I did some synthetic testing. And (not surprising at all) its not so
clear. Any
I used a modified zsmalloc debug stats (to also account and report ZS_FULL
zspages). Automatic compaction was disabled.
the results are:
almost_full full almost_empty obj_allocated obj_used pages_used
Base
Total 3 163 25 2265 1691 302
Total 2 161 26 2297 1688 298
Total 2 145 27 2396 1701 311
Total 3 152 26 2364 1696 312
Total 3 162 25 2243 1701 302
Patched
Total 3 155 22 2259 1691 293
Total 4 153 20 2177 1697 292
Total 2 157 23 2229 1696 298
Total 2 164 24 2242 1694 301
Total 2 159 24 2286 1696 301
Sooo... I don't know. The numbers are weird. On my x86_64 I saw somewhat
lowered 'almost_empty', 'obj_allocated', 'obj_used', 'pages_used'. But
it's a bit suspicious.
The patch was not expected to dramatically improve things anyway. It's
rather a theoretical improvement -- we sometimes keep busiest zspages first
and, at the same time, we can re-use recently used zspages.
I think it makes sense to also consider 'fullness_group fullness' in
insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
IOW, something like this
---
mm/zsmalloc.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 692b7dc..d576397 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
* We want to see more ZS_FULL pages and less almost
* empty/full. Put pages with higher ->inuse first.
*/
- if (page->inuse < (*head)->inuse)
- list_add_tail(&page->lru, &(*head)->lru);
- else
+ if (fullness == ZS_ALMOST_FULL ||
+ (page->inuse >= (*head)->inuse))
list_add(&page->lru, &(*head)->lru);
+ else
+ list_add_tail(&page->lru, &(*head)->lru);
}
*head = page;
---
test script
modprobe zram
echo 4 > /sys/block/zram0/max_comp_streams
echo lzo > /sys/block/zram0/comp_algorithm
echo 3g > /sys/block/zram0/disksize
mkfs.ext4 /dev/zram0
mount -o relatime,defaults /dev/zram0 /zram
cd /zram/
sync
for i in {1..8192}; do
dd if=/media/dump/down/zero_file of=/zram/$i iflag=direct bs=4K count=20 > /dev/null 2>&1
done
sync
head -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
tail -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
cd /
umount /zram
rmmod zram
-ss
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-18 12:13 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:13 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Hello,
Minchan, I didn't publish this patch separately yet, mostly to keep
the discussion in on thread. If we decide that this patch is good
enough, I'll resubmit it separately.
I did some synthetic testing. And (not surprising at all) its not so
clear. Any
I used a modified zsmalloc debug stats (to also account and report ZS_FULL
zspages). Automatic compaction was disabled.
the results are:
almost_full full almost_empty obj_allocated obj_used pages_used
Base
Total 3 163 25 2265 1691 302
Total 2 161 26 2297 1688 298
Total 2 145 27 2396 1701 311
Total 3 152 26 2364 1696 312
Total 3 162 25 2243 1701 302
Patched
Total 3 155 22 2259 1691 293
Total 4 153 20 2177 1697 292
Total 2 157 23 2229 1696 298
Total 2 164 24 2242 1694 301
Total 2 159 24 2286 1696 301
Sooo... I don't know. The numbers are weird. On my x86_64 I saw somewhat
lowered 'almost_empty', 'obj_allocated', 'obj_used', 'pages_used'. But
it's a bit suspicious.
The patch was not expected to dramatically improve things anyway. It's
rather a theoretical improvement -- we sometimes keep busiest zspages first
and, at the same time, we can re-use recently used zspages.
I think it makes sense to also consider 'fullness_group fullness' in
insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
IOW, something like this
---
mm/zsmalloc.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 692b7dc..d576397 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
* We want to see more ZS_FULL pages and less almost
* empty/full. Put pages with higher ->inuse first.
*/
- if (page->inuse < (*head)->inuse)
- list_add_tail(&page->lru, &(*head)->lru);
- else
+ if (fullness == ZS_ALMOST_FULL ||
+ (page->inuse >= (*head)->inuse))
list_add(&page->lru, &(*head)->lru);
+ else
+ list_add_tail(&page->lru, &(*head)->lru);
}
*head = page;
---
test script
modprobe zram
echo 4 > /sys/block/zram0/max_comp_streams
echo lzo > /sys/block/zram0/comp_algorithm
echo 3g > /sys/block/zram0/disksize
mkfs.ext4 /dev/zram0
mount -o relatime,defaults /dev/zram0 /zram
cd /zram/
sync
for i in {1..8192}; do
dd if=/media/dump/down/zero_file of=/zram/$i iflag=direct bs=4K count=20 > /dev/null 2>&1
done
sync
head -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
tail -n 1 /sys/kernel/debug/zsmalloc/zram0/classes
cd /
umount /zram
rmmod zram
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCH v3 0/7] introduce automatic pool compaction
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-18 12:17 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:17 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Minchan, FYI,
I re-done the test (untouched almost_empty waterline).
Kernel log
test started [ 788.064886]
test ended [ 4025.914190]
test (doing `cp' in parallel):
(a) for i in {1..X}; do cp -R ~/git .; sync; rm -fr git/; done
# compiled kernel, with object files. 2.2G
(b) for i in {1..X}; do cp -R ~/linux/ .; sync; rm -fr linux/; done
(c) for i in {1..X}; do cp -R ~/glibc/ .; sync; rm -fr glibc/; done
Minimal si_meminfo(&si)->si.freeram observed on the system was: 6390
cat /sys/block/zram0/stat
31253 0 250024 316 12281689 0 98253512 141263 0 141590 141763
cat /sys/block/zram0/mm_stat
3183374336 2105262569 2140762112 0 2821394432 1864 358173
The results are:
compaction nr:2335 (full:953 part:60956)
ratio: 0.01539 (~1.53% of classes were fully compacted)
More or less same numbers.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCH v3 0/7] introduce automatic pool compaction
@ 2015-06-18 12:17 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:17 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
Minchan, FYI,
I re-done the test (untouched almost_empty waterline).
Kernel log
test started [ 788.064886]
test ended [ 4025.914190]
test (doing `cp' in parallel):
(a) for i in {1..X}; do cp -R ~/git .; sync; rm -fr git/; done
# compiled kernel, with object files. 2.2G
(b) for i in {1..X}; do cp -R ~/linux/ .; sync; rm -fr linux/; done
(c) for i in {1..X}; do cp -R ~/glibc/ .; sync; rm -fr glibc/; done
Minimal si_meminfo(&si)->si.freeram observed on the system was: 6390
cat /sys/block/zram0/stat
31253 0 250024 316 12281689 0 98253512 141263 0 141590 141763
cat /sys/block/zram0/mm_stat
3183374336 2105262569 2140762112 0 2821394432 1864 358173
The results are:
compaction nr:2335 (full:953 part:60956)
ratio: 0.01539 (~1.53% of classes were fully compacted)
More or less same numbers.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 12:13 ` Sergey Senozhatsky
@ 2015-06-18 12:45 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:45 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Minchan Kim, Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I think it makes sense to also consider 'fullness_group fullness' in
> insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
> head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
>
> IOW, something like this
>
> ---
>
> mm/zsmalloc.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 692b7dc..d576397 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
> * We want to see more ZS_FULL pages and less almost
> * empty/full. Put pages with higher ->inuse first.
> */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> + if (fullness == ZS_ALMOST_FULL ||
> + (page->inuse >= (*head)->inuse))
> list_add(&page->lru, &(*head)->lru);
> + else
> + list_add_tail(&page->lru, &(*head)->lru);
> }
>
> *head = page;
>
almost_full full almost_empty obj_allocated obj_used pages_used
Base
Total 3 168 26 2324 1822 307
Total 3 167 29 2391 1815 314
Total 5 172 25 2392 1827 313
Patched
Total 4 180 27 2425 1835 327
Total 4 169 27 2405 1825 312
Total 2 176 28 2452 1825 315
... no chance the test is right.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-18 12:45 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:45 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Minchan Kim, Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I think it makes sense to also consider 'fullness_group fullness' in
> insert_zspage(). Unconditionally put ZS_ALMOST_FULL pages to list
> head, or (if zspage is !ZS_ALMOST_FULL) compage ->inuse.
>
> IOW, something like this
>
> ---
>
> mm/zsmalloc.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 692b7dc..d576397 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
> * We want to see more ZS_FULL pages and less almost
> * empty/full. Put pages with higher ->inuse first.
> */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> + if (fullness == ZS_ALMOST_FULL ||
> + (page->inuse >= (*head)->inuse))
> list_add(&page->lru, &(*head)->lru);
> + else
> + list_add_tail(&page->lru, &(*head)->lru);
> }
>
> *head = page;
>
almost_full full almost_empty obj_allocated obj_used pages_used
Base
Total 3 168 26 2324 1822 307
Total 3 167 29 2391 1815 314
Total 5 172 25 2392 1827 313
Patched
Total 4 180 27 2425 1835 327
Total 4 169 27 2405 1825 312
Total 2 176 28 2452 1825 315
... no chance the test is right.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 12:13 ` Sergey Senozhatsky
@ 2015-06-18 12:48 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:48 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I used a modified zsmalloc debug stats (to also account and report ZS_FULL
> zspages).
just in case. account and report per-class ZS_FULL numbers.
----
mm/zsmalloc.c | 29 ++++++++++++++++++++---------
1 file changed, 20 insertions(+), 9 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e0d28a2..97ca25d 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -166,6 +166,7 @@ enum zs_stat_type {
OBJ_USED,
CLASS_ALMOST_FULL,
CLASS_ALMOST_EMPTY,
+ CLASS_FULL,
NR_ZS_STAT_TYPE,
};
@@ -483,13 +484,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
struct zs_pool *pool = s->private;
struct size_class *class;
int objs_per_zspage;
- unsigned long class_almost_full, class_almost_empty;
+ unsigned long class_almost_full, class_full, class_almost_empty;
unsigned long obj_allocated, obj_used, pages_used;
- unsigned long total_class_almost_full = 0, total_class_almost_empty = 0;
+ unsigned long total_class_almost_full = 0, total_class_full = 0, total_class_almost_empty = 0;
unsigned long total_objs = 0, total_used_objs = 0, total_pages = 0;
- seq_printf(s, " %5s %5s %11s %12s %13s %10s %10s %16s\n",
- "class", "size", "almost_full", "almost_empty",
+ seq_printf(s, " %5s %5s %11s %12s %12s %13s %10s %10s %16s\n",
+ "class", "size", "almost_full", "full", "almost_empty",
"obj_allocated", "obj_used", "pages_used",
"pages_per_zspage");
@@ -501,6 +502,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
spin_lock(&class->lock);
class_almost_full = zs_stat_get(class, CLASS_ALMOST_FULL);
+ class_full = zs_stat_get(class, CLASS_FULL);
class_almost_empty = zs_stat_get(class, CLASS_ALMOST_EMPTY);
obj_allocated = zs_stat_get(class, OBJ_ALLOCATED);
obj_used = zs_stat_get(class, OBJ_USED);
@@ -511,12 +513,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
pages_used = obj_allocated / objs_per_zspage *
class->pages_per_zspage;
- seq_printf(s, " %5u %5u %11lu %12lu %13lu %10lu %10lu %16d\n",
- i, class->size, class_almost_full, class_almost_empty,
+ seq_printf(s, " %5u %5u %11lu %12lu %12lu %13lu %10lu %10lu %16d\n",
+ i, class->size, class_almost_full, class_full, class_almost_empty,
obj_allocated, obj_used, pages_used,
class->pages_per_zspage);
total_class_almost_full += class_almost_full;
+ total_class_full += class_full;
total_class_almost_empty += class_almost_empty;
total_objs += obj_allocated;
total_used_objs += obj_used;
@@ -524,8 +527,9 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
}
seq_puts(s, "\n");
- seq_printf(s, " %5s %5s %11lu %12lu %13lu %10lu %10lu\n",
+ seq_printf(s, " %5s %5s %11lu %12lu %12lu %13lu %10lu %10lu\n",
"Total", "", total_class_almost_full,
+ total_class_full,
total_class_almost_empty, total_objs,
total_used_objs, total_pages);
@@ -652,7 +656,10 @@ static void insert_zspage(struct page *page, struct size_class *class,
}
*head = page;
- zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+ if (fullness == ZS_FULL)
+ zs_stat_inc(class, CLASS_FULL, 1);
+ else
+ zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
}
@@ -679,7 +686,10 @@ static void remove_zspage(struct page *page, struct size_class *class,
struct page, lru);
list_del_init(&page->lru);
- zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
+ if (fullness == ZS_FULL)
+ zs_stat_dec(class, CLASS_FULL, 1);
+ else
+ zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
}
@@ -1410,6 +1420,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size)
spin_lock(&class->lock);
zs_stat_inc(class, OBJ_ALLOCATED, get_maxobj_per_zspage(
class->size, class->pages_per_zspage));
+ zs_stat_inc(class, CLASS_FULL, 1);
}
obj = obj_malloc(first_page, class, handle);
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-18 12:48 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 12:48 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> I used a modified zsmalloc debug stats (to also account and report ZS_FULL
> zspages).
just in case. account and report per-class ZS_FULL numbers.
----
mm/zsmalloc.c | 29 ++++++++++++++++++++---------
1 file changed, 20 insertions(+), 9 deletions(-)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e0d28a2..97ca25d 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -166,6 +166,7 @@ enum zs_stat_type {
OBJ_USED,
CLASS_ALMOST_FULL,
CLASS_ALMOST_EMPTY,
+ CLASS_FULL,
NR_ZS_STAT_TYPE,
};
@@ -483,13 +484,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
struct zs_pool *pool = s->private;
struct size_class *class;
int objs_per_zspage;
- unsigned long class_almost_full, class_almost_empty;
+ unsigned long class_almost_full, class_full, class_almost_empty;
unsigned long obj_allocated, obj_used, pages_used;
- unsigned long total_class_almost_full = 0, total_class_almost_empty = 0;
+ unsigned long total_class_almost_full = 0, total_class_full = 0, total_class_almost_empty = 0;
unsigned long total_objs = 0, total_used_objs = 0, total_pages = 0;
- seq_printf(s, " %5s %5s %11s %12s %13s %10s %10s %16s\n",
- "class", "size", "almost_full", "almost_empty",
+ seq_printf(s, " %5s %5s %11s %12s %12s %13s %10s %10s %16s\n",
+ "class", "size", "almost_full", "full", "almost_empty",
"obj_allocated", "obj_used", "pages_used",
"pages_per_zspage");
@@ -501,6 +502,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
spin_lock(&class->lock);
class_almost_full = zs_stat_get(class, CLASS_ALMOST_FULL);
+ class_full = zs_stat_get(class, CLASS_FULL);
class_almost_empty = zs_stat_get(class, CLASS_ALMOST_EMPTY);
obj_allocated = zs_stat_get(class, OBJ_ALLOCATED);
obj_used = zs_stat_get(class, OBJ_USED);
@@ -511,12 +513,13 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
pages_used = obj_allocated / objs_per_zspage *
class->pages_per_zspage;
- seq_printf(s, " %5u %5u %11lu %12lu %13lu %10lu %10lu %16d\n",
- i, class->size, class_almost_full, class_almost_empty,
+ seq_printf(s, " %5u %5u %11lu %12lu %12lu %13lu %10lu %10lu %16d\n",
+ i, class->size, class_almost_full, class_full, class_almost_empty,
obj_allocated, obj_used, pages_used,
class->pages_per_zspage);
total_class_almost_full += class_almost_full;
+ total_class_full += class_full;
total_class_almost_empty += class_almost_empty;
total_objs += obj_allocated;
total_used_objs += obj_used;
@@ -524,8 +527,9 @@ static int zs_stats_size_show(struct seq_file *s, void *v)
}
seq_puts(s, "\n");
- seq_printf(s, " %5s %5s %11lu %12lu %13lu %10lu %10lu\n",
+ seq_printf(s, " %5s %5s %11lu %12lu %12lu %13lu %10lu %10lu\n",
"Total", "", total_class_almost_full,
+ total_class_full,
total_class_almost_empty, total_objs,
total_used_objs, total_pages);
@@ -652,7 +656,10 @@ static void insert_zspage(struct page *page, struct size_class *class,
}
*head = page;
- zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+ if (fullness == ZS_FULL)
+ zs_stat_inc(class, CLASS_FULL, 1);
+ else
+ zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
}
@@ -679,7 +686,10 @@ static void remove_zspage(struct page *page, struct size_class *class,
struct page, lru);
list_del_init(&page->lru);
- zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
+ if (fullness == ZS_FULL)
+ zs_stat_dec(class, CLASS_FULL, 1);
+ else
+ zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
}
@@ -1410,6 +1420,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size)
spin_lock(&class->lock);
zs_stat_inc(class, OBJ_ALLOCATED, get_maxobj_per_zspage(
class->size, class->pages_per_zspage));
+ zs_stat_inc(class, CLASS_FULL, 1);
}
obj = obj_malloc(first_page, class, handle);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 12:13 ` Sergey Senozhatsky
@ 2015-06-18 14:43 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 14:43 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
> * We want to see more ZS_FULL pages and less almost
> * empty/full. Put pages with higher ->inuse first.
> */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> + if (fullness == ZS_ALMOST_FULL ||
> + (page->inuse >= (*head)->inuse))
> list_add(&page->lru, &(*head)->lru);
> + else
> + list_add_tail(&page->lru, &(*head)->lru);
> }
>
> *head = page;
oh, dear. what I was thinking of. this is just stupid. please ignore
this part.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-18 14:43 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-18 14:43 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky,
Sergey Senozhatsky
On (06/18/15 21:13), Sergey Senozhatsky wrote:
> @@ -645,10 +645,11 @@ static void insert_zspage(struct page *page, struct size_class *class,
> * We want to see more ZS_FULL pages and less almost
> * empty/full. Put pages with higher ->inuse first.
> */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> + if (fullness == ZS_ALMOST_FULL ||
> + (page->inuse >= (*head)->inuse))
> list_add(&page->lru, &(*head)->lru);
> + else
> + list_add_tail(&page->lru, &(*head)->lru);
> }
>
> *head = page;
oh, dear. what I was thinking of. this is just stupid. please ignore
this part.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-29 6:40 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:40 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> Always account per-class `zs_size_stat' stats. This data will
> help us make better decisions during compaction. We are especially
> interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> class compaction will result in any memory gain.
>
> For instance, we know the number of allocated objects in the class,
> the number of objects being used (so we also know how many objects
> are not used) and the number of objects per-page. So we can ensure
> if we have enough unused objects to form at least one ZS_EMPTY
> zspage during compaction.
>
> We calculate this value on per-class basis so we can calculate a
> total number of zspages that can be released. Which is exactly what
> a shrinker wants to know.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
@ 2015-06-29 6:40 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:40 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> Always account per-class `zs_size_stat' stats. This data will
> help us make better decisions during compaction. We are especially
> interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> class compaction will result in any memory gain.
>
> For instance, we know the number of allocated objects in the class,
> the number of objects being used (so we also know how many objects
> are not used) and the number of objects per-page. So we can ensure
> if we have enough unused objects to form at least one ZS_EMPTY
> zspage during compaction.
>
> We calculate this value on per-class basis so we can calculate a
> total number of zspages that can be released. Which is exactly what
> a shrinker wants to know.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Minchan Kim <minchan@kernel.org>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-29 6:45 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:45 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:41PM +0900, Sergey Senozhatsky wrote:
> This function checks if class compaction will free any pages.
> Rephrasing -- do we have enough unused objects to form at least
> one ZS_EMPTY page and free it. It aborts compaction if class
> compaction will not result in any (further) savings.
>
> EXAMPLE (this debug output is not part of this patch set):
>
> -- class size
> -- number of allocated objects
> -- number of used objects
> -- max objects per zspage
> -- pages per zspage
> -- estimated number of pages that will be freed
>
> [..]
> class-512 objs:544 inuse:540 maxobj-per-zspage:8 pages-per-zspage:1 zspages-to-free:0
> ... class-512 compaction is useless. break
> class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
> class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
> class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
> ... class-496 compaction is useless. break
> class-448 objs:657 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:4
> class-448 objs:648 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:3
> class-448 objs:639 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:2
> class-448 objs:630 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:1
> class-448 objs:621 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:0
> ... class-448 compaction is useless. break
> class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
> class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
> ... class-432 compaction is useless. break
> class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
> class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
> class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
> ... class-416 compaction is useless. break
> class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
> class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
> ... class-400 compaction is useless. break
> class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
> ... class-384 compaction is useless. break
> [..]
>
> Every "compaction is useless" indicates that we saved CPU cycles.
>
> class-512 has
> 544 object allocated
> 540 objects used
> 8 objects per-page
>
> Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
> migrate all of its objects and free this zspage; so compaction will not
> make a lot of sense, it's better to just leave it as is.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
> mm/zsmalloc.c | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 4b6f12e..18e10a4 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
> return page;
> }
>
> +/*
> + * Make sure that we actually can compact this class,
> + * IOW if migration will release at least one szpage.
zspage,
> + *
> + * Should be called under class->lock
Please comment about return.
> + */
> +static unsigned long zs_can_compact(struct size_class *class)
> +{
> + /*
> + * Calculate how many unused allocated objects we
> + * have and see if we can free any zspages. Otherwise,
> + * compaction can just move objects back and forth w/o
> + * any memory gain.
> + */
> + unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> + zs_stat_get(class, OBJ_USED);
> +
I want to check one more thing.
We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
In this implementation, compaction cannot have a source so
it would better to bail out.
IOW,
if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
return 0;
> + obj_wasted /= get_maxobj_per_zspage(class->size,
> + class->pages_per_zspage);
> + return obj_wasted;
> +}
> +
> static unsigned long __zs_compact(struct zs_pool *pool,
> struct size_class *class)
> {
> @@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
>
> BUG_ON(!is_first_page(src_page));
>
> + if (!zs_can_compact(class))
> + break;
> +
> cc.index = 0;
> cc.s_page = src_page;
>
> --
> 2.4.4
>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
@ 2015-06-29 6:45 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:45 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:41PM +0900, Sergey Senozhatsky wrote:
> This function checks if class compaction will free any pages.
> Rephrasing -- do we have enough unused objects to form at least
> one ZS_EMPTY page and free it. It aborts compaction if class
> compaction will not result in any (further) savings.
>
> EXAMPLE (this debug output is not part of this patch set):
>
> -- class size
> -- number of allocated objects
> -- number of used objects
> -- max objects per zspage
> -- pages per zspage
> -- estimated number of pages that will be freed
>
> [..]
> class-512 objs:544 inuse:540 maxobj-per-zspage:8 pages-per-zspage:1 zspages-to-free:0
> ... class-512 compaction is useless. break
> class-496 objs:660 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:2
> class-496 objs:627 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:1
> class-496 objs:594 inuse:570 maxobj-per-zspage:33 pages-per-zspage:4 zspages-to-free:0
> ... class-496 compaction is useless. break
> class-448 objs:657 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:4
> class-448 objs:648 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:3
> class-448 objs:639 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:2
> class-448 objs:630 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:1
> class-448 objs:621 inuse:617 maxobj-per-zspage:9 pages-per-zspage:1 zspages-to-free:0
> ... class-448 compaction is useless. break
> class-432 objs:728 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:1
> class-432 objs:700 inuse:685 maxobj-per-zspage:28 pages-per-zspage:3 zspages-to-free:0
> ... class-432 compaction is useless. break
> class-416 objs:819 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:2
> class-416 objs:780 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:1
> class-416 objs:741 inuse:705 maxobj-per-zspage:39 pages-per-zspage:4 zspages-to-free:0
> ... class-416 compaction is useless. break
> class-400 objs:690 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:1
> class-400 objs:680 inuse:674 maxobj-per-zspage:10 pages-per-zspage:1 zspages-to-free:0
> ... class-400 compaction is useless. break
> class-384 objs:736 inuse:709 maxobj-per-zspage:32 pages-per-zspage:3 zspages-to-free:0
> ... class-384 compaction is useless. break
> [..]
>
> Every "compaction is useless" indicates that we saved CPU cycles.
>
> class-512 has
> 544 object allocated
> 540 objects used
> 8 objects per-page
>
> Even if we have a ALMOST_EMPTY zspage, we still don't have enough room to
> migrate all of its objects and free this zspage; so compaction will not
> make a lot of sense, it's better to just leave it as is.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
> mm/zsmalloc.c | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 4b6f12e..18e10a4 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1684,6 +1684,28 @@ static struct page *isolate_source_page(struct size_class *class)
> return page;
> }
>
> +/*
> + * Make sure that we actually can compact this class,
> + * IOW if migration will release at least one szpage.
zspage,
> + *
> + * Should be called under class->lock
Please comment about return.
> + */
> +static unsigned long zs_can_compact(struct size_class *class)
> +{
> + /*
> + * Calculate how many unused allocated objects we
> + * have and see if we can free any zspages. Otherwise,
> + * compaction can just move objects back and forth w/o
> + * any memory gain.
> + */
> + unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> + zs_stat_get(class, OBJ_USED);
> +
I want to check one more thing.
We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
In this implementation, compaction cannot have a source so
it would better to bail out.
IOW,
if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
return 0;
> + obj_wasted /= get_maxobj_per_zspage(class->size,
> + class->pages_per_zspage);
> + return obj_wasted;
> +}
> +
> static unsigned long __zs_compact(struct zs_pool *pool,
> struct size_class *class)
> {
> @@ -1697,6 +1719,9 @@ static unsigned long __zs_compact(struct zs_pool *pool,
>
> BUG_ON(!is_first_page(src_page));
>
> + if (!zs_can_compact(class))
> + break;
> +
> cc.index = 0;
> cc.s_page = src_page;
>
> --
> 2.4.4
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-29 6:52 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:52 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:39PM +0900, Sergey Senozhatsky wrote:
> We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
> pages. Put a page with higher ->inuse count first within its
> ->fullness_list, which will give us better chances to fill up this
> page with new objects (find_get_zspage() return ->fullness_list head
> for new object allocation), so some zspages will become
> ZS_ALMOST_FULL/ZS_FULL quicker.
>
> It performs a trivial and cheap ->inuse compare which does not slow
> down zsmalloc, and in the worst case it keeps the list pages not in
> any particular order, just like we do it now.
>
> A more expensive solution could sort fullness_list by ->inuse count.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
> mm/zsmalloc.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 7d816c2..6e2ebb6 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
> return;
>
> head = &class->fullness_list[fullness];
> - if (*head)
> - list_add_tail(&page->lru, &(*head)->lru);
> + if (*head) {
> + /*
> + * We want to see more ZS_FULL pages and less almost
> + * empty/full. Put pages with higher ->inuse first.
> + */
> + if (page->inuse < (*head)->inuse)
> + list_add_tail(&page->lru, &(*head)->lru);
> + else
> + list_add(&page->lru, &(*head)->lru);
> + }
>
> *head = page;
Why do you want to always put @page in the head?
How about this?
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e8cb31c..1c5fde9 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
if (fullness >= _ZS_NR_FULLNESS_GROUPS)
return;
+ zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+ CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+
head = &class->fullness_list[fullness];
- if (*head) {
- /*
- * We want to see more ZS_FULL pages and less almost
- * empty/full. Put pages with higher ->inuse first.
- */
- if (page->inuse < (*head)->inuse)
- list_add_tail(&page->lru, &(*head)->lru);
- else
- list_add(&page->lru, &(*head)->lru);
+ if (!*head) {
+ *head = page;
+ return;
}
- *head = page;
- zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
- CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+ /*
+ * We want to see more ZS_FULL pages and less almost
+ * empty/full. Put pages with higher ->inuse first.
+ */
+ list_add_tail(&page->lru, &(*head)->lru);
+ if (page->inuse >= (*head)->inuse)
+ *head = page;
}
/*
--
1.7.9.5
> zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> --
> 2.4.4
>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-29 6:52 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 6:52 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:39PM +0900, Sergey Senozhatsky wrote:
> We want to see more ZS_FULL pages and less ZS_ALMOST_{FULL, EMPTY}
> pages. Put a page with higher ->inuse count first within its
> ->fullness_list, which will give us better chances to fill up this
> page with new objects (find_get_zspage() return ->fullness_list head
> for new object allocation), so some zspages will become
> ZS_ALMOST_FULL/ZS_FULL quicker.
>
> It performs a trivial and cheap ->inuse compare which does not slow
> down zsmalloc, and in the worst case it keeps the list pages not in
> any particular order, just like we do it now.
>
> A more expensive solution could sort fullness_list by ->inuse count.
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
> mm/zsmalloc.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 7d816c2..6e2ebb6 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -659,8 +659,16 @@ static void insert_zspage(struct page *page, struct size_class *class,
> return;
>
> head = &class->fullness_list[fullness];
> - if (*head)
> - list_add_tail(&page->lru, &(*head)->lru);
> + if (*head) {
> + /*
> + * We want to see more ZS_FULL pages and less almost
> + * empty/full. Put pages with higher ->inuse first.
> + */
> + if (page->inuse < (*head)->inuse)
> + list_add_tail(&page->lru, &(*head)->lru);
> + else
> + list_add(&page->lru, &(*head)->lru);
> + }
>
> *head = page;
Why do you want to always put @page in the head?
How about this?
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index e8cb31c..1c5fde9 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
if (fullness >= _ZS_NR_FULLNESS_GROUPS)
return;
+ zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
+ CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+
head = &class->fullness_list[fullness];
- if (*head) {
- /*
- * We want to see more ZS_FULL pages and less almost
- * empty/full. Put pages with higher ->inuse first.
- */
- if (page->inuse < (*head)->inuse)
- list_add_tail(&page->lru, &(*head)->lru);
- else
- list_add(&page->lru, &(*head)->lru);
+ if (!*head) {
+ *head = page;
+ return;
}
- *head = page;
- zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
- CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
+ /*
+ * We want to see more ZS_FULL pages and less almost
+ * empty/full. Put pages with higher ->inuse first.
+ */
+ list_add_tail(&page->lru, &(*head)->lru);
+ if (page->inuse >= (*head)->inuse)
+ *head = page;
}
/*
--
1.7.9.5
> zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> --
> 2.4.4
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
2015-06-18 11:46 ` Sergey Senozhatsky
@ 2015-06-29 7:07 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 7:07 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:44PM +0900, Sergey Senozhatsky wrote:
> Perform automatic pool compaction by a shrinker when system
> is getting tight on memory.
>
> User-space has a very little knowledge regarding zsmalloc fragmentation
> and basically has no mechanism to tell whether compaction will result
> in any memory gain. Another issue is that user space is not always
> aware of the fact that system is getting tight on memory. Which leads
> to very uncomfortable scenarios when user space may start issuing
> compaction 'randomly' or from crontab (for example). Fragmentation
> is not always necessarily bad, allocated and unused objects, after all,
> may be filled with the data later, w/o the need of allocating a new
> zspage. On the other hand, we obviously don't want to waste memory
> when systems needs it.
>
> Compaction now has a relatively quick pool scan so we are able to
> estimate the number of pages that will be freed easily, which makes it
> possible to call this function from a shrinker->count_objects() callback.
> We also abort compaction as soon as we detect that we can't free any
> pages any more, preventing wasteful objects migrations.
>
> Minchan Kim proposed to use the shrinker (the original patch was too
> aggressive and was attempting to perform compaction for every
> ALMOST_EMPTY zspage).
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Suggested-by: Minchan Kim <minchan@kernel.org>
> ---
> mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 71 insertions(+), 7 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index c9aea0a..692b7dc 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -247,7 +247,9 @@ struct zs_pool {
> atomic_long_t pages_allocated;
> /* How many objects were migrated */
> unsigned long num_migrated;
> -
> + /* Compact classes */
> + struct shrinker shrinker;
> + bool shrinker_enabled;
> #ifdef CONFIG_ZSMALLOC_STAT
> struct dentry *stat_dentry;
> #endif
> @@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>
> while ((dst_page = isolate_target_page(class))) {
> cc.d_page = dst_page;
> - /*
> - * If there is no more space in dst_page, resched
> - * and see if anyone had allocated another zspage.
> - */
> +
> if (!migrate_zspage(pool, class, &cc))
> - break;
> + goto out;
It should retry with another target_page instead of going out.
>
> putback_zspage(pool, class, dst_page);
> }
> @@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
> cond_resched();
> spin_lock(&class->lock);
> }
> -
> +out:
> + if (dst_page)
> + putback_zspage(pool, class, dst_page);
> if (src_page)
> putback_zspage(pool, class, src_page);
>
> @@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
> }
> EXPORT_SYMBOL_GPL(zs_compact);
>
> +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> + struct shrink_control *sc)
> +{
> + unsigned long freed;
> + struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> + shrinker);
> +
> + freed = pool->num_migrated;
> + /* Compact classes and calculate compaction delta */
> + freed = zs_compact(pool) - freed;
Returns migrated object count.
> +
> + return freed ? freed : SHRINK_STOP;
> +}
> +
> +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> + struct shrink_control *sc)
> +{
> + int i;
> + struct size_class *class;
> + unsigned long to_free = 0;
> + struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> + shrinker);
> +
> + if (!pool->shrinker_enabled)
> + return 0;
> +
> + for (i = zs_size_classes - 1; i >= 0; i--) {
> + class = pool->size_class[i];
> + if (!class)
> + continue;
> + if (class->index != i)
> + continue;
> +
> + spin_lock(&class->lock);
> + to_free += zs_can_compact(class);
But it returns wasted_obj / max_obj_per_zspage?
> + spin_unlock(&class->lock);
> + }
> +
> + return to_free;
> +}
> +
> +static void zs_unregister_shrinker(struct zs_pool *pool)
> +{
> + if (pool->shrinker_enabled) {
> + unregister_shrinker(&pool->shrinker);
> + pool->shrinker_enabled = false;
> + }
> +}
> +
> +static int zs_register_shrinker(struct zs_pool *pool)
> +{
> + pool->shrinker.scan_objects = zs_shrinker_scan;
> + pool->shrinker.count_objects = zs_shrinker_count;
> + pool->shrinker.batch = 0;
> + pool->shrinker.seeks = DEFAULT_SEEKS;
> +
> + return register_shrinker(&pool->shrinker);
> +}
> +
> /**
> * zs_create_pool - Creates an allocation pool to work from.
> * @flags: allocation flags used to allocate pool metadata
> @@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
> if (zs_pool_stat_create(name, pool))
> goto err;
>
> + /* Not critical, we still can use the pool */
> + if (zs_register_shrinker(pool) == 0)
> + pool->shrinker_enabled = true;
> return pool;
>
> err:
> @@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
> {
> int i;
>
> + zs_unregister_shrinker(pool);
> zs_pool_stat_destroy(pool);
>
> for (i = 0; i < zs_size_classes; i++) {
> --
> 2.4.4
>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
@ 2015-06-29 7:07 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 7:07 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Andrew Morton, linux-mm, linux-kernel, Sergey Senozhatsky
On Thu, Jun 18, 2015 at 08:46:44PM +0900, Sergey Senozhatsky wrote:
> Perform automatic pool compaction by a shrinker when system
> is getting tight on memory.
>
> User-space has a very little knowledge regarding zsmalloc fragmentation
> and basically has no mechanism to tell whether compaction will result
> in any memory gain. Another issue is that user space is not always
> aware of the fact that system is getting tight on memory. Which leads
> to very uncomfortable scenarios when user space may start issuing
> compaction 'randomly' or from crontab (for example). Fragmentation
> is not always necessarily bad, allocated and unused objects, after all,
> may be filled with the data later, w/o the need of allocating a new
> zspage. On the other hand, we obviously don't want to waste memory
> when systems needs it.
>
> Compaction now has a relatively quick pool scan so we are able to
> estimate the number of pages that will be freed easily, which makes it
> possible to call this function from a shrinker->count_objects() callback.
> We also abort compaction as soon as we detect that we can't free any
> pages any more, preventing wasteful objects migrations.
>
> Minchan Kim proposed to use the shrinker (the original patch was too
> aggressive and was attempting to perform compaction for every
> ALMOST_EMPTY zspage).
>
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Suggested-by: Minchan Kim <minchan@kernel.org>
> ---
> mm/zsmalloc.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 71 insertions(+), 7 deletions(-)
>
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index c9aea0a..692b7dc 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -247,7 +247,9 @@ struct zs_pool {
> atomic_long_t pages_allocated;
> /* How many objects were migrated */
> unsigned long num_migrated;
> -
> + /* Compact classes */
> + struct shrinker shrinker;
> + bool shrinker_enabled;
> #ifdef CONFIG_ZSMALLOC_STAT
> struct dentry *stat_dentry;
> #endif
> @@ -1730,12 +1732,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
>
> while ((dst_page = isolate_target_page(class))) {
> cc.d_page = dst_page;
> - /*
> - * If there is no more space in dst_page, resched
> - * and see if anyone had allocated another zspage.
> - */
> +
> if (!migrate_zspage(pool, class, &cc))
> - break;
> + goto out;
It should retry with another target_page instead of going out.
>
> putback_zspage(pool, class, dst_page);
> }
> @@ -1750,7 +1749,9 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
> cond_resched();
> spin_lock(&class->lock);
> }
> -
> +out:
> + if (dst_page)
> + putback_zspage(pool, class, dst_page);
> if (src_page)
> putback_zspage(pool, class, src_page);
>
> @@ -1774,6 +1775,65 @@ unsigned long zs_compact(struct zs_pool *pool)
> }
> EXPORT_SYMBOL_GPL(zs_compact);
>
> +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> + struct shrink_control *sc)
> +{
> + unsigned long freed;
> + struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> + shrinker);
> +
> + freed = pool->num_migrated;
> + /* Compact classes and calculate compaction delta */
> + freed = zs_compact(pool) - freed;
Returns migrated object count.
> +
> + return freed ? freed : SHRINK_STOP;
> +}
> +
> +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> + struct shrink_control *sc)
> +{
> + int i;
> + struct size_class *class;
> + unsigned long to_free = 0;
> + struct zs_pool *pool = container_of(shrinker, struct zs_pool,
> + shrinker);
> +
> + if (!pool->shrinker_enabled)
> + return 0;
> +
> + for (i = zs_size_classes - 1; i >= 0; i--) {
> + class = pool->size_class[i];
> + if (!class)
> + continue;
> + if (class->index != i)
> + continue;
> +
> + spin_lock(&class->lock);
> + to_free += zs_can_compact(class);
But it returns wasted_obj / max_obj_per_zspage?
> + spin_unlock(&class->lock);
> + }
> +
> + return to_free;
> +}
> +
> +static void zs_unregister_shrinker(struct zs_pool *pool)
> +{
> + if (pool->shrinker_enabled) {
> + unregister_shrinker(&pool->shrinker);
> + pool->shrinker_enabled = false;
> + }
> +}
> +
> +static int zs_register_shrinker(struct zs_pool *pool)
> +{
> + pool->shrinker.scan_objects = zs_shrinker_scan;
> + pool->shrinker.count_objects = zs_shrinker_count;
> + pool->shrinker.batch = 0;
> + pool->shrinker.seeks = DEFAULT_SEEKS;
> +
> + return register_shrinker(&pool->shrinker);
> +}
> +
> /**
> * zs_create_pool - Creates an allocation pool to work from.
> * @flags: allocation flags used to allocate pool metadata
> @@ -1859,6 +1919,9 @@ struct zs_pool *zs_create_pool(char *name, gfp_t flags)
> if (zs_pool_stat_create(name, pool))
> goto err;
>
> + /* Not critical, we still can use the pool */
> + if (zs_register_shrinker(pool) == 0)
> + pool->shrinker_enabled = true;
> return pool;
>
> err:
> @@ -1871,6 +1934,7 @@ void zs_destroy_pool(struct zs_pool *pool)
> {
> int i;
>
> + zs_unregister_shrinker(pool);
> zs_pool_stat_destroy(pool);
>
> for (i = 0; i < zs_size_classes; i++) {
> --
> 2.4.4
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
2015-06-29 7:07 ` Minchan Kim
@ 2015-06-29 8:57 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 8:57 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
Hello,
thanks for review.
On (06/29/15 16:07), Minchan Kim wrote:
[..]
> > if (!migrate_zspage(pool, class, &cc))
> > - break;
> > + goto out;
>
> It should retry with another target_page instead of going out.
yep.
[..]
> > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > + struct shrink_control *sc)
> > +{
[..]
>
> Returns migrated object count.
>
[..]
> > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > + struct shrink_control *sc)
> > +{
[..]
>
> But it returns wasted_obj / max_obj_per_zspage?
>
Good catch.
So, __zs_compact() and zs_shrinker_count() are ok. Returning
"wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
sense there. The only place is zs_shrinker_scan()->zs_compact().
Hm, I can think of:
(a) We can change zs_compact() to return the total number of
freed zspages. That will not really change a user visible
interface. We export (fwiw) the number of compacted objects
in mm_stat. Basically, this is internal zsmalloc() counter and
no user space program can ever do anything with that data. From
that prospective we will just replace one senseless number with
another (equally meaningless) one.
(b) replace zs_compact() call in zs_shrinker_scan() with a class loop
1764 int i;
1765 unsigned long nr_migrated = 0;
1766 struct size_class *class;
1767
1768 for (i = zs_size_classes - 1; i >= 0; i--) {
1769 class = pool->size_class[i];
1770 if (!class)
1771 continue;
1772 if (class->index != i)
1773 continue;
1774 nr_migrated += __zs_compact(pool, class);
1775 }
1776
1777 return nr_migrated;
But on every iteration increment nr_migrated with
"nr_migrated += just_migrated / max_obj_per_zspage"
(which will be unnecessary if zs_compact() will return the number of freed
zspages).
So, (b) is mostly fine, except that we already have several pool->size_class
loops, with same `if (!class)' and `if (class->index...)' checks; and it
asks for some sort of refactoring or... a tricky for_each_class() define.
In both cases, however, we don't tell anything valuable to user space.
Thus,
(c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
And change compaction to operate in terms of pages (PAGE_SIZE). At
least mm_stat::compacted will turn into something useful for user
space.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
@ 2015-06-29 8:57 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 8:57 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
Hello,
thanks for review.
On (06/29/15 16:07), Minchan Kim wrote:
[..]
> > if (!migrate_zspage(pool, class, &cc))
> > - break;
> > + goto out;
>
> It should retry with another target_page instead of going out.
yep.
[..]
> > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > + struct shrink_control *sc)
> > +{
[..]
>
> Returns migrated object count.
>
[..]
> > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > + struct shrink_control *sc)
> > +{
[..]
>
> But it returns wasted_obj / max_obj_per_zspage?
>
Good catch.
So, __zs_compact() and zs_shrinker_count() are ok. Returning
"wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
sense there. The only place is zs_shrinker_scan()->zs_compact().
Hm, I can think of:
(a) We can change zs_compact() to return the total number of
freed zspages. That will not really change a user visible
interface. We export (fwiw) the number of compacted objects
in mm_stat. Basically, this is internal zsmalloc() counter and
no user space program can ever do anything with that data. From
that prospective we will just replace one senseless number with
another (equally meaningless) one.
(b) replace zs_compact() call in zs_shrinker_scan() with a class loop
1764 int i;
1765 unsigned long nr_migrated = 0;
1766 struct size_class *class;
1767
1768 for (i = zs_size_classes - 1; i >= 0; i--) {
1769 class = pool->size_class[i];
1770 if (!class)
1771 continue;
1772 if (class->index != i)
1773 continue;
1774 nr_migrated += __zs_compact(pool, class);
1775 }
1776
1777 return nr_migrated;
But on every iteration increment nr_migrated with
"nr_migrated += just_migrated / max_obj_per_zspage"
(which will be unnecessary if zs_compact() will return the number of freed
zspages).
So, (b) is mostly fine, except that we already have several pool->size_class
loops, with same `if (!class)' and `if (class->index...)' checks; and it
asks for some sort of refactoring or... a tricky for_each_class() define.
In both cases, however, we don't tell anything valuable to user space.
Thus,
(c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
And change compaction to operate in terms of pages (PAGE_SIZE). At
least mm_stat::compacted will turn into something useful for user
space.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
2015-06-29 6:45 ` Minchan Kim
@ 2015-06-29 8:58 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 8:58 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
On (06/29/15 15:45), Minchan Kim wrote:
[..]
> > +/*
> > + * Make sure that we actually can compact this class,
> > + * IOW if migration will release at least one szpage.
>
> zspage,
ok.
> > + *
> > + * Should be called under class->lock
>
> Please comment about return.
ok.
> > + */
> > +static unsigned long zs_can_compact(struct size_class *class)
> > +{
> > + /*
> > + * Calculate how many unused allocated objects we
> > + * have and see if we can free any zspages. Otherwise,
> > + * compaction can just move objects back and forth w/o
> > + * any memory gain.
> > + */
> > + unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> > + zs_stat_get(class, OBJ_USED);
> > +
>
> I want to check one more thing.
>
> We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
> In this implementation, compaction cannot have a source so
> it would better to bail out.
> IOW,
>
> if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
> return 0;
ok.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function
@ 2015-06-29 8:58 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 8:58 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
On (06/29/15 15:45), Minchan Kim wrote:
[..]
> > +/*
> > + * Make sure that we actually can compact this class,
> > + * IOW if migration will release at least one szpage.
>
> zspage,
ok.
> > + *
> > + * Should be called under class->lock
>
> Please comment about return.
ok.
> > + */
> > +static unsigned long zs_can_compact(struct size_class *class)
> > +{
> > + /*
> > + * Calculate how many unused allocated objects we
> > + * have and see if we can free any zspages. Otherwise,
> > + * compaction can just move objects back and forth w/o
> > + * any memory gain.
> > + */
> > + unsigned long obj_wasted = zs_stat_get(class, OBJ_ALLOCATED) -
> > + zs_stat_get(class, OBJ_USED);
> > +
>
> I want to check one more thing.
>
> We could have lots of ZS_ALMOST_FULL but no ZS_ALMOST_EMPTY.
> In this implementation, compaction cannot have a source so
> it would better to bail out.
> IOW,
>
> if (!zs_stat_get(class, CLASS_ALMOST_EMPTY))
> return 0;
ok.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
2015-06-29 6:40 ` Minchan Kim
@ 2015-06-29 9:06 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 9:06 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
On (06/29/15 15:40), Minchan Kim wrote:
> On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> > Always account per-class `zs_size_stat' stats. This data will
> > help us make better decisions during compaction. We are especially
> > interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> > class compaction will result in any memory gain.
> >
> > For instance, we know the number of allocated objects in the class,
> > the number of objects being used (so we also know how many objects
> > are not used) and the number of objects per-page. So we can ensure
> > if we have enough unused objects to form at least one ZS_EMPTY
> > zspage during compaction.
> >
> > We calculate this value on per-class basis so we can calculate a
> > total number of zspages that can be released. Which is exactly what
> > a shrinker wants to know.
> >
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Acked-by: Minchan Kim <minchan@kernel.org>
>
thanks.
-ss
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats
@ 2015-06-29 9:06 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 9:06 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
On (06/29/15 15:40), Minchan Kim wrote:
> On Thu, Jun 18, 2015 at 08:46:40PM +0900, Sergey Senozhatsky wrote:
> > Always account per-class `zs_size_stat' stats. This data will
> > help us make better decisions during compaction. We are especially
> > interested in OBJ_ALLOCATED and OBJ_USED, which can tell us if
> > class compaction will result in any memory gain.
> >
> > For instance, we know the number of allocated objects in the class,
> > the number of objects being used (so we also know how many objects
> > are not used) and the number of objects per-page. So we can ensure
> > if we have enough unused objects to form at least one ZS_EMPTY
> > zspage during compaction.
> >
> > We calculate this value on per-class basis so we can calculate a
> > total number of zspages that can be released. Which is exactly what
> > a shrinker wants to know.
> >
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Acked-by: Minchan Kim <minchan@kernel.org>
>
thanks.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
2015-06-29 8:57 ` Sergey Senozhatsky
@ 2015-06-29 13:39 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 13:39 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel
Hi Sergey,
Sorry for too late reply.
On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> Hello,
>
> thanks for review.
>
> On (06/29/15 16:07), Minchan Kim wrote:
> [..]
> > > if (!migrate_zspage(pool, class, &cc))
> > > - break;
> > > + goto out;
> >
> > It should retry with another target_page instead of going out.
>
> yep.
>
> [..]
> > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > + struct shrink_control *sc)
> > > +{
> [..]
> >
> > Returns migrated object count.
> >
> [..]
> > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > + struct shrink_control *sc)
> > > +{
> [..]
> >
> > But it returns wasted_obj / max_obj_per_zspage?
> >
>
> Good catch.
> So, __zs_compact() and zs_shrinker_count() are ok. Returning
> "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> sense there. The only place is zs_shrinker_scan()->zs_compact().
I want to make zs_can_compact return freeable page unit.
ie,
return obj_wasted * class->pages_per_zspage;
and let's make __zs_compact returns the number of freed pages.
IOW, I like your (c).
>
> Hm, I can think of:
>
> (a) We can change zs_compact() to return the total number of
> freed zspages. That will not really change a user visible
> interface. We export (fwiw) the number of compacted objects
> in mm_stat. Basically, this is internal zsmalloc() counter and
> no user space program can ever do anything with that data. From
> that prospective we will just replace one senseless number with
> another (equally meaningless) one.
>
>
> (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
>
> 1764 int i;
> 1765 unsigned long nr_migrated = 0;
> 1766 struct size_class *class;
> 1767
> 1768 for (i = zs_size_classes - 1; i >= 0; i--) {
> 1769 class = pool->size_class[i];
> 1770 if (!class)
> 1771 continue;
> 1772 if (class->index != i)
> 1773 continue;
> 1774 nr_migrated += __zs_compact(pool, class);
> 1775 }
> 1776
> 1777 return nr_migrated;
>
> But on every iteration increment nr_migrated with
> "nr_migrated += just_migrated / max_obj_per_zspage"
>
> (which will be unnecessary if zs_compact() will return the number of freed
> zspages).
>
> So, (b) is mostly fine, except that we already have several pool->size_class
> loops, with same `if (!class)' and `if (class->index...)' checks; and it
> asks for some sort of refactoring or... a tricky for_each_class() define.
>
>
> In both cases, however, we don't tell anything valuable to user space.
> Thus,
>
> (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> And change compaction to operate in terms of pages (PAGE_SIZE). At
> least mm_stat::compacted will turn into something useful for user
> space.
Yes.
Thanks.
>
> -ss
--
Kind regards,
Minchan Kim
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
@ 2015-06-29 13:39 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-29 13:39 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel
Hi Sergey,
Sorry for too late reply.
On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> Hello,
>
> thanks for review.
>
> On (06/29/15 16:07), Minchan Kim wrote:
> [..]
> > > if (!migrate_zspage(pool, class, &cc))
> > > - break;
> > > + goto out;
> >
> > It should retry with another target_page instead of going out.
>
> yep.
>
> [..]
> > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > + struct shrink_control *sc)
> > > +{
> [..]
> >
> > Returns migrated object count.
> >
> [..]
> > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > + struct shrink_control *sc)
> > > +{
> [..]
> >
> > But it returns wasted_obj / max_obj_per_zspage?
> >
>
> Good catch.
> So, __zs_compact() and zs_shrinker_count() are ok. Returning
> "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> sense there. The only place is zs_shrinker_scan()->zs_compact().
I want to make zs_can_compact return freeable page unit.
ie,
return obj_wasted * class->pages_per_zspage;
and let's make __zs_compact returns the number of freed pages.
IOW, I like your (c).
>
> Hm, I can think of:
>
> (a) We can change zs_compact() to return the total number of
> freed zspages. That will not really change a user visible
> interface. We export (fwiw) the number of compacted objects
> in mm_stat. Basically, this is internal zsmalloc() counter and
> no user space program can ever do anything with that data. From
> that prospective we will just replace one senseless number with
> another (equally meaningless) one.
>
>
> (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
>
> 1764 int i;
> 1765 unsigned long nr_migrated = 0;
> 1766 struct size_class *class;
> 1767
> 1768 for (i = zs_size_classes - 1; i >= 0; i--) {
> 1769 class = pool->size_class[i];
> 1770 if (!class)
> 1771 continue;
> 1772 if (class->index != i)
> 1773 continue;
> 1774 nr_migrated += __zs_compact(pool, class);
> 1775 }
> 1776
> 1777 return nr_migrated;
>
> But on every iteration increment nr_migrated with
> "nr_migrated += just_migrated / max_obj_per_zspage"
>
> (which will be unnecessary if zs_compact() will return the number of freed
> zspages).
>
> So, (b) is mostly fine, except that we already have several pool->size_class
> loops, with same `if (!class)' and `if (class->index...)' checks; and it
> asks for some sort of refactoring or... a tricky for_each_class() define.
>
>
> In both cases, however, we don't tell anything valuable to user space.
> Thus,
>
> (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> And change compaction to operate in terms of pages (PAGE_SIZE). At
> least mm_stat::compacted will turn into something useful for user
> space.
Yes.
Thanks.
>
> -ss
--
Kind regards,
Minchan Kim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
2015-06-29 13:39 ` Minchan Kim
@ 2015-06-29 23:36 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 23:36 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Sergey Senozhatsky, Andrew Morton, linux-mm,
linux-kernel
On (06/29/15 22:39), Minchan Kim wrote:
> Hi Sergey,
>
> Sorry for too late reply.
>
Hello Minchan,
Sure, no problem.
Will send out a new patchset later today. Thanks.
-ss
> On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> > Hello,
> >
> > thanks for review.
> >
> > On (06/29/15 16:07), Minchan Kim wrote:
> > [..]
> > > > if (!migrate_zspage(pool, class, &cc))
> > > > - break;
> > > > + goto out;
> > >
> > > It should retry with another target_page instead of going out.
> >
> > yep.
> >
> > [..]
> > > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > > + struct shrink_control *sc)
> > > > +{
> > [..]
> > >
> > > Returns migrated object count.
> > >
> > [..]
> > > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > > + struct shrink_control *sc)
> > > > +{
> > [..]
> > >
> > > But it returns wasted_obj / max_obj_per_zspage?
> > >
> >
> > Good catch.
> > So, __zs_compact() and zs_shrinker_count() are ok. Returning
> > "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> > sense there. The only place is zs_shrinker_scan()->zs_compact().
>
> I want to make zs_can_compact return freeable page unit.
>
> ie,
>
> return obj_wasted * class->pages_per_zspage;
>
> and let's make __zs_compact returns the number of freed pages.
>
> IOW, I like your (c).
>
> >
> > Hm, I can think of:
> >
> > (a) We can change zs_compact() to return the total number of
> > freed zspages. That will not really change a user visible
> > interface. We export (fwiw) the number of compacted objects
> > in mm_stat. Basically, this is internal zsmalloc() counter and
> > no user space program can ever do anything with that data. From
> > that prospective we will just replace one senseless number with
> > another (equally meaningless) one.
> >
> >
> > (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> >
> > 1764 int i;
> > 1765 unsigned long nr_migrated = 0;
> > 1766 struct size_class *class;
> > 1767
> > 1768 for (i = zs_size_classes - 1; i >= 0; i--) {
> > 1769 class = pool->size_class[i];
> > 1770 if (!class)
> > 1771 continue;
> > 1772 if (class->index != i)
> > 1773 continue;
> > 1774 nr_migrated += __zs_compact(pool, class);
> > 1775 }
> > 1776
> > 1777 return nr_migrated;
> >
> > But on every iteration increment nr_migrated with
> > "nr_migrated += just_migrated / max_obj_per_zspage"
> >
> > (which will be unnecessary if zs_compact() will return the number of freed
> > zspages).
> >
> > So, (b) is mostly fine, except that we already have several pool->size_class
> > loops, with same `if (!class)' and `if (class->index...)' checks; and it
> > asks for some sort of refactoring or... a tricky for_each_class() define.
> >
> >
> > In both cases, however, we don't tell anything valuable to user space.
> > Thus,
> >
> > (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> > And change compaction to operate in terms of pages (PAGE_SIZE). At
> > least mm_stat::compacted will turn into something useful for user
> > space.
>
> Yes.
>
> Thanks.
>
> >
> > -ss
>
> --
> Kind regards,
> Minchan Kim
>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction
@ 2015-06-29 23:36 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 23:36 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Sergey Senozhatsky, Andrew Morton, linux-mm,
linux-kernel
On (06/29/15 22:39), Minchan Kim wrote:
> Hi Sergey,
>
> Sorry for too late reply.
>
Hello Minchan,
Sure, no problem.
Will send out a new patchset later today. Thanks.
-ss
> On Mon, Jun 29, 2015 at 05:57:44PM +0900, Sergey Senozhatsky wrote:
> > Hello,
> >
> > thanks for review.
> >
> > On (06/29/15 16:07), Minchan Kim wrote:
> > [..]
> > > > if (!migrate_zspage(pool, class, &cc))
> > > > - break;
> > > > + goto out;
> > >
> > > It should retry with another target_page instead of going out.
> >
> > yep.
> >
> > [..]
> > > > +static unsigned long zs_shrinker_scan(struct shrinker *shrinker,
> > > > + struct shrink_control *sc)
> > > > +{
> > [..]
> > >
> > > Returns migrated object count.
> > >
> > [..]
> > > > +static unsigned long zs_shrinker_count(struct shrinker *shrinker,
> > > > + struct shrink_control *sc)
> > > > +{
> > [..]
> > >
> > > But it returns wasted_obj / max_obj_per_zspage?
> > >
> >
> > Good catch.
> > So, __zs_compact() and zs_shrinker_count() are ok. Returning
> > "wasted_obj / max_obj_per_zspage" from zs_can_compact() makes
> > sense there. The only place is zs_shrinker_scan()->zs_compact().
>
> I want to make zs_can_compact return freeable page unit.
>
> ie,
>
> return obj_wasted * class->pages_per_zspage;
>
> and let's make __zs_compact returns the number of freed pages.
>
> IOW, I like your (c).
>
> >
> > Hm, I can think of:
> >
> > (a) We can change zs_compact() to return the total number of
> > freed zspages. That will not really change a user visible
> > interface. We export (fwiw) the number of compacted objects
> > in mm_stat. Basically, this is internal zsmalloc() counter and
> > no user space program can ever do anything with that data. From
> > that prospective we will just replace one senseless number with
> > another (equally meaningless) one.
> >
> >
> > (b) replace zs_compact() call in zs_shrinker_scan() with a class loop
> >
> > 1764 int i;
> > 1765 unsigned long nr_migrated = 0;
> > 1766 struct size_class *class;
> > 1767
> > 1768 for (i = zs_size_classes - 1; i >= 0; i--) {
> > 1769 class = pool->size_class[i];
> > 1770 if (!class)
> > 1771 continue;
> > 1772 if (class->index != i)
> > 1773 continue;
> > 1774 nr_migrated += __zs_compact(pool, class);
> > 1775 }
> > 1776
> > 1777 return nr_migrated;
> >
> > But on every iteration increment nr_migrated with
> > "nr_migrated += just_migrated / max_obj_per_zspage"
> >
> > (which will be unnecessary if zs_compact() will return the number of freed
> > zspages).
> >
> > So, (b) is mostly fine, except that we already have several pool->size_class
> > loops, with same `if (!class)' and `if (class->index...)' checks; and it
> > asks for some sort of refactoring or... a tricky for_each_class() define.
> >
> >
> > In both cases, however, we don't tell anything valuable to user space.
> > Thus,
> >
> > (c) Return from zs_compact() the number of pages (PAGE_SIZE) freed.
> > And change compaction to operate in terms of pages (PAGE_SIZE). At
> > least mm_stat::compacted will turn into something useful for user
> > space.
>
> Yes.
>
> Thanks.
>
> >
> > -ss
>
> --
> Kind regards,
> Minchan Kim
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-29 6:52 ` Minchan Kim
@ 2015-06-29 23:41 ` Sergey Senozhatsky
-1 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 23:41 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
Hello,
Sorry for long reply.
On (06/29/15 15:52), Minchan Kim wrote:
[..]
> > head = &class->fullness_list[fullness];
> > - if (*head)
> > - list_add_tail(&page->lru, &(*head)->lru);
> > + if (*head) {
> > + /*
> > + * We want to see more ZS_FULL pages and less almost
> > + * empty/full. Put pages with higher ->inuse first.
> > + */
> > + if (page->inuse < (*head)->inuse)
> > + list_add_tail(&page->lru, &(*head)->lru);
> > + else
> > + list_add(&page->lru, &(*head)->lru);
> > + }
>
> >
> > *head = page;
>
> Why do you want to always put @page in the head?
> How about this?
Yeah, right. Looks OK to me. How do we want to handle it? Do you want
to submit it with Suggested-by: ss (I'm fine) or you want me to submit
it later today with Suggested-by: Minchan Kim?
-ss
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index e8cb31c..1c5fde9 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> return;
>
> + zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> + CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +
> head = &class->fullness_list[fullness];
> - if (*head) {
> - /*
> - * We want to see more ZS_FULL pages and less almost
> - * empty/full. Put pages with higher ->inuse first.
> - */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> - list_add(&page->lru, &(*head)->lru);
> + if (!*head) {
> + *head = page;
> + return;
> }
>
> - *head = page;
> - zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> - CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> + /*
> + * We want to see more ZS_FULL pages and less almost
> + * empty/full. Put pages with higher ->inuse first.
> + */
> + list_add_tail(&page->lru, &(*head)->lru);
> + if (page->inuse >= (*head)->inuse)
> + *head = page;
> }
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-29 23:41 ` Sergey Senozhatsky
0 siblings, 0 replies; 48+ messages in thread
From: Sergey Senozhatsky @ 2015-06-29 23:41 UTC (permalink / raw)
To: Minchan Kim
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel,
Sergey Senozhatsky
Hello,
Sorry for long reply.
On (06/29/15 15:52), Minchan Kim wrote:
[..]
> > head = &class->fullness_list[fullness];
> > - if (*head)
> > - list_add_tail(&page->lru, &(*head)->lru);
> > + if (*head) {
> > + /*
> > + * We want to see more ZS_FULL pages and less almost
> > + * empty/full. Put pages with higher ->inuse first.
> > + */
> > + if (page->inuse < (*head)->inuse)
> > + list_add_tail(&page->lru, &(*head)->lru);
> > + else
> > + list_add(&page->lru, &(*head)->lru);
> > + }
>
> >
> > *head = page;
>
> Why do you want to always put @page in the head?
> How about this?
Yeah, right. Looks OK to me. How do we want to handle it? Do you want
to submit it with Suggested-by: ss (I'm fine) or you want me to submit
it later today with Suggested-by: Minchan Kim?
-ss
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index e8cb31c..1c5fde9 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> return;
>
> + zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> + CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> +
> head = &class->fullness_list[fullness];
> - if (*head) {
> - /*
> - * We want to see more ZS_FULL pages and less almost
> - * empty/full. Put pages with higher ->inuse first.
> - */
> - if (page->inuse < (*head)->inuse)
> - list_add_tail(&page->lru, &(*head)->lru);
> - else
> - list_add(&page->lru, &(*head)->lru);
> + if (!*head) {
> + *head = page;
> + return;
> }
>
> - *head = page;
> - zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> - CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> + /*
> + * We want to see more ZS_FULL pages and less almost
> + * empty/full. Put pages with higher ->inuse first.
> + */
> + list_add_tail(&page->lru, &(*head)->lru);
> + if (page->inuse >= (*head)->inuse)
> + *head = page;
> }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
2015-06-29 23:41 ` Sergey Senozhatsky
@ 2015-06-30 0:42 ` Minchan Kim
-1 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-30 0:42 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel
On Tue, Jun 30, 2015 at 08:41:24AM +0900, Sergey Senozhatsky wrote:
> Hello,
>
> Sorry for long reply.
>
> On (06/29/15 15:52), Minchan Kim wrote:
> [..]
> > > head = &class->fullness_list[fullness];
> > > - if (*head)
> > > - list_add_tail(&page->lru, &(*head)->lru);
> > > + if (*head) {
> > > + /*
> > > + * We want to see more ZS_FULL pages and less almost
> > > + * empty/full. Put pages with higher ->inuse first.
> > > + */
> > > + if (page->inuse < (*head)->inuse)
> > > + list_add_tail(&page->lru, &(*head)->lru);
> > > + else
> > > + list_add(&page->lru, &(*head)->lru);
> > > + }
> >
> > >
> > > *head = page;
> >
> > Why do you want to always put @page in the head?
> > How about this?
>
>
> Yeah, right. Looks OK to me. How do we want to handle it? Do you want
> to submit it with Suggested-by: ss (I'm fine) or you want me to submit
> it later today with Suggested-by: Minchan Kim?
Hi Sergey,
Thanks for considering the credit.
However, it was just review so no need my sign. :)
I really appreciate your good works.
Thanks.
>
> -ss
>
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index e8cb31c..1c5fde9 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> > if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> > return;
> >
> > + zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > + CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +
> > head = &class->fullness_list[fullness];
> > - if (*head) {
> > - /*
> > - * We want to see more ZS_FULL pages and less almost
> > - * empty/full. Put pages with higher ->inuse first.
> > - */
> > - if (page->inuse < (*head)->inuse)
> > - list_add_tail(&page->lru, &(*head)->lru);
> > - else
> > - list_add(&page->lru, &(*head)->lru);
> > + if (!*head) {
> > + *head = page;
> > + return;
> > }
> >
> > - *head = page;
> > - zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > - CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > + /*
> > + * We want to see more ZS_FULL pages and less almost
> > + * empty/full. Put pages with higher ->inuse first.
> > + */
> > + list_add_tail(&page->lru, &(*head)->lru);
> > + if (page->inuse >= (*head)->inuse)
> > + *head = page;
> > }
--
Kind regards,
Minchan Kim
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list
@ 2015-06-30 0:42 ` Minchan Kim
0 siblings, 0 replies; 48+ messages in thread
From: Minchan Kim @ 2015-06-30 0:42 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: Sergey Senozhatsky, Andrew Morton, linux-mm, linux-kernel
On Tue, Jun 30, 2015 at 08:41:24AM +0900, Sergey Senozhatsky wrote:
> Hello,
>
> Sorry for long reply.
>
> On (06/29/15 15:52), Minchan Kim wrote:
> [..]
> > > head = &class->fullness_list[fullness];
> > > - if (*head)
> > > - list_add_tail(&page->lru, &(*head)->lru);
> > > + if (*head) {
> > > + /*
> > > + * We want to see more ZS_FULL pages and less almost
> > > + * empty/full. Put pages with higher ->inuse first.
> > > + */
> > > + if (page->inuse < (*head)->inuse)
> > > + list_add_tail(&page->lru, &(*head)->lru);
> > > + else
> > > + list_add(&page->lru, &(*head)->lru);
> > > + }
> >
> > >
> > > *head = page;
> >
> > Why do you want to always put @page in the head?
> > How about this?
>
>
> Yeah, right. Looks OK to me. How do we want to handle it? Do you want
> to submit it with Suggested-by: ss (I'm fine) or you want me to submit
> it later today with Suggested-by: Minchan Kim?
Hi Sergey,
Thanks for considering the credit.
However, it was just review so no need my sign. :)
I really appreciate your good works.
Thanks.
>
> -ss
>
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index e8cb31c..1c5fde9 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -658,21 +658,25 @@ static void insert_zspage(struct page *page, struct size_class *class,
> > if (fullness >= _ZS_NR_FULLNESS_GROUPS)
> > return;
> >
> > + zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > + CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > +
> > head = &class->fullness_list[fullness];
> > - if (*head) {
> > - /*
> > - * We want to see more ZS_FULL pages and less almost
> > - * empty/full. Put pages with higher ->inuse first.
> > - */
> > - if (page->inuse < (*head)->inuse)
> > - list_add_tail(&page->lru, &(*head)->lru);
> > - else
> > - list_add(&page->lru, &(*head)->lru);
> > + if (!*head) {
> > + *head = page;
> > + return;
> > }
> >
> > - *head = page;
> > - zs_stat_inc(class, fullness == ZS_ALMOST_EMPTY ?
> > - CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
> > + /*
> > + * We want to see more ZS_FULL pages and less almost
> > + * empty/full. Put pages with higher ->inuse first.
> > + */
> > + list_add_tail(&page->lru, &(*head)->lru);
> > + if (page->inuse >= (*head)->inuse)
> > + *head = page;
> > }
--
Kind regards,
Minchan Kim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 48+ messages in thread
end of thread, other threads:[~2015-06-30 0:43 UTC | newest]
Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-18 11:46 [RFC][PATCH v3 0/7] introduce automatic pool compaction Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 1/7] zsmalloc: drop unused variable `nr_to_migrate' Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 2/7] zsmalloc: partial page ordering within a fullness_list Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-18 12:13 ` Sergey Senozhatsky
2015-06-18 12:13 ` Sergey Senozhatsky
2015-06-18 12:45 ` Sergey Senozhatsky
2015-06-18 12:45 ` Sergey Senozhatsky
2015-06-18 12:48 ` Sergey Senozhatsky
2015-06-18 12:48 ` Sergey Senozhatsky
2015-06-18 14:43 ` Sergey Senozhatsky
2015-06-18 14:43 ` Sergey Senozhatsky
2015-06-29 6:52 ` Minchan Kim
2015-06-29 6:52 ` Minchan Kim
2015-06-29 23:41 ` Sergey Senozhatsky
2015-06-29 23:41 ` Sergey Senozhatsky
2015-06-30 0:42 ` Minchan Kim
2015-06-30 0:42 ` Minchan Kim
2015-06-18 11:46 ` [RFC][PATCHv3 3/7] zsmalloc: always keep per-class stats Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-29 6:40 ` Minchan Kim
2015-06-29 6:40 ` Minchan Kim
2015-06-29 9:06 ` Sergey Senozhatsky
2015-06-29 9:06 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 4/7] zsmalloc: introduce zs_can_compact() function Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-29 6:45 ` Minchan Kim
2015-06-29 6:45 ` Minchan Kim
2015-06-29 8:58 ` Sergey Senozhatsky
2015-06-29 8:58 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 5/7] zsmalloc: cosmetic compaction code adjustments Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 6/7] zsmalloc/zram: move `num_migrated' to zs_pool Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-18 11:46 ` [RFC][PATCHv3 7/7] zsmalloc: register a shrinker to trigger auto-compaction Sergey Senozhatsky
2015-06-18 11:46 ` Sergey Senozhatsky
2015-06-29 7:07 ` Minchan Kim
2015-06-29 7:07 ` Minchan Kim
2015-06-29 8:57 ` Sergey Senozhatsky
2015-06-29 8:57 ` Sergey Senozhatsky
2015-06-29 13:39 ` Minchan Kim
2015-06-29 13:39 ` Minchan Kim
2015-06-29 23:36 ` Sergey Senozhatsky
2015-06-29 23:36 ` Sergey Senozhatsky
2015-06-18 12:17 ` [RFC][PATCH v3 0/7] introduce automatic pool compaction Sergey Senozhatsky
2015-06-18 12:17 ` Sergey Senozhatsky
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.