[PATCH 0/2] zram: support for specific numa node for zram

[PATCH 0/2] zram: support for specific numa node for zram

[Ze Zuo]

This patch series adds a parameter "numa_id" to zram to support the use
of memory in a specific node, and attempts to obtain the benefits of using
kvzalloc_node to obtain huge page table mappings.

Ze Zuo (2):
  zram: add a parameter "node_id" for zram
  zram: take the benefit of kvzalloc_node

 drivers/block/zram/zram_drv.c | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)

-- 
2.25.1

[PATCH 1/2] zram: add a parameter "node_id" for zram

[Ze Zuo]

Add a parameter "node_id" to zram to support storing pages on specific
node_id node memory.

Now, zram memory allocation is random, however in some cases, specifying
specific nodes for memory allocation for zram may have good effects. In
addition, when memory tier is supported, demotion can be achieved not
only through page migration,  it is also possible to apply for memory by
specifying zram on low-speed device nodes, such as CXL memory devices,
and compressing pages to these devices through memory reclamation to
achieve similar effects to migration.

Signed-off-by: Ze Zuo <zuoze1@huawei.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
 drivers/block/zram/zram_drv.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 06673c6ca255..692993e48e93 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -45,6 +45,8 @@ static const char *default_compressor = CONFIG_ZRAM_DEF_COMP;
 
 /* Module params (documentation at end) */
 static unsigned int num_devices = 1;
+
+static int node_id = NUMA_NO_NODE;
 /*
  * Pages that compress to sizes equals or greater than this are stored
  * uncompressed in memory.
@@ -53,6 +55,28 @@ static size_t huge_class_size;
 
 static const struct block_device_operations zram_devops;
 
+static int zram_node_id_store(const char *val,
+		const struct kernel_param *kp)
+{
+	int ret, nid;
+
+	ret = kstrtoint(val, 10, &nid);
+	if (ret)
+		return ret;
+	if (nid != NUMA_NO_NODE && !node_online(nid))
+		return -EINVAL;
+	node_id = nid;
+	return 0;
+}
+
+static const struct kernel_param_ops node_id_param_ops = {
+	.set = zram_node_id_store,
+	.get = param_get_int,
+};
+
+module_param_cb(node_id, &node_id_param_ops, &node_id, 0600);
+MODULE_PARM_DESC(node_id, "The node of pre-created zram devices memory alloc");
+
 static void zram_free_page(struct zram *zram, size_t index);
 static int zram_read_page(struct zram *zram, struct page *page, u32 index,
 			  struct bio *parent);
@@ -1233,7 +1257,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize)
 	size_t num_pages;
 
 	num_pages = disksize >> PAGE_SHIFT;
-	zram->table = vzalloc(array_size(num_pages, sizeof(*zram->table)));
+	zram->table = vzalloc_node(array_size(num_pages, sizeof(*zram->table)), node_id);
 	if (!zram->table)
 		return false;
 
-- 
2.25.1

[PATCH 2/2] zram: take the benefit of kvzalloc_node

[Ze Zuo]

Using kvzalloc and zram can benefit from huge page table mapping, so lets
use kvzalloc_node/kvfree instead of vzalloc_node/vfree.

Signed-off-by: Ze Zuo <zuoze1@huawei.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
 drivers/block/zram/zram_drv.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 692993e48e93..97241f166324 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1249,7 +1249,7 @@ static void zram_meta_free(struct zram *zram, u64 disksize)
 		zram_free_page(zram, index);
 
 	zs_destroy_pool(zram->mem_pool);
-	vfree(zram->table);
+	kvfree(zram->table);
 }
 
 static bool zram_meta_alloc(struct zram *zram, u64 disksize)
@@ -1257,13 +1257,14 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize)
 	size_t num_pages;
 
 	num_pages = disksize >> PAGE_SHIFT;
-	zram->table = vzalloc_node(array_size(num_pages, sizeof(*zram->table)), node_id);
+	zram->table = kvzalloc_node(array_size(num_pages, sizeof(*zram->table)),
+				    GFP_KERNEL, node_id);
 	if (!zram->table)
 		return false;
 
 	zram->mem_pool = zs_create_pool(zram->disk->disk_name);
 	if (!zram->mem_pool) {
-		vfree(zram->table);
+		kvfree(zram->table);
 		return false;
 	}
 
-- 
2.25.1

Re: [PATCH 1/2] zram: add a parameter "node_id" for zram

[Andrew Morton]

On Fri, 1 Sep 2023 15:19:41 +0800 Ze Zuo <zuoze1@huawei.com> wrote:

> Add a parameter "node_id" to zram to support storing pages on specific
> node_id node memory.

Please document this in Documentation/admin-guide/blockdev/zram.rst

Should it be a group of nodes rather than a single one?  If this
feature is useful, people will want that.

> Now, zram memory allocation is random, however in some cases, specifying
> specific nodes for memory allocation for zram may have good effects. In
> addition, when memory tier is supported, demotion can be achieved not
> only through page migration,  it is also possible to apply for memory by
> specifying zram on low-speed device nodes, such as CXL memory devices,
> and compressing pages to these devices through memory reclamation to
> achieve similar effects to migration.

Are any performance testing results available from using this feature?

Re: [PATCH 1/2] zram: add a parameter "node_id" for zram

[Sergey Senozhatsky]

On (23/09/01 15:19), Ze Zuo wrote:
> Now, zram memory allocation is random, however in some cases, specifying
> specific nodes for memory allocation for zram may have good effects.

I'm sorry, but that needs benchmarks in order to be justified.

> In addition, when memory tier is supported, demotion can be achieved not
> only through page migration,  it is also possible to apply for memory by
> specifying zram on low-speed device nodes, such as CXL memory devices,
> and compressing pages to these devices through memory reclamation to
> achieve similar effects to migration.

zram->table has nothing to do with zsmalloc pool. zram->table is a
fixed size (it depends on block device size) array that maps block
index to zsmalloc handle. It's allocated once, when the device is
initialized. Compressed pages are not stored there, zsmalloc pool
is a separate thing.

[..]

> +	zram->table = vzalloc_node(array_size(num_pages, sizeof(*zram->table)), node_id);