linux-bcache.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Coly Li <colyli@suse.de>
To: linux-bcache@vger.kernel.org
Cc: linux-block@vger.kernel.org, Coly Li <colyli@suse.de>
Subject: [PATCH v2 17/17] bcache: avoid extra memory consumption in struct bbio for large bucket size
Date: Wed, 15 Jul 2020 13:46:12 +0800	[thread overview]
Message-ID: <20200715054612.6349-18-colyli@suse.de> (raw)
In-Reply-To: <20200715054612.6349-1-colyli@suse.de>

Bcache uses struct bbio to do I/Os for meta data pages like uuids,
disk_buckets, prio_buckets, and btree nodes.

Example writing a btree node onto cache device, the process is,
- Allocate a struct bbio from mempool c->bio_meta.
- Inside struct bbio embedded a struct bio, initialize bi_inline_vecs
  for this embedded bio.
- Call bch_bio_map() to map each meta data page to each bv from the
  inlined  bi_io_vec table.
- Call bch_submit_bbio() to submit the bio into underlying block layer.
- When the I/O completed, only release the struct bbio, don't touch the
  reference counter of the meta data pages.

The struct bbio is defined as,
738 struct bbio {
739     unsigned int            submit_time_us;
	[snipped]
748     struct bio              bio;
749 };

Because struct bio is embedded at the end of struct bbio, therefore the
actual size of struct bbio is sizeof(struct bio) + size of the embedded
bio->bi_inline_vecs.

Now all the meta data bucket size are limited to meta_bucket_pages(), if
the bucket size is large than meta_bucket_pages()*PAGE_SECTORS, rested
space in the bucket is unused. Therefore the most used space in meta
bucket is (1<<MAX_ORDER) pages, or (1<<CONFIG_FORCE_MAX_ZONEORDER) if it
is configured.

Therefore for large bucket size, it is unnecessary to calculate the
allocation size of mempool c->bio_meta as,
	mempool_init_kmalloc_pool(&c->bio_meta, 2,
			sizeof(struct bbio) +
			sizeof(struct bio_vec) * bucket_pages(c))
It is too large, neither the Linux buddy allocator cannot allocate so
much continuous pages, nor the extra allocated pages are wasted.

This patch replace bucket_pages() to meta_bucket_pages() in two places,
- In bch_cache_set_alloc(), when initialize mempool c->bio_meta, uses
  sizeof(struct bbio) + sizeof(struct bio_vec) * bucket_pages(c) to set
  the allocating object size.
- In bch_bbio_alloc(), when calling bio_init() to set inline bvec talbe
  bi_inline_bvecs, uses meta_bucket_pages() to indicate number of the
  inline bio vencs number.

Now the maximum size of embedded bio inside struct bbio exactly matches
the limit of meta_bucket_pages(), no extra page wasted.

Signed-off-by: Coly Li <colyli@suse.de>
---
 drivers/md/bcache/io.c    | 2 +-
 drivers/md/bcache/super.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
index b25ee33b0d0b..a14a445618b4 100644
--- a/drivers/md/bcache/io.c
+++ b/drivers/md/bcache/io.c
@@ -26,7 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
 	struct bbio *b = mempool_alloc(&c->bio_meta, GFP_NOIO);
 	struct bio *bio = &b->bio;
 
-	bio_init(bio, bio->bi_inline_vecs, bucket_pages(c));
+	bio_init(bio, bio->bi_inline_vecs, meta_bucket_pages(&c->sb));
 
 	return bio;
 }
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 3d05f7f46b01..aba535ecc335 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1920,7 +1920,7 @@ struct cache_set *bch_cache_set_alloc(struct cache_sb *sb)
 
 	if (mempool_init_kmalloc_pool(&c->bio_meta, 2,
 			sizeof(struct bbio) +
-			sizeof(struct bio_vec) * bucket_pages(c)))
+			sizeof(struct bio_vec) * meta_bucket_pages(&c->sb)))
 		goto err;
 
 	if (mempool_init_kmalloc_pool(&c->fill_iter, 1, iter_size))
-- 
2.26.2


      parent reply	other threads:[~2020-07-15  5:46 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-15  5:45 [PATCH v2 00/17] bcache: extend bucket size to 32bit width Coly Li
2020-07-15  5:45 ` [PATCH v2 01/17] bcache: add comments to mark member offset of struct cache_sb_disk Coly Li
2020-07-15  6:02   ` Hannes Reinecke
2020-07-15  9:03     ` Coly Li
2020-07-15  9:08       ` Johannes Thumshirn
2020-07-15 10:35         ` Coly Li
2020-07-15  5:45 ` [PATCH v2 02/17] bcache: add read_super_basic() to read major part of super block Coly Li
2020-07-15  6:03   ` Hannes Reinecke
2020-07-15  5:45 ` [PATCH v2 03/17] bcache: add more accurate error information in read_super_basic() Coly Li
2020-07-15  6:04   ` Hannes Reinecke
2020-07-15  5:45 ` [PATCH v2 04/17] bcache: disassemble the big if() checks in bch_cache_set_alloc() Coly Li
2020-07-15  6:08   ` Hannes Reinecke
2020-07-15  5:46 ` [PATCH v2 05/17] bcache: fix super block seq numbers comparision in register_cache_set() Coly Li
2020-07-15  6:10   ` Hannes Reinecke
2020-07-15  5:46 ` [PATCH v2 06/17] bcache: increase super block version for cache device and backing device Coly Li
2020-07-15  6:11   ` Hannes Reinecke
2020-07-15  5:46 ` [PATCH v2 07/17] bcache: move bucket related code into read_super_basic() Coly Li
2020-07-15  6:44   ` Hannes Reinecke
2020-07-15  5:46 ` [PATCH v2 08/17] bcache: struct cache_sb is only for in-memory super block now Coly Li
2020-07-15  6:45   ` Hannes Reinecke
2020-07-15  5:46 ` [PATCH v2 09/17] bcache: introduce meta_bucket_pages() related helper routines Coly Li
2020-07-15  5:46 ` [PATCH v2 10/17] bcache: handle c->uuids properly for bucket size > 8MB Coly Li
2020-07-15  5:46 ` [PATCH v2 11/17] bcache: handle cache prio_buckets and disk_buckets " Coly Li
2020-07-15  5:46 ` [PATCH v2 12/17] bcache: handle cache set verify_ondisk " Coly Li
2020-07-15  5:46 ` [PATCH v2 13/17] bcache: handle btree node memory allocation " Coly Li
2020-07-15  5:46 ` [PATCH v2 14/17] bcache: add bucket_size_hi into struct cache_sb_disk for large bucket Coly Li
2020-07-15  5:46 ` [PATCH v2 15/17] bcache: add sysfs file to display feature sets information of cache set Coly Li
2020-07-15  5:46 ` [PATCH v2 16/17] bcache: avoid extra memory allocation from mempool c->fill_iter Coly Li
2020-07-15  5:46 ` Coly Li [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200715054612.6349-18-colyli@suse.de \
    --to=colyli@suse.de \
    --cc=linux-bcache@vger.kernel.org \
    --cc=linux-block@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).