From: Christoph Hellwig <hch@lst.de> To: Jens Axboe <axboe@kernel.dk> Cc: linux-bcache@vger.kernel.org, linux-xtensa@linux-xtensa.org, linux-m68k@vger.kernel.org, linux-nvdimm@lists.01.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-raid@vger.kernel.org, dm-devel@redhat.com, drbd-dev@tron.linbit.com, linuxppc-dev@lists.ozlabs.org Subject: [PATCH 18/20] block: refator submit_bio_noacct Date: Mon, 29 Jun 2020 21:39:45 +0200 [thread overview] Message-ID: <20200629193947.2705954-19-hch@lst.de> (raw) In-Reply-To: <20200629193947.2705954-1-hch@lst.de> Split out a __submit_bio_noacct helper for the actual de-recursion algorithm, and simplify the loop by using a continue when we can't enter the queue for a bio. Signed-off-by: Christoph Hellwig <hch@lst.de> --- block/blk-core.c | 131 +++++++++++++++++++++++++---------------------- 1 file changed, 71 insertions(+), 60 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 1caeb01e127768..b82f48c86e6f7a 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1085,6 +1085,74 @@ static blk_qc_t do_make_request(struct bio *bio) return ret; } +/* + * The loop in this function may be a bit non-obvious, and so deserves some + * explanation: + * + * - Before entering the loop, bio->bi_next is NULL (as all callers ensure + * that), so we have a list with a single bio. + * - We pretend that we have just taken it off a longer list, so we assign + * bio_list to a pointer to the bio_list_on_stack, thus initialising the + * bio_list of new bios to be added. ->submit_bio() may indeed add some more + * bios through a recursive call to submit_bio_noacct. If it did, we find a + * non-NULL value in bio_list and re-enter the loop from the top. + * - In this case we really did just take the bio of the top of the list (no + * pretending) and so remove it from bio_list, and call into ->submit_bio() + * again. + * + * bio_list_on_stack[0] contains bios submitted by the current ->submit_bio. + * bio_list_on_stack[1] contains bios that were submitted before the current + * ->submit_bio_bio, but that haven't been processed yet. + */ +static blk_qc_t __submit_bio_noacct(struct bio *bio) +{ + struct bio_list bio_list_on_stack[2]; + blk_qc_t ret = BLK_QC_T_NONE; + + BUG_ON(bio->bi_next); + + bio_list_init(&bio_list_on_stack[0]); + current->bio_list = bio_list_on_stack; + + do { + struct request_queue *q = bio->bi_disk->queue; + struct bio_list lower, same; + + if (unlikely(bio_queue_enter(bio) != 0)) + continue; + + /* + * Create a fresh bio_list for all subordinate requests. + */ + bio_list_on_stack[1] = bio_list_on_stack[0]; + bio_list_init(&bio_list_on_stack[0]); + + ret = do_make_request(bio); + + /* + * Sort new bios into those for a lower level and those for the + * same level. + */ + bio_list_init(&lower); + bio_list_init(&same); + while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL) + if (q == bio->bi_disk->queue) + bio_list_add(&same, bio); + else + bio_list_add(&lower, bio); + + /* + * Now assemble so we handle the lowest level first. + */ + bio_list_merge(&bio_list_on_stack[0], &lower); + bio_list_merge(&bio_list_on_stack[0], &same); + bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]); + } while ((bio = bio_list_pop(&bio_list_on_stack[0]))); + + current->bio_list = NULL; + return ret; +} + /** * submit_bio_noacct - re-submit a bio to the block device layer for I/O * @bio: The bio describing the location in memory and on the device. @@ -1096,17 +1164,8 @@ static blk_qc_t do_make_request(struct bio *bio) */ blk_qc_t submit_bio_noacct(struct bio *bio) { - /* - * bio_list_on_stack[0] contains bios submitted by the current - * ->submit_bio. - * bio_list_on_stack[1] contains bios that were submitted before the - * current ->submit_bio_bio, but that haven't been processed yet. - */ - struct bio_list bio_list_on_stack[2]; - blk_qc_t ret = BLK_QC_T_NONE; - if (!submit_bio_checks(bio)) - goto out; + return BLK_QC_T_NONE; /* * We only want one ->submit_bio to be active at a time, else @@ -1120,58 +1179,10 @@ blk_qc_t submit_bio_noacct(struct bio *bio) */ if (current->bio_list) { bio_list_add(¤t->bio_list[0], bio); - goto out; + return BLK_QC_T_NONE; } - /* following loop may be a bit non-obvious, and so deserves some - * explanation. - * Before entering the loop, bio->bi_next is NULL (as all callers - * ensure that) so we have a list with a single bio. - * We pretend that we have just taken it off a longer list, so - * we assign bio_list to a pointer to the bio_list_on_stack, - * thus initialising the bio_list of new bios to be - * added. ->submit_bio() may indeed add some more bios - * through a recursive call to submit_bio_noacct. If it - * did, we find a non-NULL value in bio_list and re-enter the loop - * from the top. In this case we really did just take the bio - * of the top of the list (no pretending) and so remove it from - * bio_list, and call into ->submit_bio() again. - */ - BUG_ON(bio->bi_next); - bio_list_init(&bio_list_on_stack[0]); - current->bio_list = bio_list_on_stack; - do { - struct request_queue *q = bio->bi_disk->queue; - - if (likely(bio_queue_enter(bio) == 0)) { - struct bio_list lower, same; - - /* Create a fresh bio_list for all subordinate requests */ - bio_list_on_stack[1] = bio_list_on_stack[0]; - bio_list_init(&bio_list_on_stack[0]); - ret = do_make_request(bio); - - /* sort new bios into those for a lower level - * and those for the same level - */ - bio_list_init(&lower); - bio_list_init(&same); - while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL) - if (q == bio->bi_disk->queue) - bio_list_add(&same, bio); - else - bio_list_add(&lower, bio); - /* now assemble so we handle the lowest level first */ - bio_list_merge(&bio_list_on_stack[0], &lower); - bio_list_merge(&bio_list_on_stack[0], &same); - bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]); - } - bio = bio_list_pop(&bio_list_on_stack[0]); - } while (bio); - current->bio_list = NULL; /* deactivate */ - -out: - return ret; + return __submit_bio_noacct(bio); } EXPORT_SYMBOL(submit_bio_noacct); -- 2.26.2
WARNING: multiple messages have this Message-ID (diff)
From: Christoph Hellwig <hch@lst.de> To: Jens Axboe <axboe@kernel.dk> Cc: dm-devel@redhat.com, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-xtensa@linux-xtensa.org, drbd-dev@lists.linbit.com, linuxppc-dev@lists.ozlabs.org, linux-bcache@vger.kernel.org, linux-raid@vger.kernel.org, linux-nvdimm@lists.01.org, linux-nvme@lists.infradead.org, linux-s390@vger.kernel.org Subject: [PATCH 18/20] block: refator submit_bio_noacct Date: Mon, 29 Jun 2020 21:39:45 +0200 [thread overview] Message-ID: <20200629193947.2705954-19-hch@lst.de> (raw) Message-ID: <20200629193945.NLazdjSsEa47dZEDmZPSqUCU8UBEC7a3Z6KjQJ6oLmU@z> (raw) In-Reply-To: <20200629193947.2705954-1-hch@lst.de> Split out a __submit_bio_noacct helper for the actual de-recursion algorithm, and simplify the loop by using a continue when we can't enter the queue for a bio. Signed-off-by: Christoph Hellwig <hch@lst.de> --- block/blk-core.c | 131 +++++++++++++++++++++++++---------------------- 1 file changed, 71 insertions(+), 60 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 1caeb01e127768..b82f48c86e6f7a 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1085,6 +1085,74 @@ static blk_qc_t do_make_request(struct bio *bio) return ret; } +/* + * The loop in this function may be a bit non-obvious, and so deserves some + * explanation: + * + * - Before entering the loop, bio->bi_next is NULL (as all callers ensure + * that), so we have a list with a single bio. + * - We pretend that we have just taken it off a longer list, so we assign + * bio_list to a pointer to the bio_list_on_stack, thus initialising the + * bio_list of new bios to be added. ->submit_bio() may indeed add some more + * bios through a recursive call to submit_bio_noacct. If it did, we find a + * non-NULL value in bio_list and re-enter the loop from the top. + * - In this case we really did just take the bio of the top of the list (no + * pretending) and so remove it from bio_list, and call into ->submit_bio() + * again. + * + * bio_list_on_stack[0] contains bios submitted by the current ->submit_bio. + * bio_list_on_stack[1] contains bios that were submitted before the current + * ->submit_bio_bio, but that haven't been processed yet. + */ +static blk_qc_t __submit_bio_noacct(struct bio *bio) +{ + struct bio_list bio_list_on_stack[2]; + blk_qc_t ret = BLK_QC_T_NONE; + + BUG_ON(bio->bi_next); + + bio_list_init(&bio_list_on_stack[0]); + current->bio_list = bio_list_on_stack; + + do { + struct request_queue *q = bio->bi_disk->queue; + struct bio_list lower, same; + + if (unlikely(bio_queue_enter(bio) != 0)) + continue; + + /* + * Create a fresh bio_list for all subordinate requests. + */ + bio_list_on_stack[1] = bio_list_on_stack[0]; + bio_list_init(&bio_list_on_stack[0]); + + ret = do_make_request(bio); + + /* + * Sort new bios into those for a lower level and those for the + * same level. + */ + bio_list_init(&lower); + bio_list_init(&same); + while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL) + if (q == bio->bi_disk->queue) + bio_list_add(&same, bio); + else + bio_list_add(&lower, bio); + + /* + * Now assemble so we handle the lowest level first. + */ + bio_list_merge(&bio_list_on_stack[0], &lower); + bio_list_merge(&bio_list_on_stack[0], &same); + bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]); + } while ((bio = bio_list_pop(&bio_list_on_stack[0]))); + + current->bio_list = NULL; + return ret; +} + /** * submit_bio_noacct - re-submit a bio to the block device layer for I/O * @bio: The bio describing the location in memory and on the device. @@ -1096,17 +1164,8 @@ static blk_qc_t do_make_request(struct bio *bio) */ blk_qc_t submit_bio_noacct(struct bio *bio) { - /* - * bio_list_on_stack[0] contains bios submitted by the current - * ->submit_bio. - * bio_list_on_stack[1] contains bios that were submitted before the - * current ->submit_bio_bio, but that haven't been processed yet. - */ - struct bio_list bio_list_on_stack[2]; - blk_qc_t ret = BLK_QC_T_NONE; - if (!submit_bio_checks(bio)) - goto out; + return BLK_QC_T_NONE; /* * We only want one ->submit_bio to be active at a time, else @@ -1120,58 +1179,10 @@ blk_qc_t submit_bio_noacct(struct bio *bio) */ if (current->bio_list) { bio_list_add(¤t->bio_list[0], bio); - goto out; + return BLK_QC_T_NONE; } - /* following loop may be a bit non-obvious, and so deserves some - * explanation. - * Before entering the loop, bio->bi_next is NULL (as all callers - * ensure that) so we have a list with a single bio. - * We pretend that we have just taken it off a longer list, so - * we assign bio_list to a pointer to the bio_list_on_stack, - * thus initialising the bio_list of new bios to be - * added. ->submit_bio() may indeed add some more bios - * through a recursive call to submit_bio_noacct. If it - * did, we find a non-NULL value in bio_list and re-enter the loop - * from the top. In this case we really did just take the bio - * of the top of the list (no pretending) and so remove it from - * bio_list, and call into ->submit_bio() again. - */ - BUG_ON(bio->bi_next); - bio_list_init(&bio_list_on_stack[0]); - current->bio_list = bio_list_on_stack; - do { - struct request_queue *q = bio->bi_disk->queue; - - if (likely(bio_queue_enter(bio) == 0)) { - struct bio_list lower, same; - - /* Create a fresh bio_list for all subordinate requests */ - bio_list_on_stack[1] = bio_list_on_stack[0]; - bio_list_init(&bio_list_on_stack[0]); - ret = do_make_request(bio); - - /* sort new bios into those for a lower level - * and those for the same level - */ - bio_list_init(&lower); - bio_list_init(&same); - while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL) - if (q == bio->bi_disk->queue) - bio_list_add(&same, bio); - else - bio_list_add(&lower, bio); - /* now assemble so we handle the lowest level first */ - bio_list_merge(&bio_list_on_stack[0], &lower); - bio_list_merge(&bio_list_on_stack[0], &same); - bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]); - } - bio = bio_list_pop(&bio_list_on_stack[0]); - } while (bio); - current->bio_list = NULL; /* deactivate */ - -out: - return ret; + return __submit_bio_noacct(bio); } EXPORT_SYMBOL(submit_bio_noacct); -- 2.26.2
next prev parent reply other threads:[~2020-06-29 19:39 UTC|newest] Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-29 19:39 rename ->make_request_fn and move it to the block_device_operations Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 01/20] nfblock: stop using ->queuedata Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 21:47 ` Geert Uytterhoeven 2020-06-29 21:47 ` Geert Uytterhoeven 2020-06-29 19:39 ` [PATCH 02/20] simdisk: " Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 03/20] drbd: " Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 04/20] null_blk: stop using ->queuedata for bio mode Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 05/20] ps3vram: stop using ->queuedata Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 06/20] rsxx: " Christoph Hellwig 2020-06-29 19:39 ` [PATCH 07/20] umem: " Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 08/20] zram: " Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 09/20] bcache: stop setting ->queuedata Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 10/20] dm: stop using ->queuedata Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 11/20] fs: remove a weird comment in submit_bh_wbc Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-30 13:54 ` Jens Axboe 2020-06-30 13:54 ` Jens Axboe 2020-06-29 19:39 ` [PATCH 12/20] block: remove the request_queue argument from blk_queue_split Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 13/20] block: tidy up a warning in bio_check_ro Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 14/20] block: remove the NULL queue check in generic_make_request_checks Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 15/20] block: remove the nr_sectors variable " Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 16/20] block: move ->make_request_fn to struct block_device_operations Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 17/20] block: rename generic_make_request to submit_bio_noacct Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig [this message] 2020-06-29 19:39 ` [PATCH 18/20] block: refator submit_bio_noacct Christoph Hellwig 2020-07-02 14:10 ` Qian Cai 2020-07-02 14:10 ` Qian Cai [not found] ` <20200702141001.GA3834-J5quhbR+WMc@public.gmane.org> 2020-07-02 15:14 ` Christoph Hellwig 2020-07-02 15:14 ` Christoph Hellwig 2020-07-02 15:52 ` Naresh Kamboju 2020-07-02 15:52 ` Naresh Kamboju 2020-07-02 15:15 ` Naresh Kamboju 2020-07-02 15:15 ` Naresh Kamboju 2020-06-29 19:39 ` [PATCH 19/20] block: shortcut __submit_bio_noacct for blk-mq drivers Christoph Hellwig 2020-06-29 19:39 ` Christoph Hellwig 2020-06-29 19:39 ` [PATCH 20/20] block: remove direct_make_request Christoph Hellwig 2020-06-30 13:57 ` rename ->make_request_fn and move it to the block_device_operations Jens Axboe 2020-06-30 13:57 ` Jens Axboe 2020-06-30 15:43 ` Jens Axboe 2020-06-30 15:43 ` Jens Axboe 2020-06-30 18:19 ` Christoph Hellwig 2020-06-30 18:19 ` Christoph Hellwig 2020-06-30 18:21 ` Jens Axboe 2020-06-30 18:21 ` Jens Axboe 2020-06-30 18:55 ` Jens Axboe 2020-07-01 8:59 rename ->make_request_fn and move it to the block_device_operations v2 Christoph Hellwig 2020-07-01 8:59 ` [PATCH 18/20] block: refator submit_bio_noacct Christoph Hellwig 2020-07-01 8:59 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200629193947.2705954-19-hch@lst.de \ --to=hch@lst.de \ --cc=axboe@kernel.dk \ --cc=dm-devel@redhat.com \ --cc=drbd-dev@tron.linbit.com \ --cc=linux-bcache@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-m68k@vger.kernel.org \ --cc=linux-nvdimm@lists.01.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-raid@vger.kernel.org \ --cc=linux-s390@vger.kernel.org \ --cc=linux-xtensa@linux-xtensa.org \ --cc=linuxppc-dev@lists.ozlabs.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).