From: dannyshih <dannyshih@synology.com> To: axboe@kernel.dk Cc: agk@redhat.com, snitzer@redhat.com, dm-devel@redhat.com, song@kernel.org, linux-block@vger.kernel.org, linux-raid@vger.kernel.org, Danny Shih <dannyshih@synology.com> Subject: [PATCH 0/4] Fix order when split bio and send remaining back to itself Date: Tue, 29 Dec 2020 17:18:38 +0800 [thread overview] Message-ID: <1609233522-25837-1-git-send-email-dannyshih@synology.com> (raw) From: Danny Shih <dannyshih@synology.com> We found out that split bios might handle not in order when a big bio had split by blk_queue_split() and also split in stacking block device, such as md device because chunk size boundary limit. Stacking block device normally use submit_bio_noacct() add the remaining bio to current->bio_list's tail after they split original bio. Therefore, when bio split first time, the last part of bio was add to bio_list. After then, when bio split second time, the middle part of bio was add to bio_list. Results that the middle part is now behind the last part of bio. For example: There is a RAID0 md device, with max_sectors_kb = 2 KB, and chunk_size = 1 KB 1. a read bio come to md device wants to read 0-7 KB 2. In blk_queue_split(), bio split into (0-1), (2-7), and send (2-7) back to md device current->bio_list = bio_list_on_stack[0]: (md 2-7) 3. RAID0 split bio (0-1) into (0) and (1), since chunk size is 1 KB and send (1) back to md device bio_list_on_stack[0]: (md 2-7) -> (md 1) 4. remap and send (0) to lower layer device bio_list_on_stack[0]: (md 2-7) -> (md 1) -> (lower 0) 5. __submit_bio_noacct() sorting bio let lower bio handle firstly bio_list_on_stack[0]: (lower 0) -> (md 2-7) -> (md 1) pop (lower 0) move bio_list_on_stack[0] to bio_list_on_stack[1] bio_list_on_stack[1]: (md 2-7) -> (md 1) 6. after handle lower bio, it handle (md 2-7) firstly, and split in blk_queue_split() into (2-3), (4-7), send (4-7) back bio_list_on_stack[0]: (md 4-7) bio_list_on_stack[1]: (md 1) 7. RAID0 split bio (2-3) into (2) and (3) and send (3) back bio_list_on_stack[0]: (md 4-7) -> (md 3) bio_list_on_stack[1]: (md 1) ... In the end, the split bio handle's order will become 0 -> 2 -> 4 -> 6 -> 7 -> 5 -> 3 -> 1 Reverse the order of same queue bio when sorting bio in __submit_bio_noacct() can solve this issue, but it might influence too much. So we provide alternative version of submit_bio_noacct(), named submit_bio_noacct_add_head(), for the case which need to add bio to the head of current->bio_list. And replace submit_bio_noacct() with submit_bio_noacct_add_head() in block device layer when we want to split bio and send remaining back to itself. Danny Shih (4): block: introduce submit_bio_noacct_add_head block: use submit_bio_noacct_add_head for split bio sending back dm: use submit_bio_noacct_add_head for split bio sending back md: use submit_bio_noacct_add_head for split bio sending back block/blk-core.c | 44 +++++++++++++++++++++++++++++++++----------- block/blk-merge.c | 2 +- block/bounce.c | 2 +- drivers/md/dm.c | 2 +- drivers/md/md-linear.c | 2 +- drivers/md/raid0.c | 4 ++-- drivers/md/raid1.c | 4 ++-- drivers/md/raid10.c | 4 ++-- drivers/md/raid5.c | 2 +- include/linux/blkdev.h | 1 + 10 files changed, 45 insertions(+), 22 deletions(-) -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: dannyshih <dannyshih@synology.com> To: axboe@kernel.dk Cc: linux-raid@vger.kernel.org, snitzer@redhat.com, linux-block@vger.kernel.org, song@kernel.org, dm-devel@redhat.com, Danny Shih <dannyshih@synology.com>, agk@redhat.com Subject: [dm-devel] [PATCH 0/4] Fix order when split bio and send remaining back to itself Date: Tue, 29 Dec 2020 17:18:38 +0800 [thread overview] Message-ID: <1609233522-25837-1-git-send-email-dannyshih@synology.com> (raw) From: Danny Shih <dannyshih@synology.com> We found out that split bios might handle not in order when a big bio had split by blk_queue_split() and also split in stacking block device, such as md device because chunk size boundary limit. Stacking block device normally use submit_bio_noacct() add the remaining bio to current->bio_list's tail after they split original bio. Therefore, when bio split first time, the last part of bio was add to bio_list. After then, when bio split second time, the middle part of bio was add to bio_list. Results that the middle part is now behind the last part of bio. For example: There is a RAID0 md device, with max_sectors_kb = 2 KB, and chunk_size = 1 KB 1. a read bio come to md device wants to read 0-7 KB 2. In blk_queue_split(), bio split into (0-1), (2-7), and send (2-7) back to md device current->bio_list = bio_list_on_stack[0]: (md 2-7) 3. RAID0 split bio (0-1) into (0) and (1), since chunk size is 1 KB and send (1) back to md device bio_list_on_stack[0]: (md 2-7) -> (md 1) 4. remap and send (0) to lower layer device bio_list_on_stack[0]: (md 2-7) -> (md 1) -> (lower 0) 5. __submit_bio_noacct() sorting bio let lower bio handle firstly bio_list_on_stack[0]: (lower 0) -> (md 2-7) -> (md 1) pop (lower 0) move bio_list_on_stack[0] to bio_list_on_stack[1] bio_list_on_stack[1]: (md 2-7) -> (md 1) 6. after handle lower bio, it handle (md 2-7) firstly, and split in blk_queue_split() into (2-3), (4-7), send (4-7) back bio_list_on_stack[0]: (md 4-7) bio_list_on_stack[1]: (md 1) 7. RAID0 split bio (2-3) into (2) and (3) and send (3) back bio_list_on_stack[0]: (md 4-7) -> (md 3) bio_list_on_stack[1]: (md 1) ... In the end, the split bio handle's order will become 0 -> 2 -> 4 -> 6 -> 7 -> 5 -> 3 -> 1 Reverse the order of same queue bio when sorting bio in __submit_bio_noacct() can solve this issue, but it might influence too much. So we provide alternative version of submit_bio_noacct(), named submit_bio_noacct_add_head(), for the case which need to add bio to the head of current->bio_list. And replace submit_bio_noacct() with submit_bio_noacct_add_head() in block device layer when we want to split bio and send remaining back to itself. Danny Shih (4): block: introduce submit_bio_noacct_add_head block: use submit_bio_noacct_add_head for split bio sending back dm: use submit_bio_noacct_add_head for split bio sending back md: use submit_bio_noacct_add_head for split bio sending back block/blk-core.c | 44 +++++++++++++++++++++++++++++++++----------- block/blk-merge.c | 2 +- block/bounce.c | 2 +- drivers/md/dm.c | 2 +- drivers/md/md-linear.c | 2 +- drivers/md/raid0.c | 4 ++-- drivers/md/raid1.c | 4 ++-- drivers/md/raid10.c | 4 ++-- drivers/md/raid5.c | 2 +- include/linux/blkdev.h | 1 + 10 files changed, 45 insertions(+), 22 deletions(-) -- 2.7.4 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
next reply other threads:[~2020-12-29 9:29 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-29 9:18 dannyshih [this message] 2020-12-29 9:18 ` [dm-devel] [PATCH 0/4] Fix order when split bio and send remaining back to itself dannyshih 2020-12-29 9:18 ` [PATCH 1/4] block: introduce submit_bio_noacct_add_head dannyshih 2020-12-29 9:18 ` [dm-devel] " dannyshih 2020-12-30 0:00 ` John Stoffel 2020-12-30 0:00 ` [dm-devel] " John Stoffel 2020-12-30 9:51 ` Danny Shih 2020-12-30 9:51 ` [dm-devel] " Danny Shih 2020-12-30 17:06 ` John Stoffel 2020-12-30 17:06 ` [dm-devel] " John Stoffel 2020-12-30 17:53 ` antlists 2020-12-30 17:53 ` [dm-devel] " antlists 2020-12-30 11:35 ` antlists 2020-12-30 11:35 ` [dm-devel] " antlists 2020-12-30 16:53 ` John Stoffel 2020-12-30 16:53 ` [dm-devel] " John Stoffel 2020-12-29 9:18 ` [PATCH 2/4] block: use submit_bio_noacct_add_head for split bio sending back dannyshih 2020-12-29 9:18 ` [dm-devel] " dannyshih 2020-12-29 9:18 ` [PATCH 3/4] dm: " dannyshih 2020-12-29 9:18 ` [dm-devel] " dannyshih 2020-12-29 9:18 ` [PATCH 4/4] md: " dannyshih 2020-12-29 9:18 ` [dm-devel] " dannyshih 2020-12-30 23:34 ` [PATCH 0/4] Fix order when split bio and send remaining back to itself Mike Snitzer 2020-12-30 23:34 ` [dm-devel] " Mike Snitzer 2020-12-31 8:28 ` Danny Shih 2020-12-31 8:28 ` [dm-devel] " Danny Shih
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1609233522-25837-1-git-send-email-dannyshih@synology.com \ --to=dannyshih@synology.com \ --cc=agk@redhat.com \ --cc=axboe@kernel.dk \ --cc=dm-devel@redhat.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-raid@vger.kernel.org \ --cc=snitzer@redhat.com \ --cc=song@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.