* [PATCH V4 0/1] block: fix I/O errors in BLKRRPART
@ 2021-01-05 12:27 Minwoo Im
2021-01-05 12:27 ` [PATCH V4 1/1] block: reject I/O for same fd if block size changed Minwoo Im
0 siblings, 1 reply; 3+ messages in thread
From: Minwoo Im @ 2021-01-05 12:27 UTC (permalink / raw)
To: linux-block, linux-kernel, linux-fsdevel, linux-nvme
Cc: Jens Axboe, Alexander Viro, Christoph Hellwig,
Chaitanya Kulkarni, Minwoo Im
Hello,
This patch fixes I/O errors during BLKRRPART ioctl() behavior right
after format operation that changed logical block size of the block
device with a same file descriptor opened.
Testcase:
The following testcase is a case of NVMe namespace with the following
conditions:
- Current LBA format is lbaf=0 (512 bytes logical block size)
- LBA Format(lbaf=1) has 4096 bytes logical block size
# Format block device logical block size 512B to 4096B
nvme format /dev/nvme0n1 --lbaf=1 --force
This will cause I/O errors because BLKRRPART ioctl() happened right after
the format command with same file descriptor opened in application
(e.g., nvme-cli) like:
fd = open("/dev/nvme0n1", O_RDONLY);
nvme_format(fd, ...);
if (ioctl(fd, BLKRRPART) < 0)
...
Errors:
We can see the Read command with Number of LBA(NLB) 0xffff(65535) which
was under-flowed because BLKRRPART operation requested request size based
on i_blkbits of the block device which is 9 via buffer_head.
[dmesg-snip]
[ 10.771740] blk_update_request: operation not supported error, dev nvme0n1, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 10.780262] Buffer I/O error on dev nvme0n1, logical block 0, async page read
[event-snip]
kworker/0:1H-56 [000] .... 913.456922: nvme_setup_cmd: nvme0: disk=nvme0n1, qid=1, cmdid=216, nsid=1, flags=0x0, meta=0x0, cmd=(nvme_cmd_read slba=0, len=65535, ctrl=0x0, dsmgmt=0, reftag=0)
ksoftirqd/0-9 [000] .Ns. 916.566351: nvme_complete_rq: nvme0: disk=nvme0n1, qid=1, cmdid=216, res=0x0, retries=0, flags=0x0, status=0x4002
The patch below fixes the I/O errors by rejecting I/O requests from the
block layer with setting a flag to request_queue until the file descriptor
re-opened to be updated by __blkdev_get(). This is based on the previous
discussion [1].
Since V3(RFC):
- Move flag from gendisk to request_queue for future clean-ups.
(Christoph, [3])
Since V2(RFC):
- Cover letter with testcase and error logs attached. Removed un-related
changes: empty line. (Chaitanya, [2])
- Put blkdev with blkdev_put_no_open().
Since V1(RFC):
- Updated patch to reject I/O rather than updating i_blkbits of the
block device's inode directly from driver. (Christoph, [1])
[1] https://lore.kernel.org/linux-nvme/20201223183143.GB13354@localhost.localdomain/T/#t
[2] https://lore.kernel.org/linux-nvme/20201230140504.GB7917@localhost.localdomain/T/#t
[3] https://lore.kernel.org/linux-block/20210105101202.GA9970@localhost.localdomain/T/#u
Thanks,
Minwoo Im (1):
block: reject I/O for same fd if block size changed
block/blk-settings.c | 3 +++
block/partitions/core.c | 12 ++++++++++++
fs/block_dev.c | 8 ++++++++
include/linux/blkdev.h | 1 +
4 files changed, 24 insertions(+)
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH V4 1/1] block: reject I/O for same fd if block size changed
2021-01-05 12:27 [PATCH V4 0/1] block: fix I/O errors in BLKRRPART Minwoo Im
@ 2021-01-05 12:27 ` Minwoo Im
2021-01-05 15:02 ` Christoph Hellwig
0 siblings, 1 reply; 3+ messages in thread
From: Minwoo Im @ 2021-01-05 12:27 UTC (permalink / raw)
To: linux-block, linux-kernel, linux-fsdevel, linux-nvme
Cc: Jens Axboe, Alexander Viro, Christoph Hellwig,
Chaitanya Kulkarni, Minwoo Im
This patch fixes I/O errors during BLKRRPART ioctl() behavior right
after format operation that changed logical block size of the block
device with a same file descriptor opened.
This issue can be easily reproduced with a single format command in case
of NVMe (logical block size 512B to 4096B).
nvme format /dev/nvme0n1 --lbaf=1 --force
This is because the application, nvme-cli format subcommand issues an
admin command followed by BLKRRPART ioctl to re-read partition
information without closing the file descriptor. If file descriptor
stays opened, __blkdev_get() will not be invoked at all even logical
block size has been changed.
It will cause I/O errors with invalid Read operations during the
BLKRRPART ioctl due to i_blkbits mismatch. The invalid operations in
BLKRRPART happens with under-flowed Number of LBA(NLB) values
0xffff(65535) because i_blkbits is still set to 9 even the logical block
size has been updated to 4096. The BLKRRPART will lead buffer_head to
hold 512B data which is less than the logical lock size of the block
device.
The root cause, which is because i_blkbits of inode of the block device
is not updated, can be solved easily by re-opening file descriptor
again from application. But, that's just for application's business
and kernel should reject invalid Read operations during the BLKRRPART
ioctl.
This patch rejects I/O from the path of add_partitions() to avoid
issuing invalid Read operations to device. It sets a flag to
request_queue in blk_queue_logical_block_size to minimize caller-side
updates.
Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com>
---
block/blk-settings.c | 3 +++
block/partitions/core.c | 12 ++++++++++++
fs/block_dev.c | 8 ++++++++
include/linux/blkdev.h | 1 +
4 files changed, 24 insertions(+)
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 43990b1d148b..48a6fc7bb5f5 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -329,6 +329,9 @@ void blk_queue_logical_block_size(struct request_queue *q, unsigned int size)
{
struct queue_limits *limits = &q->limits;
+ if (limits->logical_block_size != size)
+ blk_queue_flag_set(QUEUE_FLAG_LBSZ_CHANGED, q);
+
limits->logical_block_size = size;
if (limits->physical_block_size < size)
diff --git a/block/partitions/core.c b/block/partitions/core.c
index e7d776db803b..6f175ea18ff3 100644
--- a/block/partitions/core.c
+++ b/block/partitions/core.c
@@ -612,12 +612,24 @@ static bool blk_add_partition(struct gendisk *disk, struct block_device *bdev,
int blk_add_partitions(struct gendisk *disk, struct block_device *bdev)
{
+ struct request_queue *q = bdev_get_queue(bdev);
struct parsed_partitions *state;
int ret = -EAGAIN, p, highest;
if (!disk_part_scan_enabled(disk))
return 0;
+ /*
+ * Reject to check partition information if block size has been changed
+ * in the runtime. If block size of a block device has been changed,
+ * the file descriptor should be opened agian to update the blkbits.
+ */
+ if (test_bit(QUEUE_FLAG_LBSZ_CHANGED, &q->queue_flags)) {
+ pr_warn("%s: rejecting checking partition. fd should be opened again.\n",
+ disk->disk_name);
+ return -EBADFD;
+ }
+
state = check_partition(disk, bdev);
if (!state)
return 0;
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 9293045e128c..8056a412a3d1 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -130,7 +130,15 @@ EXPORT_SYMBOL(truncate_bdev_range);
static void set_init_blocksize(struct block_device *bdev)
{
+ struct request_queue *q = bdev_get_queue(bdev);
+
bdev->bd_inode->i_blkbits = blksize_bits(bdev_logical_block_size(bdev));
+
+ /*
+ * Allow I/O commands for this block device. We can say that this
+ * block device has proper blkbits updated.
+ */
+ blk_queue_flag_clear(QUEUE_FLAG_LBSZ_CHANGED, q);
}
int set_blocksize(struct block_device *bdev, int size)
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 070de09425ad..6d0542434be6 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -625,6 +625,7 @@ struct request_queue {
#define QUEUE_FLAG_RQ_ALLOC_TIME 27 /* record rq->alloc_time_ns */
#define QUEUE_FLAG_HCTX_ACTIVE 28 /* at least one blk-mq hctx is active */
#define QUEUE_FLAG_NOWAIT 29 /* device supports NOWAIT */
+#define QUEUE_FLAG_LBSZ_CHANGED 30 /* logical block size changed */
#define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \
(1 << QUEUE_FLAG_SAME_COMP) | \
--
2.17.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH V4 1/1] block: reject I/O for same fd if block size changed
2021-01-05 12:27 ` [PATCH V4 1/1] block: reject I/O for same fd if block size changed Minwoo Im
@ 2021-01-05 15:02 ` Christoph Hellwig
0 siblings, 0 replies; 3+ messages in thread
From: Christoph Hellwig @ 2021-01-05 15:02 UTC (permalink / raw)
To: Minwoo Im
Cc: linux-block, linux-kernel, linux-fsdevel, linux-nvme, Jens Axboe,
Alexander Viro, Christoph Hellwig, Chaitanya Kulkarni
Looks good to me, thanks a lot!
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-01-05 15:02 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-05 12:27 [PATCH V4 0/1] block: fix I/O errors in BLKRRPART Minwoo Im
2021-01-05 12:27 ` [PATCH V4 1/1] block: reject I/O for same fd if block size changed Minwoo Im
2021-01-05 15:02 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).