[PATCH] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

[PATCH] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

[Mikulas Patocka]

We get these I/O errors when we run md-raid1 on the top of dm-integrity on 
the top of ramdisk:
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The 
dm-integrity device uses logical_block_size 4096 and it doesn't affect the 
"max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have 
a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it
will split the bios on 255-sector boundary, making the bios unaligned on
logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org

---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================
--- linux-2.6.orig/block/blk-settings.c	2020-10-29 12:20:46.000000000 +0100
+++ linux-2.6/block/blk-settings.c	2020-11-19 21:20:18.000000000 +0100
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
 		ret = -1;
 	}
 
+	t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_sectors = PAGE_SIZE / 512;
+	t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_hw_sectors = PAGE_SIZE / 512;
+	t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_dev_sectors = PAGE_SIZE / 512;
+
 	/* Discard alignment and granularity */
 	if (b->discard_granularity) {
 		alignment = queue_limit_discard_alignment(b, start);

Re: [dm-devel] [PATCH] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

[John Dorminy]

Greetings;

Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >>
SECTOR_SHIFT instead of / 512.

I don't understand the three conditionals. I believe max_sectors is
supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't
understand why max_sectors being small should adjust max_hw_sectors
and max_dev_sectors. Are the conditions perhaps supposed to be
different, adjusting each max_*sectors up to at least PAGE_SIZE /
SECTOR_SIZE? Perhaps, like e.g. blk_queue_max_hw_sectors(), the
conditionals should log if they are adjusting max_*sectors up to the
minimum.

Thanks!

John Dorminy

On Thu, Nov 19, 2020 at 3:37 PM Mikulas Patocka <mpatocka@redhat.com> wrote:
>
> We get these I/O errors when we run md-raid1 on the top of dm-integrity on
> the top of ramdisk:
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb
>
> The ramdisk device has logical_block_size 512 and max_sectors 255. The
> dm-integrity device uses logical_block_size 4096 and it doesn't affect the
> "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have
> a device with max_sectors not aligned on logical_block_size.
>
> The md-raid device sees that the underlying leg has max_sectors 255 and it
> will split the bios on 255-sector boundary, making the bios unaligned on
> logical_block_size.
>
> In order to fix the bug, we round down max_sectors to logical_block_size.
>
> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
> Cc: stable@vger.kernel.org
>
> ---
>  block/blk-settings.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> Index: linux-2.6/block/blk-settings.c
> ===================================================================
> --- linux-2.6.orig/block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100
> +++ linux-2.6/block/blk-settings.c      2020-11-19 21:20:18.000000000 +0100
> @@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
>                 ret = -1;
>         }
>
> +       t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
> +       if (t->max_sectors < PAGE_SIZE / 512)
> +               t->max_sectors = PAGE_SIZE / 512;
> +       t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
> +       if (t->max_sectors < PAGE_SIZE / 512)
> +               t->max_hw_sectors = PAGE_SIZE / 512;
> +       t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
> +       if (t->max_sectors < PAGE_SIZE / 512)
> +               t->max_dev_sectors = PAGE_SIZE / 512;
> +
>         /* Discard alignment and granularity */
>         if (b->discard_granularity) {
>                 alignment = queue_limit_discard_alignment(b, start);
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

Re: [dm-devel] [PATCH] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

[Mikulas Patocka]

On Thu, 19 Nov 2020, John Dorminy wrote:

> Greetings;
> 
> Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >>
> SECTOR_SHIFT instead of / 512.

Yes, that's a good point.

> I don't understand the three conditionals. I believe max_sectors is
> supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't
> understand why max_sectors being small should adjust max_hw_sectors
> and max_dev_sectors. Are the conditions perhaps supposed to be
> different, adjusting each max_*sectors up to at least PAGE_SIZE /
> SECTOR_SIZE?

I copied this pattern from blk_queue_max_hw_sectors. Perhaps, we could 
use:
	t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
	if (!t->max_sectors)
		t->max_sectors = t->logical_block_size / 512;
instead.

Jens, what do you think is better?

Mikulas

> Perhaps, like e.g. blk_queue_max_hw_sectors(), the
> conditionals should log if they are adjusting max_*sectors up to the
> minimum.
> 
> Thanks!
> 
> John Dorminy
> 
> On Thu, Nov 19, 2020 at 3:37 PM Mikulas Patocka <mpatocka@redhat.com> wrote:
> >
> > We get these I/O errors when we run md-raid1 on the top of dm-integrity on
> > the top of ramdisk:
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
> > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb
> >
> > The ramdisk device has logical_block_size 512 and max_sectors 255. The
> > dm-integrity device uses logical_block_size 4096 and it doesn't affect the
> > "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have
> > a device with max_sectors not aligned on logical_block_size.
> >
> > The md-raid device sees that the underlying leg has max_sectors 255 and it
> > will split the bios on 255-sector boundary, making the bios unaligned on
> > logical_block_size.
> >
> > In order to fix the bug, we round down max_sectors to logical_block_size.
> >
> > Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
> > Cc: stable@vger.kernel.org
> >
> > ---
> >  block/blk-settings.c |   10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > Index: linux-2.6/block/blk-settings.c
> > ===================================================================
> > --- linux-2.6.orig/block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100
> > +++ linux-2.6/block/blk-settings.c      2020-11-19 21:20:18.000000000 +0100
> > @@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
> >                 ret = -1;
> >         }
> >
> > +       t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
> > +       if (t->max_sectors < PAGE_SIZE / 512)
> > +               t->max_sectors = PAGE_SIZE / 512;
> > +       t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
> > +       if (t->max_sectors < PAGE_SIZE / 512)
> > +               t->max_hw_sectors = PAGE_SIZE / 512;
> > +       t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
> > +       if (t->max_sectors < PAGE_SIZE / 512)
> > +               t->max_dev_sectors = PAGE_SIZE / 512;
> > +
> >         /* Discard alignment and granularity */
> >         if (b->discard_granularity) {
> >                 alignment = queue_limit_discard_alignment(b, start);
> >
> > --
> > dm-devel mailing list
> > dm-devel@redhat.com
> > https://www.redhat.com/mailman/listinfo/dm-devel
> >
> 

[PATCH v2] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

[Mikulas Patocka]

We get I/O errors when we run md-raid1 on the top of dm-integrity on the
top of ramdisk.
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The
dm-integrity device uses logical_block_size 4096 and it doesn't affect the
"max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have
a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it
will split the bios on 255-sector boundary, making the bios unaligned on
logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org

---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================
--- linux-2.6.orig/block/blk-settings.c	2020-10-29 12:20:46.000000000 +0100
+++ linux-2.6/block/blk-settings.c	2020-11-20 15:07:59.000000000 +0100
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
 		ret = -1;
 	}
 
+	t->max_sectors = round_down(t->max_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+	t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_hw_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_hw_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+	t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_dev_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_dev_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+
 	/* Discard alignment and granularity */
 	if (b->discard_granularity) {
 		alignment = queue_limit_discard_alignment(b, start);

Re: [PATCH v2] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary

[Jens Axboe]

Applied, thanks.

-- 
Jens Axboe

Re: [PATCH v2] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary

[Ming Lei]

On Tue, Feb 23, 2021 at 11:28:27AM -0500, Mikulas Patocka wrote:
> 
> 
> On Tue, 23 Feb 2021, Ming Lei wrote:
> 
> > I'd suggest to add a helper(such as, blk_round_down_sectors()) to round_down each
> > one.
> 
> Yes - Here I'm sending the updated patch.
> 
> > -- 
> > Ming
> 
> From: Mikulas Patocka <mpatocka@redhat.com>
> 
> We get I/O errors when we run md-raid1 on the top of dm-integrity on the
> top of ramdisk.
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
> device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb
> 
> The ramdisk device has logical_block_size 512 and max_sectors 255. The
> dm-integrity device uses logical_block_size 4096 and it doesn't affect the
> "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have
> a device with max_sectors not aligned on logical_block_size.
> 
> The md-raid device sees that the underlying leg has max_sectors 255 and it
> will split the bios on 255-sector boundary, making the bios unaligned on
> logical_block_size.
> 
> In order to fix the bug, we round down max_sectors to logical_block_size.
> 
> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
> Cc: stable@vger.kernel.org
> 
> ---
>  block/blk-settings.c |   12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> Index: linux-2.6/block/blk-settings.c
> ===================================================================
> --- linux-2.6.orig/block/blk-settings.c	2021-02-23 17:18:59.000000000 +0100
> +++ linux-2.6/block/blk-settings.c	2021-02-23 17:23:58.000000000 +0100
> @@ -481,6 +481,14 @@ void blk_queue_io_opt(struct request_que
>  }
>  EXPORT_SYMBOL(blk_queue_io_opt);
>  
> +static unsigned int blk_round_down_sectors(unsigned int sectors, unsigned int lbs)
> +{
> +	sectors = round_down(sectors, lbs >> SECTOR_SHIFT);
> +	if (sectors < PAGE_SIZE >> SECTOR_SHIFT)
> +		sectors = PAGE_SIZE >> SECTOR_SHIFT;
> +	return sectors;
> +}
> +
>  /**
>   * blk_stack_limits - adjust queue_limits for stacked devices
>   * @t:	the stacking driver limits (top device)
> @@ -607,6 +615,10 @@ int blk_stack_limits(struct queue_limits
>  		ret = -1;
>  	}
>  
> +	t->max_sectors = blk_round_down_sectors(t->max_sectors, t->logical_block_size);
> +	t->max_hw_sectors = blk_round_down_sectors(t->max_hw_sectors, t->logical_block_size);
> +	t->max_dev_sectors = blk_round_down_sectors(t->max_dev_sectors, t->logical_block_size);
> +
>  	/* Discard alignment and granularity */
>  	if (b->discard_granularity) {
>  		alignment = queue_limit_discard_alignment(b, start);

Reviewed-by: Ming Lei <ming.lei@redhat.com>

-- 
Ming

[PATCH v2] blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary

[Mikulas Patocka]

On Tue, 23 Feb 2021, Ming Lei wrote:

> I'd suggest to add a helper(such as, blk_round_down_sectors()) to round_down each
> one.

Yes - Here I'm sending the updated patch.

> -- 
> Ming

From: Mikulas Patocka <mpatocka@redhat.com>

We get I/O errors when we run md-raid1 on the top of dm-integrity on the
top of ramdisk.
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The
dm-integrity device uses logical_block_size 4096 and it doesn't affect the
"max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have
a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it
will split the bios on 255-sector boundary, making the bios unaligned on
logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org

---
 block/blk-settings.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================
--- linux-2.6.orig/block/blk-settings.c	2021-02-23 17:18:59.000000000 +0100
+++ linux-2.6/block/blk-settings.c	2021-02-23 17:23:58.000000000 +0100
@@ -481,6 +481,14 @@ void blk_queue_io_opt(struct request_que
 }
 EXPORT_SYMBOL(blk_queue_io_opt);
 
+static unsigned int blk_round_down_sectors(unsigned int sectors, unsigned int lbs)
+{
+	sectors = round_down(sectors, lbs >> SECTOR_SHIFT);
+	if (sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		sectors = PAGE_SIZE >> SECTOR_SHIFT;
+	return sectors;
+}
+
 /**
  * blk_stack_limits - adjust queue_limits for stacked devices
  * @t:	the stacking driver limits (top device)
@@ -607,6 +615,10 @@ int blk_stack_limits(struct queue_limits
 		ret = -1;
 	}
 
+	t->max_sectors = blk_round_down_sectors(t->max_sectors, t->logical_block_size);
+	t->max_hw_sectors = blk_round_down_sectors(t->max_hw_sectors, t->logical_block_size);
+	t->max_dev_sectors = blk_round_down_sectors(t->max_dev_sectors, t->logical_block_size);
+
 	/* Discard alignment and granularity */
 	if (b->discard_granularity) {
 		alignment = queue_limit_discard_alignment(b, start);