From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 Sender: pankaj.scorpio@gmail.com In-Reply-To: <20180514175354.GA15053@redhat.com> References: <20180514161312.GA14685@redhat.com> <20180514175354.GA15053@redhat.com> From: Pankaj Agarwal Date: Mon, 14 May 2018 19:57:48 -0400 Message-ID: Content-Type: multipart/alternative; boundary="000000000000c6effa056c334073" Subject: Re: [linux-lvm] how to set higher then 128 nr_requests on LV's Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: Mike Snitzer Cc: Hannes Reinecke , dm-devel@redhat.com, linux-lvm@redhat.com --000000000000c6effa056c334073 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hannes, While you are working on the feature code, is there a sequence of system commands I can use ? thanks, Pankaj On Mon, May 14, 2018 at 1:53 PM, Mike Snitzer wrote: > On Mon, May 14 2018 at 12:13pm -0400, > Mike Snitzer wrote: > > > On Sun, May 13 2018 at 8:47pm -0400, > > Pankaj Agarwal wrote: > > > > > Hi, > > > How do i set the nr_request value for LV's as it's not writable > like other > > > drives on a linux system. > > > LV's are set as dm-0 and dm-1 on my system. > > > #cat /sys/block/dm-0/queue/nr_requests > > > 128 > > > # echo 256 > /sys/block/dm-0/queue/nr_requests > > > -bash: echo: write error: Invalid argument > > > > In the block layer, bio-based devices cannot have their nr_requests > > changed, see: block/blk-sysfs.c:queue_requests_store() > > > > So any bio-based DM device (e.g. linear target) won't be allowed to > > change nr_requests. > > > > The only way to change this would be from the bottom up: > > 1) change the underlying request-based device(s) (e.g. /dev/sda) > > 2) dmsetup suspend + dmsetup resume any bio-based DM device(s) that are > > stacked ontop of the request-based device(s) you changed in 1) > > -- this will restack the queue_limits from the bottom up; so the DM > > device will then reflect the underlying devices' limits. > > I was mistaken. DM core (in the kernel) will only restack the > queue_limits (nr_requests included) if a new DM table is loaded. > > So a simple suspend+resume will _not_ change the bio-based DM device's > queue_limits: > > [root@thegoat ~]# lsblk /dev/nullb0 > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT > nullb0 253:0 0 250G 0 disk > =E2=94=94=E2=94=80foo 252:0 0 250G 0 dm > [root@thegoat ~]# cat /sys/block/nullb0/queue/nr_requests > 128 > [root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests > 128 > [root@thegoat ~]# echo 64 > /sys/block/nullb0/queue/nr_requests > [root@thegoat ~]# cat /sys/block/nullb0/queue/nr_requests > 64 > [root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests > 128 > [root@thegoat ~]# dmsetup suspend foo > [root@thegoat ~]# dmsetup resume foo > [root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests > 128 > > There was some related work started in this area by Hannes (now cc'd) > during LSF last month. He added a chain-notifier to DM devices so that > if any underlying devices (devices referenced in a DM device's table) > were changed then the enture device stack would have its limits > reloaded. > > Not sure where Hannes ended up with that. Would love to get it polished > now and staged for 4.18 inclusion. > > Hannes? > --000000000000c6effa056c334073 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hannes,

While you are working on the fe= ature code, is there a sequence of system commands I can use ?
thanks,

Pankaj



On Mon, May 14, 2018 at 1:53 PM, Mike Snitzer <snitzer@redhat.com&g= t; wrote:
On Mon,= May 14 2018 at 12:13pm -0400,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Sun, May 13 2018 at=C2=A0 8:47pm -0400,
> Pankaj Agarwal <pankaj@= releasemanager.in> wrote:
>
> >=C2=A0 =C2=A0 Hi,
> >=C2=A0 =C2=A0 How do i set the nr_request value for LV's as it= 's not writable like other
> >=C2=A0 =C2=A0 drives on a linux system.
> >=C2=A0 =C2=A0 LV's are set as dm-0 and dm-1 on my system.
> >=C2=A0 =C2=A0 =C2=A0#cat /sys/block/dm-0/queue/nr_requests > >=C2=A0 =C2=A0 128
> >=C2=A0 =C2=A0 # echo 256 > /sys/block/dm-0/queue/nr_reques= ts
> >=C2=A0 =C2=A0 -bash: echo: write error: Invalid argument
>
> In the block layer, bio-based devices cannot have their nr_requests > changed, see: block/blk-sysfs.c:queue_requests_store()
>
> So any bio-based DM device (e.g. linear target) won't be allowed t= o
> change nr_requests.
>
> The only way to change this would be from the bottom up:
> 1) change the underlying request-based device(s) (e.g. /dev/sda)
> 2) dmsetup suspend + dmsetup resume any bio-based DM device(s) that ar= e
>=C2=A0 =C2=A0 stacked ontop of the request-based device(s) you changed = in 1)
>=C2=A0 =C2=A0 -- this will restack the queue_limits from the bottom up;= so the DM
>=C2=A0 =C2=A0 device will then reflect the underlying devices' limi= ts.

I was mistaken.=C2=A0 DM core (in the kernel) will only restack the<= br> queue_limits (nr_requests included) if a new DM table is loaded.

So a simple suspend+resume will _not_ change the bio-based DM device's<= br> queue_limits:

[root@thegoat ~]# lsblk /dev/nullb0
NAME=C2=A0 =C2=A0MAJ:MIN RM=C2=A0 SIZE RO TYPE MOUNTPOINT
nullb0 253:0=C2=A0 =C2=A0 0=C2=A0 250G=C2=A0 0 disk
=E2=94=94=E2=94=80foo=C2=A0 252:0=C2=A0 =C2=A0 0=C2=A0 250G=C2=A0 0 dm
[root@thegoat ~]# cat /sys/block/nullb0/queue/nr_requests
128
[root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests
128
[root@thegoat ~]# echo 64 > /sys/block/nullb0/queue/nr_requests
[root@thegoat ~]# cat /sys/block/nullb0/queue/nr_requests
64
[root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests
128
[root@thegoat ~]# dmsetup suspend foo
[root@thegoat ~]# dmsetup resume foo
[root@thegoat ~]# cat /sys/block/dm-0/queue/nr_requests
128

There was some related work started in this area by Hannes (now cc'd) during LSF last month.=C2=A0 He added a chain-notifier to DM devices so tha= t
if any underlying devices (devices referenced in a DM device's table) were changed then the enture device stack would have its limits
reloaded.

Not sure where Hannes ended up with that.=C2=A0 Would love to get it polish= ed
now and staged for 4.18 inclusion.

Hannes?

--000000000000c6effa056c334073--