Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: Steven Davies <btrfs-list@steev.me.uk>
To: Anand Jain <anand.jain@oracle.com>,
	dsterba@suse.cz, dsterba@suse.com, linux-btrfs@vger.kernel.org
Subject: Re: [PATCH v7 rebased 0/5] readmirror feature (sysfs and in-memory only approach; with new read_policy device)
Date: Fri, 22 May 2020 20:15:32 +0100
Message-ID: <5a43edf5-6b0e-9523-eb9d-f3744bceeabd@steev.me.uk> (raw)
In-Reply-To: <c61a44bf-04ab-01a0-3fbe-4d5970827085@oracle.com>

On 19/05/2020 11:02, Anand Jain wrote:
> 
> 
> On 16/5/20 3:58 am, David Sterba wrote:
>> On Thu, Apr 30, 2020 at 05:02:27PM +0800, Anand Jain wrote:
>>>    I am not sure if this will be integrated in 5.8 and worth the time to
>>>    rebase. Kindly suggest.
>>
>> The preparatory work is ok, but the actual mirror selection policy
>> addresses a usecase that I think is not the one most users are
>> interested in. Devices of vastly different performance capabilities like
>> rotational disks vs nvme vs ssd vs network block devices in one
>> filesystem are not something commonly found.
>>
>> What we really need is a saner balancing mechanism than pid-based, that
>> is also going to be used any time there are more devices from the same
>> speed class for the fast devices too.

I can't find documentation about how mdadm chooses which mirror to read 
from (in the absence of --write-mostly) but everything suggests it uses 
different mirrors for different processes which sounds the same as or 
very much like %pid. Are you thinking along the lines of checking I/O 
queue length or average seek/read time when choosing the device?

> There are two things here, the read_policy framework in the preparatory
> patches and a new balancing or read_policy, device.
> 
>> So, no the patchset is not on track for a merge without the improved
>> default balancing.
> 
> It can be worked on top of the preparatory read_policy framework?
> This patchset does not change any default read_policy (or balancing)
> which is pid as of now. Working on a default read_policy/balancing
> was out of the scope of this patchset.
> 
>> The preferred device for reads can be one of the
>> policies, I understand the usecase and have not problem with that
>> although wouldn't probably have use for it.
> 
> For us, read_policy:device helps to reproduce raid1 data corruption
>     https://patchwork.kernel.org/patch/11475417/
> And xfstests btrfs/14[0-3] can be improved so that the reads directly
> go the device of the choice, instead of waiting for the odd/even pid.
> 
> Common configuration won't need this, advance configurations assembled
> with heterogeneous devices where read performance is more critical than
> write will find read_policy:device useful.

+1 for merging this as-is; I'm one of the users with heterogeneous 
devices (SATA SSD and NVMe SSD) and it would bring a feature similar to 
mdadm's --write-mostly. I've seen a few other requests for this feature 
on other discussion fora as well.

-- 
Steven Davies

      parent reply index

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-06 11:51 Anand Jain
2020-04-06 11:51 ` [PATCH v7 1/5] btrfs: add btrfs_strmatch helper Anand Jain
2020-04-06 11:51 ` [PATCH v7 2/5] btrfs: create read policy framework Anand Jain
2020-04-06 11:51 ` [PATCH v7 3/5] btrfs: create read policy sysfs attribute, pid Anand Jain
2020-05-19 10:07   ` Johannes Thumshirn
2020-05-20  8:54     ` Anand Jain
2020-05-20  8:55       ` Johannes Thumshirn
2020-04-06 11:51 ` [PATCH v7 4/5] btrfs: introduce new device-state read_preferred Anand Jain
2020-04-06 11:51 ` [PATCH v7 5/5] btrfs: introduce new read_policy device Anand Jain
2020-04-30  9:02 ` [PATCH v7 rebased 0/5] readmirror feature (sysfs and in-memory only approach; with new read_policy device) Anand Jain
2020-05-15 19:58   ` David Sterba
2020-05-19 10:02     ` Anand Jain
2020-05-22 13:46       ` David Sterba
2020-05-26  7:23         ` Anand Jain
2020-05-22 19:15       ` Steven Davies [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5a43edf5-6b0e-9523-eb9d-f3744bceeabd@steev.me.uk \
    --to=btrfs-list@steev.me.uk \
    --cc=anand.jain@oracle.com \
    --cc=dsterba@suse.com \
    --cc=dsterba@suse.cz \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org
	public-inbox-index linux-btrfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git