dedicated metadata drives?

dedicated metadata drives?

[Tomasz Chmielewski]

According to btrfs wiki, some patches have been submitted to support 
metadata on different devices than data (i.e. metadata on SSD, data on 
HDD):


https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata_drives

    Dedicated metadata drives

    Not claimed — submitted — Not in kernel yet

    We're able to split data and metadata IO very easily. Metadata tends 
to be dominated by seeks and for many applications it makes sense to put 
the metadata onto faster SSDs.




This article (almost 2.5 years old) claims one company is already using 
either these patches or something similar:

https://lwn.net/Articles/698090/

    August 24, 2016

    To combat that, he has a set of patches to automatically put the 
Btrfs metadata on SSDs. The block layer provides information on whether 
the storage is rotational; for now, his patch assumes that if
    it is not rotational then it is fast. The patch has made a huge 
difference in the latencies and requires less flash storage (e.g. 450GB 
for 40TB filesystem) for Facebook's file workload that
    consists of a wide variety of file sizes.



Do these patches exist anywhere? I couldn't find them in the list 
archive.



Tomasz

RE: dedicated metadata drives?

[Paul Jones]

> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org <linux-btrfs-
> owner@vger.kernel.org> On Behalf Of Tomasz Chmielewski
> Sent: Saturday, 5 January 2019 12:25 AM
> To: Btrfs BTRFS <linux-btrfs@vger.kernel.org>
> Subject: dedicated metadata drives?
> 
> According to btrfs wiki, some patches have been submitted to support
> metadata on different devices than data (i.e. metadata on SSD, data on
> HDD):
> 
> 
> https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata
> _drives
> 
>     Dedicated metadata drives
> 
>     Not claimed — submitted — Not in kernel yet
> 
>     We're able to split data and metadata IO very easily. Metadata tends to be
> dominated by seeks and for many applications it makes sense to put the
> metadata onto faster SSDs.
> 
> 
> 
> 
> This article (almost 2.5 years old) claims one company is already using either
> these patches or something similar:
> 
> https://lwn.net/Articles/698090/
> 
>     August 24, 2016
> 
>     To combat that, he has a set of patches to automatically put the Btrfs
> metadata on SSDs. The block layer provides information on whether the
> storage is rotational; for now, his patch assumes that if
>     it is not rotational then it is fast. The patch has made a huge difference in
> the latencies and requires less flash storage (e.g. 450GB for 40TB filesystem)
> for Facebook's file workload that
>     consists of a wide variety of file sizes.
> 
> 
> 
> Do these patches exist anywhere? I couldn't find them in the list archive.

I managed to find this suggestion about how it could be done:
https://patchwork.kernel.org/patch/9556973/

I would love to use this feature as well. My current setup uses btrfs on lvm with caching enabled, but I would prefer a simpler setup.

Paul.

Re: dedicated metadata drives?

[Eli V]

On Tue, Jan 8, 2019 at 9:43 PM Paul Jones <paul@pauljones.id.au> wrote:
>
> > -----Original Message-----
> > From: linux-btrfs-owner@vger.kernel.org <linux-btrfs-
> > owner@vger.kernel.org> On Behalf Of Tomasz Chmielewski
> > Sent: Saturday, 5 January 2019 12:25 AM
> > To: Btrfs BTRFS <linux-btrfs@vger.kernel.org>
> > Subject: dedicated metadata drives?
> >
> > According to btrfs wiki, some patches have been submitted to support
> > metadata on different devices than data (i.e. metadata on SSD, data on
> > HDD):
> >
> >
> > https://btrfs.wiki.kernel.org/index.php/Project_ideas#Dedicated_metadata
> > _drives
> >
> >     Dedicated metadata drives
> >
> >     Not claimed — submitted — Not in kernel yet
> >
> >     We're able to split data and metadata IO very easily. Metadata tends to be
> > dominated by seeks and for many applications it makes sense to put the
> > metadata onto faster SSDs.
> >
> >
> >
> >
> > This article (almost 2.5 years old) claims one company is already using either
> > these patches or something similar:
> >
> > https://lwn.net/Articles/698090/
> >
> >     August 24, 2016
> >
> >     To combat that, he has a set of patches to automatically put the Btrfs
> > metadata on SSDs. The block layer provides information on whether the
> > storage is rotational; for now, his patch assumes that if
> >     it is not rotational then it is fast. The patch has made a huge difference in
> > the latencies and requires less flash storage (e.g. 450GB for 40TB filesystem)
> > for Facebook's file workload that
> >     consists of a wide variety of file sizes.
> >
> >
> >
> > Do these patches exist anywhere? I couldn't find them in the list archive.
>
> I managed to find this suggestion about how it could be done:
> https://patchwork.kernel.org/patch/9556973/
>
> I would love to use this feature as well. My current setup uses btrfs on lvm with caching enabled, but I would prefer a simpler setup.
>
> Paul.

Another out of tree option are the patches from Timofey Titovets
"Btrfs: enhance raid1/10 balance heuristic"

https://lore.kernel.org/linux-btrfs/20180925183841.26936-1-nefelim4ag@gmail.com/

Then you'd put all your data drives in a single LVM volume, could be
MD raid or what not, and then your SSDs in another lvol. Make your
data single, and your metadata RAID1 and then the SSDs should be
preferentially used for metadata access. I haven't actually tried the
patches yet myself, but have a test system with the proper hardware
setup now running normal btrfs 4.9. Just not enough time in a day to
test everything. The SSD's do make a huge difference for metadata
heavy things like running compsize on an fs.