Confining scrub to a subvolume

Confining scrub to a subvolume

[Sree Harsha Totakura]

Hi,

Is it possible to confine scrubbing to a subvolume instead of the whole
file system?

The problem I am facing is that I have a 5 TB btrfs FS.  On it I have
created subvolumes for weekly backups, personal photos, music, and
documents.  Obviously, I am more concerned about my photos and documents
than my backups and music.  Therefore, I would like to scrub the photos
and documents subvolumes more often than the backups subvolume.  Would
this be possible with the current tools?

Regards,
Sree

Re: Confining scrub to a subvolume

[David Sterba]

On Wed, Dec 30, 2015 at 01:00:34AM +0100, Sree Harsha Totakura wrote:
> Is it possible to confine scrubbing to a subvolume instead of the whole
> file system?

No. Srub reads the blocks from devices (without knowing which files own
them) and compares them to the stored checksums.

> [...]  Therefore, I would like to scrub the photos
> and documents subvolumes more often than the backups subvolume.  Would
> this be possible with the current tools?

The closest would be to read the files and look for any reported errors.

Re: Confining scrub to a subvolume

[Duncan]

David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:

> On Wed, Dec 30, 2015 at 01:00:34AM +0100, Sree Harsha Totakura wrote:
>> Is it possible to confine scrubbing to a subvolume instead of the whole
>> file system?
> 
> No. Srub reads the blocks from devices (without knowing which files own
> them) and compares them to the stored checksums.

Of course if like me you prefer not to have all your data eggs in one 
filesystem basket and have used partitions (or LVM) and multiple 
independent btrfs, in which case you scrub the filesystem you want, and 
don't worry about the others. =:^)

It definitely helps with maintenance time -- on SSDs with all under 50 
GiB partitions, scrub times per btrfs are typically under a minute, and 
btrfs check and balance times are similarly short.  Plus, arranging to 
have additional partitions exactly the same size to use as backups works 
pretty nicely as well.  =:^)  OTOH, people routinely report days for 
multi-terabyte btrfs maintenance commands on spinning rust. =:^(

Tho I do still have my media partition, along with backups, on reiserfs 
on spinning rust.  I should think about switching that over one of these 
days...

>> [...]  Therefore, I would like to scrub the photos and documents
>> subvolumes more often than the backups subvolume.  Would this be
>> possible with the current tools?
> 
> The closest would be to read the files and look for any reported errors.

That should work.  Cat the files to /dev/null and check dmesg.  For 
single mode it should check the only copy.  For raid1/10 or dup, running 
two checks, ensuring one is even-PID while the other is odd-PID, should 
work to check both copies, since the read-scheduler assigns copy based on 
even/odd PID.  Errors will show up in dmesg, as well as cat's STDERR.

Pretty clever thinking there. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Confining scrub to a subvolume

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

On Wed, 2015-12-30 at 18:39 +0100, David Sterba wrote:
> The closest would be to read the files and look for any reported
> errors.
Doesn't that fail for any multi-device setup, in which case btrfs reads
the blocks only from one device, and if that verifies, doesn't check
the other?


Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]

Re: Confining scrub to a subvolume

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]

On Wed, 2015-12-30 at 18:26 +0000, Duncan wrote:
> That should work.  Cat the files to /dev/null and check dmesg.  For 
> single mode it should check the only copy.  For raid1/10 or dup,
> running 
> two checks, ensuring one is even-PID while the other is odd-PID,
> should 
> work to check both copies, since the read-scheduler assigns copy
> based on 
> even/odd PID.  Errors will show up in dmesg, as well as cat's STDERR.
That doesn't seem very reliable to me, to be honest... plus it wouldn't
work in any RAID56 or dupN (with n!=2) case, when that gets sooner or
later implemented.

Also, I'd kinda guess (or better said: hope) that the kernel's cache
would destroy these efforts, at least when the two reads happen mostly
in parallel.


Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]

Re: Confining scrub to a subvolume

[Duncan]

Christoph Anton Mitterer posted on Wed, 30 Dec 2015 20:28:00 +0100 as
excerpted:

> On Wed, 2015-12-30 at 18:26 +0000, Duncan wrote:
>> That should work.  Cat the files to /dev/null and check dmesg.  For
>> single mode it should check the only copy.  For raid1/10 or dup,
>> running two checks, ensuring one is even-PID while the other is
>> odd-PID, should work to check both copies, since the read-scheduler
>> assigns copy based on even/odd PID.  Errors will show up in dmesg, as
>> well as cat's STDERR.

> That doesn't seem very reliable to me, to be honest... plus it wouldn't
> work in any RAID56 or dupN (with n!=2) case, when that gets sooner or
> later implemented.

Well, yes, but right now except on raid56... and there's a good chance 
it'll work for a year at least, as I've seen no first-patches yet to 
implement n-way (which I'm sure looking forward to), after which perhaps 
he'll have implemented the multi-btrfs on partitions or lvm thing that I 
actually prefer, myself.

Meanwhile, it's a pretty clever solution, I think. =:^)

> Also, I'd kinda guess (or better said: hope) that the kernel's cache
> would destroy these efforts, at least when the two reads happen mostly
> in parallel.

I was thinking run them in parallel, but you're right, you'd have to run 
them serially and dumpcache between runs.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Confining scrub to a subvolume

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

On Wed, 2015-12-30 at 20:57 +0000, Duncan wrote:
> Meanwhile, it's a pretty clever solution, I think. =:^)
Well the problem with such workaround-solutions is... end-users get
used to it, rely on it, and suddenly they don't work anymore (which the
user wouldn't probably notice, though).

> I was thinking run them in parallel, but you're right, you'd have to
> run 
> them serially and dumpcache between runs.
And, you'd need to clear the cache, at least for filesystems < memory
but even if not, I wouldn't be 100% that one can safely do without
clearing the cache.

Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]

Re: Confining scrub to a subvolume

[Sree Harsha Totakura]

On 12/30/2015 07:26 PM, Duncan wrote:
> David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:
> 
>> On Wed, Dec 30, 2015 at 01:00:34AM +0100, Sree Harsha Totakura wrote:
>>> Is it possible to confine scrubbing to a subvolume instead of the whole
>>> file system?
>>
>> No. Srub reads the blocks from devices (without knowing which files own
>> them) and compares them to the stored checksums.
> 
> Of course if like me you prefer not to have all your data eggs in one 
> filesystem basket and have used partitions (or LVM) and multiple 
> independent btrfs, in which case you scrub the filesystem you want, and 
> don't worry about the others. =:^)

I considered it, but after reading somewhere (couldn't find the source)
that having a single btrfs could be beneficial, I decided not to.
Clearly, it doesn't seem to be true in this case.

Thank you all for your comments and suggestions.  I will be using
different btrfs file systems based.

Regards,
Sree Harsha Totakura

Re: Confining scrub to a subvolume

[David Sterba]

On Wed, Dec 30, 2015 at 08:26:00PM +0100, Christoph Anton Mitterer wrote:
> On Wed, 2015-12-30 at 18:39 +0100, David Sterba wrote:
> > The closest would be to read the files and look for any reported
> > errors.
> Doesn't that fail for any multi-device setup, in which case btrfs reads
> the blocks only from one device, and if that verifies, doesn't check
> the other?

That's right, it's not equivalent to the scrub.

Re: Confining scrub to a subvolume

[Duncan]

Sree Harsha Totakura posted on Mon, 04 Jan 2016 12:01:58 +0100 as
excerpted:

> On 12/30/2015 07:26 PM, Duncan wrote:
>> David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:
>> 
>>> On Wed, Dec 30, 2015 at 01:00:34AM +0100, Sree Harsha Totakura wrote:
>>>> Is it possible to confine scrubbing to a subvolume instead of the
>>>> whole file system?
>>>
>>> No. Srub reads the blocks from devices (without knowing which files
>>> own them) and compares them to the stored checksums.
>> 
>> Of course if like me you prefer not to have all your data eggs in one
>> filesystem basket and have used partitions (or LVM) and multiple
>> independent btrfs, in which case you scrub the filesystem you want, and
>> don't worry about the others. =:^)
> 
> I considered it, but after reading somewhere (couldn't find the source)
> that having a single btrfs could be beneficial, I decided not to.
> Clearly, it doesn't seem to be true in this case.

It depends very much on your viewpoint and use-case.

Arguably, btrfs should /eventually/ provide a more flexible alternative 
to lvm as a volume manager, letting you do subvolumes, restrict them if 
desired to some maximum size via quotas (which remain buggy and 
unreliable on btrfs ATM so don't try to use them for that yet), and let 
you "magically" add and remove devices from a single btrfs storage pool, 
changing quota settings as desired, without the hassle of managing 
individual partitions and multiple filesystems.

But IMO at least, and I'd guess in the opinion of most on the list, btrfs 
at its present "still stabilizing, not yet fully stable and mature" 
status, remains at least partially unsuited to that.  Besides general 
stability, some options, including quotas, simply don't work reliably 
yet, and other tools and features have yet to be developed.

But even in the future, when btrfs is stable and many are using these 
sorts of logical volume management and integrated filesystem features to 
do everything on a single filesystem, I'm still very likely to consider 
that a higher risk than I'm willing to take, because it'll still be 
ultimately putting all those data eggs in the filesystem basket, which if 
the bottom falls out...

In addition, I actually tried, for instance, big partitioned mdraids, 
with several filesystems each on their own partition of that mdraid, and 
I eventually came to the conclusion that at some point, they simply get 
too big, and maintenance simply takes too long, to be practical for me.  
When adding a multi-TB device to a big mdraid takes days...  I'd much 
rather have multiple much smaller mdraids, or now, btrfs raids, and 
perhaps still take days overall to do the same thing if I'm doing it to 
all of them, but in increments of a few hours each on multiple much 
smaller capacity ones, rather than a single-shot instance that takes days.

Meanwhile, my current btrfs layout is multiple mostly raid1 btrfs, on a 
pair of partitioned SSDs, which each partition under 50 GiB, under 100 
GiB for the raid1 filesystem, 50 on each device.  On that, scrubs 
normally take, literally, under a minute, full balances well under ten, 
per filesystem.  Sure, to do every single filesystem might still take say 
a half hour, but most of the time, not all filesystems are even mounted, 
and most of the time, I only need to scrub or balance perhaps three of 
them, so while if they were all in one I might do it in say 20 minutes, 
and if I had to do all of them it might take me 30 because I have to 
repeatedly type in the command for each one, because I have to do only 
three of them, it's done in five minutes or less.

That's in addition to the fact that if a filesystem dies, I've only a 
fraction of the data to btrfs restore or to copy over from backup, 
because most of it was on other filesystems, many of which weren't even 
mounted or in some cases (my /) were mounted read-only, so they're just 
fine and I don't have to btrfs restore or copy back over from backup, for 
them.

These are lessons I've learned in a quarter century of working with 
computers, about a decade on MS, a decade and a half later this year, on 
Linux.  They may not always apply to everyone, but I've definitely 
learned how to spare myself unnecessary pain, as I've learned how they 
apply to me. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman