Honest timeline for btrfsck

Honest timeline for btrfsck

[Erik Jensen]

The lack of any information on when btrfsck might be ready is a real
headache to those deciding what to do with a corrupted file system.

I am currently sitting on a btrfs array of 10 disks that has been
reporting "parent transid verify failed" since last November. While
the data on the drive is by no means irreplaceable, it would take a
fair amount of effort. At the time I was told that a btrfsck would
almost certainly be released by the end of the year. In January, it
was "finally almost ready", and toward the end of May it was going to
be released in "a couple of days
(hopefully)".

Had I known back in November 9 months would go by with no such tool, I
would have certainly wiped the array and started over, as it was
certainly not worth the wait. So here I am, several assurances of
imminent release later, still wondering whether it would be better to
wait or cut my losses.

I understand that everyone is working hard, and I deeply appreciate
the effort being put into this filesystem. I'm not looking for an
exact date, just a rough order of magnitude on which to base
decisions.

Thank you very much.

--Erik Jensen

Re: Honest timeline for btrfsck

[Jan Schmidt]

On 03.08.2011 08:57, Erik Jensen wrote:
> Had I known back in November 9 months would go by with no such tool, I
> would have certainly wiped the array and started over, as it was
> certainly not worth the wait. So here I am, several assurances of
> imminent release later, still wondering whether it would be better to
> wait or cut my losses.

If you want to try a patch that might give you read-only access to your
data, have a look at that one:

> Date:	Thu, 23 Jun 2011 15:54:09 -0400
> From:	Josef Bacik <josef@redhat.com>
> To:	Chris Mason <chris.mason@oracle.com>
> Cc:	Andrej Podzimek <andrej@podzimek.org>,
> 	Josef Bacik <josef@redhat.com>,
> 	linux-btrfs <linux-btrfs@vger.kernel.org>
> Subject: Re: parent transid verify failures on 2.6.39
> Message-ID: <20110623195409.GA21007@dhcp231-156.rdu.redhat.com>

-Jan

Re: Honest timeline for btrfsck

[Chris Mason]

Excerpts from Erik Jensen's message of 2011-08-03 02:57:24 -0400:
> The lack of any information on when btrfsck might be ready is a real
> headache to those deciding what to do with a corrupted file system.
> 
> I am currently sitting on a btrfs array of 10 disks that has been
> reporting "parent transid verify failed" since last November. While
> the data on the drive is by no means irreplaceable, it would take a
> fair amount of effort. At the time I was told that a btrfsck would
> almost certainly be released by the end of the year. In January, it
> was "finally almost ready", and toward the end of May it was going to
> be released in "a couple of days
> (hopefully)".
> 
> Had I known back in November 9 months would go by with no such tool, I
> would have certainly wiped the array and started over, as it was
> certainly not worth the wait. So here I am, several assurances of
> imminent release later, still wondering whether it would be better to
> wait or cut my losses.
> 
> I understand that everyone is working hard, and I deeply appreciate
> the effort being put into this filesystem. I'm not looking for an
> exact date, just a rough order of magnitude on which to base
> decisions.

This part is definitely my fault.  I've gone through a bunch of
variations on bigger and smaller tools, and had to juggle the kernel
maintenance as well.

Aside from making sure the kernel code is stable, btrfsck is all I'm
working on right now.  I do expect a release in the next two weeks that
can recover your data (and many others).

Thanks,
Chris

Re: Honest timeline for btrfsck

[Francesco Riosa]

2011/8/3 Chris Mason <chris.mason@oracle.com>:
> Excerpts from Erik Jensen's message of 2011-08-03 02:57:24 -0400:
>> The lack of any information on when btrfsck might be ready is a real
>> headache to those deciding what to do with a corrupted file system.
>>
>> I am currently sitting on a btrfs array of 10 disks that has been
>> reporting "parent transid verify failed" since last November. While
>> the data on the drive is by no means irreplaceable, it would take a
>> fair amount of effort. At the time I was told that a btrfsck would
>> almost certainly be released by the end of the year. In January, it
>> was "finally almost ready", and toward the end of May it was going t=
o
>> be released in "a couple of days
>> (hopefully)".
>>
>> Had I known back in November 9 months would go by with no such tool,=
 I
>> would have certainly wiped the array and started over, as it was
>> certainly not worth the wait. So here I am, several assurances of
>> imminent release later, still wondering whether it would be better t=
o
>> wait or cut my losses.
>>
>> I understand that everyone is working hard, and I deeply appreciate
>> the effort being put into this filesystem. I'm not looking for an
>> exact date, just a rough order of magnitude on which to base
>> decisions.
>
> This part is definitely my fault. =A0I've gone through a bunch of
> variations on bigger and smaller tools, and had to juggle the kernel
> maintenance as well.
>
> Aside from making sure the kernel code is stable, btrfsck is all I'm
> working on right now. =A0I do expect a release in the next two weeks =
that
> can recover your data (and many others).
>
> Thanks,
> Chris

sleep 86400 ;\
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-pro=
gs-unstable.git
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Dave]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

On Wed, Aug 03, 2011 at 04:53:26PM -0400, Chris Mason wrote:
> I do expect a release in the next two weeks that can recover your data (and
> many others).

I actually set an appointment reminder in my Blackberry for the two week
anniversary of this email.  I expect today will be a milestone in the btrfs
ecosystem (lack of fsck being the most frequent reason for not trying btrfs).  I
know I've got two existing instances that I can test this tool on.
-- 
-=[dave]=-

Entropy isn't what it used to be.

[-- Attachment #2: Type: application/pgp-signature, Size: 230 bytes --]

Re: Honest timeline for btrfsck

[Yalonda Gishtaka]

Chris Mason <chris.mason <at> oracle.com> writes:

> 
> Aside from making sure the kernel code is stable, btrfsck is all I'm
> working on right now.  I do expect a release in the next two weeks that
> can recover your data (and many others).
> 
> Thanks,
> Chris
> --


Chris,

We're all on the edge of our seats.  Can you provide an updated ETA on the 
release of the first functional btrfsck tool?  No pressure or anything ;)

-Yalonda

Re: Honest timeline for btrfsck

[Chris Mason]

Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400:
> Chris Mason <chris.mason <at> oracle.com> writes:
> 
> > 
> > Aside from making sure the kernel code is stable, btrfsck is all I'm
> > working on right now.  I do expect a release in the next two weeks that
> > can recover your data (and many others).
> > 
> > Thanks,
> > Chris
> > --
> 
> 
> Chris,
> 
> We're all on the edge of our seats.  Can you provide an updated ETA on the 
> release of the first functional btrfsck tool?  No pressure or anything ;)

Hi everyone,

I've been working non-stop on this.  Currently fsck has four parts:

1) mount -o recovery mode.  I've posted smaller forms of these patches
in the past that bypass log tree replay.  The new versions have code to
create stub roots for trees that can't be read (like the extent
allocation tree) and will allow the mount to proceed.

2) fsck that scans for older roots.  This takes advantage of older
copies of metadata to look for consistent tree roots on disk.  The
downside is that it is currently very slow.  I'm trying to speed it up
by limiting the search to only the metadata block groups and a few other
tricks.

3) fsck that fixes the extent allocation tree and the chunk tree.  This
is where I've been spending most of my time.  The problem is that it
tends to recover some filesystems and badly break others.  While I'm
fixing up the corner cases that work poorly, I'm adding an undo log to
the fsck code so that you can get the FS back into its original state if
you don't like the result of the fsck.

4) The rest of the corruptions can be dealt with fairly well from the
kernel.  I have a series of patches to make the extent allocation tree
less strict about reference counts and other rules, basically allowing
the FS to limp along instead of crash.

These four things together are basically my minimal set of features
required for fedora and our own internal projects at Oracle to start
treating us as production filesystem.

There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
hoped to get #1 out the door before I left on vacation and I still might
post it tonight.

-chris

Re: Honest timeline for btrfsck

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3039 bytes --]

On Thu, Aug 18, 2011 at 04:50:08PM -0400, Chris Mason wrote:
> I've been working non-stop on this.  Currently fsck has four parts:

   This all looks like great stuff. Can't wait to try it out...

   One thing strikes me for purposes of automated testing and
regression testing: Do you have tools or techniques for breaking a
filesystem in specific ways?

> 1) mount -o recovery mode.  I've posted smaller forms of these patches
> in the past that bypass log tree replay.  The new versions have code to
> create stub roots for trees that can't be read (like the extent
> allocation tree) and will allow the mount to proceed.

   I can see that this will deal with some kinds of breakage, like the
log tree being missing, but most of the other trees are kind of
important for minor things like finding your data. :)

   How useful or reliable is it to ignore missing trees that aren't
the log tree? I'd have thought that if you were missing one of the 6
main trees, you'd have a pretty much unreadable FS.

> 2) fsck that scans for older roots.  This takes advantage of older
> copies of metadata to look for consistent tree roots on disk.  The
> downside is that it is currently very slow.  I'm trying to speed it up
> by limiting the search to only the metadata block groups and a few other
> tricks.

   If this is in decent shape, it's probably worth it to release it in
its current form anyway (and possibly request a moratorium on extra
patches until you've finished the optimisation). I suspect that
there's a number of people out there who wouldn't mind the speed
issues just to get a filesystem back.

> 3) fsck that fixes the extent allocation tree and the chunk tree.  This
> is where I've been spending most of my time.  The problem is that it
> tends to recover some filesystems and badly break others.  While I'm
> fixing up the corner cases that work poorly, I'm adding an undo log to
> the fsck code so that you can get the FS back into its original state if
> you don't like the result of the fsck.

> 4) The rest of the corruptions can be dealt with fairly well from the
> kernel.  I have a series of patches to make the extent allocation tree
> less strict about reference counts and other rules, basically allowing
> the FS to limp along instead of crash.

   Is that going to be always-on, with stubs to highlight where
subsequent patches can add the requisite healing code in later
revisions, or as a mount flag like -o recovery?

> These four things together are basically my minimal set of features
> required for fedora and our own internal projects at Oracle to start
> treating us as production filesystem.
> 
> There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> hoped to get #1 out the door before I left on vacation and I still might
> post it tonight.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- "You know,  the British have always been nice to mad people." ---  

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Honest timeline for btrfsck

[Maciej Marcin Piechotka]

[-- Attachment #1: Type: text/plain, Size: 2583 bytes --]

On Thu, 2011-08-18 at 16:50 -0400, Chris Mason wrote:
> Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400:
> > Chris Mason <chris.mason <at> oracle.com> writes:
> > 
> > > 
> > > Aside from making sure the kernel code is stable, btrfsck is all I'm
> > > working on right now.  I do expect a release in the next two weeks that
> > > can recover your data (and many others).
> > > 
> > > Thanks,
> > > Chris
> > > --
> > 
> > 
> > Chris,
> > 
> > We're all on the edge of our seats.  Can you provide an updated ETA on the 
> > release of the first functional btrfsck tool?  No pressure or anything ;)
> 
> Hi everyone,
> 
> I've been working non-stop on this.  Currently fsck has four parts:
> 
> 1) mount -o recovery mode.  I've posted smaller forms of these patches
> in the past that bypass log tree replay.  The new versions have code to
> create stub roots for trees that can't be read (like the extent
> allocation tree) and will allow the mount to proceed.
> 
> 2) fsck that scans for older roots.  This takes advantage of older
> copies of metadata to look for consistent tree roots on disk.  The
> downside is that it is currently very slow.  I'm trying to speed it up
> by limiting the search to only the metadata block groups and a few other
> tricks.
> 
> 3) fsck that fixes the extent allocation tree and the chunk tree.  This
> is where I've been spending most of my time.  The problem is that it
> tends to recover some filesystems and badly break others.  While I'm
> fixing up the corner cases that work poorly, I'm adding an undo log to
> the fsck code so that you can get the FS back into its original state if
> you don't like the result of the fsck.
> 
> 4) The rest of the corruptions can be dealt with fairly well from the
> kernel.  I have a series of patches to make the extent allocation tree
> less strict about reference counts and other rules, basically allowing
> the FS to limp along instead of crash.
> 
> These four things together are basically my minimal set of features
> required for fedora and our own internal projects at Oracle to start
> treating us as production filesystem.
> 
> There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> hoped to get #1 out the door before I left on vacation and I still might
> post it tonight.

Greate to hear that. Given that I get corruption every 2 week I would
like to at least test the tools - are there available anywhere?

I'd like to see #2 (it seems to be able to fix my main crashes).

Best regards


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Honest timeline for btrfsck

[Michael Cronenworth]

Chris Mason wrote:
> There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> hoped to get #1 out the door before I left on vacation and I still might
> post it tonight.

Hi Chris,

Is there an update on this? I don't see any new code for 
btrfs-progs-unstable, but I might be looking in the wrong place.

Will this fsck tool be able to handle problems such as:
"parent transid verify failed on # wanted # found #" ?

Thanks,
Michael

Re: Honest timeline for btrfsck

[Yalonda Gishtaka]

Hugo Mills <hugo <at> carfax.org.uk> writes:
> 
>    If this is in decent shape, it's probably worth it to release it in
> its current form anyway (and possibly request a moratorium on extra
> patches until you've finished the optimisation). I suspect that
> there's a number of people out there who wouldn't mind the speed
> issues just to get a filesystem back.

I second this.  I'd rather have a slow btrfsck tool than just the promise of one
that rivals the Duke Nukem Forever release schedule.

Re: Honest timeline for btrfsck

[Michael Cronenworth]

On 08/25/2011 10:06 AM, Michael Cronenworth wrote:
>
> Is there an update on this? I don't see any new code for 
> btrfs-progs-unstable, but I might be looking in the wrong place.
>
> Will this fsck tool be able to handle problems such as:
> "parent transid verify failed on # wanted # found #" ? 

Does anyone have an update on this? I haven't seen any news for several 
weeks now.

Re: Honest timeline for btrfsck

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

On Thu, Sep 01, 2011 at 02:14:00PM -0500, Michael Cronenworth wrote:
> On 08/25/2011 10:06 AM, Michael Cronenworth wrote:
> >
> >Is there an update on this? I don't see any new code for
> >btrfs-progs-unstable, but I might be looking in the wrong place.
> >
> >Will this fsck tool be able to handle problems such as:
> >"parent transid verify failed on # wanted # found #" ?
> 
> Does anyone have an update on this? I haven't seen any news for
> several weeks now.

   As a reply to your question:

On Thu, Aug 18, 2011 at 04:50:08PM -0400, Chris Mason wrote:
> There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> hoped to get #1 out the door before I left on vacation and I still might
> post it tonight.

   You may have missed the "on vacation" bit.

   The canonical place to look for btrfsck updates is the relevant FAQ
item on the btrfs wiki. When Chris has a released version of the
recovery tools, that FAQ entry will be updated at least as soon as I
read it (service may be a bit slow between 00:00 and 09:00 GMT, but
I'm sure others will step in to do the update in that instance). If
you don't see anything new there, there's nothing new to report.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- "No!  My collection of rare, incurable diseases! Violated!" ---   

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Honest timeline for btrfsck

[Michael Cronenworth]

On 09/01/2011 03:20 PM, Hugo Mills wrote:
>     You may have missed the "on vacation" bit.

I did read the "on vacation" bit. Not that it is any of my business, but 
how long is that vacation?

>     The canonical place to look for btrfsck updates is the relevant FAQ
> item on the btrfs wiki.

I know this. No need to repeat it. I'm not a complete idiot.

Re: Honest timeline for btrfsck

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

On Thu, Sep 01, 2011 at 03:24:28PM -0500, Michael Cronenworth wrote:
> On 09/01/2011 03:20 PM, Hugo Mills wrote:
> >    You may have missed the "on vacation" bit.
> 
> I did read the "on vacation" bit. Not that it is any of my business,
> but how long is that vacation?

   Your guess is as good as mine. It's only been two weeks...

> >    The canonical place to look for btrfsck updates is the relevant FAQ
> >item on the btrfs wiki.
> 
> I know this. No need to repeat it. I'm not a complete idiot.

   I never claimed you were. Many people either don't realise that
that is the right place, or don't realise that it really is updated
pretty much as soon as possible with all the information that's been
made public on the subject.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- "No!  My collection of rare, incurable diseases! Violated!" ---   

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Honest timeline for btrfsck

[Yalonda Gishtaka]

Queue the Jeopardy music.  A couple of weeks, pfft!

> Michael Cronenworth wrote:
> 
> Does anyone have an update on this? I haven't seen any news for several 
> weeks now.

Re: Honest timeline for btrfsck

[Martin Steigerwald]

Am Donnerstag, 1. September 2011 schrieb Hugo Mills:
> On Thu, Sep 01, 2011 at 03:24:28PM -0500, Michael Cronenworth wrote:
> > On 09/01/2011 03:20 PM, Hugo Mills wrote:
> > >    You may have missed the "on vacation" bit.
> >=20
> > I did read the "on vacation" bit. Not that it is any of my business=
,
> > but how long is that vacation?
>=20
>    Your guess is as good as mine. It's only been two weeks...
>=20
> > >    The canonical place to look for btrfsck updates is the relevan=
t
> > >    FAQ
> > >
> > >item on the btrfs wiki.
> >=20
> > I know this. No need to repeat it. I'm not a complete idiot.
>=20
>    I never claimed you were. Many people either don't realise that
> that is the right place, or don't realise that it really is updated
> pretty much as soon as possible with all the information that's been
> made public on the subject.

Well, I do think that a release of the fsck for BTRFS is important enou=
gh=20
to announce it on this mailinglist, too ;).

=46ortunately I didn=B4t have any major problems so far. I once had to =
use=20
btrfs-zero-log. And I am still not using BTRFS for my main machine=B4s =
home=20
directory. For this I intend to wait till it is marked stable in kernel=
=20
sources + fsck available.

--=20
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

Isn't it about time to make some hard decisions about btrfsck?  Three
years is enough time to go without this type of functionality in a
modern filesystem, especially given btrfs's fragility in the face of
power failures.

Given the lack of progress, and the inability to provide remotely
realistic time lines, I'd say it is inappropriate for development to
remain in secret.  I am aware of the type of noise that could be
created by releasing this code out to the public, but that noise will
help drive fixes and development of the tool.  We have seen months
worth of wasted effort spent on stop gap measures that could have been
funneled into the real utility, if people just had access to the code.

If we can't get btrfsck checked into btrfs-progs-unstable, should we
just cut our losses and start a recovery tool from scratch.  There was
a btrfs_rescue utility mentioned at
http://article.gmane.org/gmane.comp.file-systems.btrfs/8309/match=3Dbtr=
fs_rescue,
but the source doesn't seem to be available anymore.

I had started to work on my own repair tool/script/hack.  I was
looking into some of the current btrfs progs code, and the biggest
issue I had was all of the data validation that disk_io.c and other
code does that makes sure the FS is in a good state before you can do
anything with it.  This makes those utility methods pretty much
useless for exploring a broken FS.  So, understanding how to
incrementally read the FS without doing all of the validations
beforehand was the biggest hurdle I encountered.


On Sat, Sep 10, 2011 at 5:09 AM, Martin Steigerwald <Martin@lichtvoll.d=
e> wrote:
> Am Donnerstag, 1. September 2011 schrieb Hugo Mills:
>> On Thu, Sep 01, 2011 at 03:24:28PM -0500, Michael Cronenworth wrote:
>> > On 09/01/2011 03:20 PM, Hugo Mills wrote:
>> > > =A0 =A0You may have missed the "on vacation" bit.
>> >
>> > I did read the "on vacation" bit. Not that it is any of my busines=
s,
>> > but how long is that vacation?
>>
>> =A0 =A0Your guess is as good as mine. It's only been two weeks...
>>
>> > > =A0 =A0The canonical place to look for btrfsck updates is the re=
levant
>> > > =A0 =A0FAQ
>> > >
>> > >item on the btrfs wiki.
>> >
>> > I know this. No need to repeat it. I'm not a complete idiot.
>>
>> =A0 =A0I never claimed you were. Many people either don't realise th=
at
>> that is the right place, or don't realise that it really is updated
>> pretty much as soon as possible with all the information that's been
>> made public on the subject.
>
> Well, I do think that a release of the fsck for BTRFS is important en=
ough
> to announce it on this mailinglist, too ;).
>
> Fortunately I didn=B4t have any major problems so far. I once had to =
use
> btrfs-zero-log. And I am still not using BTRFS for my main machine=B4=
s home
> directory. For this I intend to wait till it is marked stable in kern=
el
> sources + fsck available.
>
> --
> Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
> GPG: 03B0 0D6C 0040 0710 4AFA =A0B82F 991B EAAC A599 84C7
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Erik Jensen]

Chris,

Now that you're back from vacation, I was wondering if you would be
able to provide a revised estimate on how long this will take. Also,
of the four parts, which will be necessary to correct a 'parent
transid verify failed' error?

Thank you for your time,
--Erik

On Thu, Aug 18, 2011 at 1:50 PM, Chris Mason <chris.mason@oracle.com> w=
rote:
> Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400=
:
>> Chris Mason <chris.mason <at> oracle.com> writes:
>>
>> >
>> > Aside from making sure the kernel code is stable, btrfsck is all I=
'm
>> > working on right now. =A0I do expect a release in the next two wee=
ks that
>> > can recover your data (and many others).
>> >
>> > Thanks,
>> > Chris
>> > --
>>
>>
>> Chris,
>>
>> We're all on the edge of our seats. =A0Can you provide an updated ET=
A on the
>> release of the first functional btrfsck tool? =A0No pressure or anyt=
hing ;)
>
> Hi everyone,
>
> I've been working non-stop on this. =A0Currently fsck has four parts:
>
> 1) mount -o recovery mode. =A0I've posted smaller forms of these patc=
hes
> in the past that bypass log tree replay. =A0The new versions have cod=
e to
> create stub roots for trees that can't be read (like the extent
> allocation tree) and will allow the mount to proceed.
>
> 2) fsck that scans for older roots. =A0This takes advantage of older
> copies of metadata to look for consistent tree roots on disk. =A0The
> downside is that it is currently very slow. =A0I'm trying to speed it=
 up
> by limiting the search to only the metadata block groups and a few ot=
her
> tricks.
>
> 3) fsck that fixes the extent allocation tree and the chunk tree. =A0=
This
> is where I've been spending most of my time. =A0The problem is that i=
t
> tends to recover some filesystems and badly break others. =A0While I'=
m
> fixing up the corner cases that work poorly, I'm adding an undo log t=
o
> the fsck code so that you can get the FS back into its original state=
 if
> you don't like the result of the fsck.
>
> 4) The rest of the corruptions can be dealt with fairly well from the
> kernel. =A0I have a series of patches to make the extent allocation t=
ree
> less strict about reference counts and other rules, basically allowin=
g
> the FS to limp along instead of crash.
>
> These four things together are basically my minimal set of features
> required for fedora and our own internal projects at Oracle to start
> treating us as production filesystem.
>
> There are always bugs to fix, and I have #1 and #2 mostly ready. =A0I=
 had
> hoped to get #1 out the door before I left on vacation and I still mi=
ght
> post it tonight.
>
> -chris
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

On Thu, Aug 18, 2011 at 3:50 PM, Chris Mason <chris.mason@oracle.com> w=
rote:
>=A0I had hoped to get #1 out the door before I left on vacation and I =
still might
> post it tonight.
>
> -chris

I don't think this is the honest time line that people were looking
for.  Instead of playing more guessing games, how about we just get
that source checked in so we can get more eyeballs on it.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Clemens Eisserer]

+1

2011/9/27 Jeff Putney <jeffrey.putney@gmail.com>:
> On Thu, Aug 18, 2011 at 3:50 PM, Chris Mason <chris.mason@oracle.com>=
 wrote:
>>=C2=A0I had hoped to get #1 out the door before I left on vacation an=
d I still might
>> post it tonight.
>>
>> -chris
>
> I don't think this is the honest time line that people were looking
> for. =C2=A0Instead of playing more guessing games, how about we just =
get
> that source checked in so we can get more eyeballs on it.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht=
ml
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

  That 2 week time line has now reached the 9 week mark.  The only
update anyone has seen was 7 weeks ago, with a 'maybe today'.
  Isn't it time to get that code checked in so someone else can take
over, and not have to start from scratch?  Even if there isn't any
actual working code, having any failed attempts in source control
would still be instructive to whoever takes over delivering an actual
fsck tool.


On Tue, Sep 27, 2011 at 1:00 PM, Clemens Eisserer <linuxhippy@gmail.com=
> wrote:
> +1
>
> 2011/9/27 Jeff Putney <jeffrey.putney@gmail.com>:
>> On Thu, Aug 18, 2011 at 3:50 PM, Chris Mason <chris.mason@oracle.com=
> wrote:
>>>=A0I had hoped to get #1 out the door before I left on vacation and =
I still might
>>> post it tonight.
>>>
>>> -chris
>>
>> I don't think this is the honest time line that people were looking
>> for. =A0Instead of playing more guessing games, how about we just ge=
t
>> that source checked in so we can get more eyeballs on it.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrf=
s" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Chris Mason]

On Tue, Sep 13, 2011 at 01:01:00PM -0500, Jeff Putney wrote:
> Isn't it about time to make some hard decisions about btrfsck?  Three
> years is enough time to go without this type of functionality in a
> modern filesystem, especially given btrfs's fragility in the face of
> power failures.

So this criticism is well deserved.  I'm juggling a ton of btrfs todos
and not doing a great job at communicating the current status of things.

Currently we have a pretty big queue of changes that I'm integrating for
3.2, and I had to delay btrfsck again so that I could start testing
things.  The merge window is pretty short, and there are some
fantastic changes that deserve to go in.

Inside of Oracle, we've decided to make btrfs the default filesystem for
Oracle Linux.  This is going into beta now and we'll increase our usage
of btrfs in production over the next four to six months.  This is a
really big step forward, but it doesn't cover btrfs in database
workloads (since we recommend asm for that outside of the filesystem).

What this means is that absolutely cannot move forward without btrfsck.
RH, Fujitsu, SUSE and others have spent a huge amount of time on the filesystem
and it is clearly time to start putting it into customer hands.

So over the next two weeks I'm juggling the merge window and the fsck
release.  My goal is to demo fsck at linuxcon europe.  Thanks again for
all of your patience and help with Btrfs!

-chris

Re: Honest timeline for btrfsck

[Jeff Putney]

=46urther adoption and more commitment from Oracle for production use i=
s
good news.  The fact that adoption is happening without a working fsck
seems to indicate that folks have given up waiting for it.

Not hearing anything about getting the source into the repositories is
terrible news by omission.  It seems like doubling down on maintaining
a single point of failure.  Having a firm date of when the source
repository will be released regardless of its condition, or
functionality, would go a long way to appeasing the concerns within
the user base.



On Wed, Oct 5, 2011 at 1:16 AM, Chris Mason <chris.mason@oracle.com> wr=
ote:
> On Tue, Sep 13, 2011 at 01:01:00PM -0500, Jeff Putney wrote:
>> Isn't it about time to make some hard decisions about btrfsck? =A0Th=
ree
>> years is enough time to go without this type of functionality in a
>> modern filesystem, especially given btrfs's fragility in the face of
>> power failures.
>
> So this criticism is well deserved. =A0I'm juggling a ton of btrfs to=
dos
> and not doing a great job at communicating the current status of thin=
gs.
>
> Currently we have a pretty big queue of changes that I'm integrating =
for
> 3.2, and I had to delay btrfsck again so that I could start testing
> things. =A0The merge window is pretty short, and there are some
> fantastic changes that deserve to go in.
>
> Inside of Oracle, we've decided to make btrfs the default filesystem =
for
> Oracle Linux. =A0This is going into beta now and we'll increase our u=
sage
> of btrfs in production over the next four to six months. =A0This is a
> really big step forward, but it doesn't cover btrfs in database
> workloads (since we recommend asm for that outside of the filesystem)=
=2E
>
> What this means is that absolutely cannot move forward without btrfsc=
k.
> RH, Fujitsu, SUSE and others have spent a huge amount of time on the =
filesystem
> and it is clearly time to start putting it into customer hands.
>
> So over the next two weeks I'm juggling the merge window and the fsck
> release. =A0My goal is to demo fsck at linuxcon europe. =A0Thanks aga=
in for
> all of your patience and help with Btrfs!
>
> -chris
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Chris Mason]

On Wed, Oct 05, 2011 at 08:59:47AM -0500, Jeff Putney wrote:
> Further adoption and more commitment from Oracle for production use is
> good news.  The fact that adoption is happening without a working fsck
> seems to indicate that folks have given up waiting for it.

No, in this case it means we're confident it will get rolled out.

> 
> Not hearing anything about getting the source into the repositories is
> terrible news by omission.  It seems like doubling down on maintaining
> a single point of failure.  Having a firm date of when the source
> repository will be released regardless of its condition, or
> functionality, would go a long way to appeasing the concerns within
> the user base.

I've given a number of hard dates recently and I'd prefer to show up
with the code instead.  I don't think it makes sense to put a partial
implementation out there, we'll just have a bunch of people reporting
problems that I know exist.

-chris

Re: Honest timeline for btrfsck

[Jeff Putney]

> No, in this case it means we're confident it will get rolled out.

On Aug 18th confidence was high enough to declare a possible release
that very day.  This confidence turned into 7 weeks of silence
followed by another 2 week estimate.

These confident declarations are why things like mniederle's
btrfs_rescue are considered 'interim' and not worth building on.  Had
this confidence of imminent release not been the prevalent message for
the last year, others would have stepped in to fill the void.

> I've given a number of hard dates recently and I'd prefer to show up
> with the code instead. =A0I don't think it makes sense to put a parti=
al
> implementation out there, we'll just have a bunch of people reporting
> problems that I know exist.
>
> -chris
>

This strategy of 'Lone Wolfing it' has delayed the release by a year.
Either you are flying solo because you think that you can make more
meaningful progress without the involvement of the btrfs community, or
you are willing to forfeit the contributions of the community in order
to not have to listen to any complaints.

The other problem of this flying solo plan, is that you are making the
assumption that the problems you know about are more significant than
the problems you are unaware of and could be flushed out with more
eyes on the code.  The longer you delay the release of the source, the
longer it will be until confidence can be generated that major issues
have been resolved.

http://en.wikipedia.org/wiki/Release_early,_release_often
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Andi Kleen]

Jeff Putney <jeffrey.putney@gmail.com> writes:
>
> http://en.wikipedia.org/wiki/Release_early,_release_often

Well the other principle in free software you're forgetting 
is:

"It will be released when it's ready"

If you don't like Chris' ways to do releases you're free to write
something on your own or pay someone to do so. Otherwise
you just have to deal with his time frames, as shifty
as they may be.

-Andi
-- 
ak@linux.intel.com -- Speaking for myself only

Re: Honest timeline for btrfsck

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/06/2011 04:30 PM, Andi Kleen wrote:
> Jeff Putney <jeffrey.putney@gmail.com> writes:
>> 
>> http://en.wikipedia.org/wiki/Release_early,_release_often
> 
> Well the other principle in free software you're forgetting is:
> 
> "It will be released when it's ready"
> 
> If you don't like Chris' ways to do releases you're free to write 
> something on your own or pay someone to do so. Otherwise you just
> have to deal with his time frames, as shifty as they may be.

Thanks, I was about to say the same thing.

- -Jeff

- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6OEK8ACgkQLPWxlyuTD7JgKwCfYqyslTkbq/sYUz/rcXj4M1lf
mTAAoIbIKNIZlyVFZjzrCH/ss9W3UQuh
=6vdd
-----END PGP SIGNATURE-----

Re: Honest timeline for btrfsck

[Randy Barlow]

[-- Attachment #1: Type: text/plain, Size: 1474 bytes --]

On 10/06/2011 11:31 AM, Jeff Putney wrote:
> http://en.wikipedia.org/wiki/Release_early,_release_often

I can appreciate both Jeff's and Andi's positions on this issue. I do
wonder why the fsck isn't publicly available as it is as a non-release
version, just so people can begin getting their eyes on it and make
contributions. I think that would really help with getting a higher
quality product in less time, which is a good goal to attempt to
achieve. I've only played with btrfs at this point, and I'm mostly
waiting for an fsck tool to exist (in a mature form) before using this
fine filesystem on any of my systems, so I am interested in seeing the
fsck reach maturity as I am very excited about all the features that
btrfs offers.

That said, I also think that we ought not to complain to Chris when he
is doing work that will benefit us all, without any cost to us. We may
prefer that he take a different approach in developing this tool, but in
the end he is serving us and we ought not to look a gift horse in the
mouth, as they say.

Chris, I respectfully request that the code you have be placed into a
public repository. It is your choice of course, but I believe it would
be a good thing for btrfs. However and whenever it is delivered to the
community, I am confident that btrfs will be ready for production use
very soon. Thanks to you and all the devs for working so hard to bring
Linux into the future of filesystems!

-- 
R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Honest timeline for btrfsck

[Francesco Riosa]

2011/10/6 Andi Kleen <andi@firstfloor.org>:
> Jeff Putney <jeffrey.putney@gmail.com> writes:
>>
>> http://en.wikipedia.org/wiki/Release_early,_release_often
>
> Well the other principle in free software you're forgetting
> is:
>
> "It will be released when it's ready"
>
> If you don't like Chris' ways to do releases you're free to write
> something on your own or pay someone to do so. Otherwise
> you just have to deal with his time frames, as shifty
> as they may be.

I did a different thing, I've offered Chris money to help rescue an
hosed btrfs or to point to someone who could do, we ended in doing
some tests (for free) but nothing else materialized.
While the time passed has diminished the value of the data to be
rescued I'm more on the "show us some code we can start from" than "it
will be released when ready" vagon.

Francesco R.

>
> -Andi
> --
> ak@linux.intel.com -- Speaking for myself only

Re: Honest timeline for btrfsck

[Yalonda Gishtaka]

Jeff Putney <jeffrey.putney <at> gmail.com> writes:
> This strategy of 'Lone Wolfing it' has delayed the release by a year.
> Either you are flying solo because you think that you can make more
> meaningful progress without the involvement of the btrfs community, or
> you are willing to forfeit the contributions of the community in order
> to not have to listen to any complaints.
> 

Couldn't have put it better.  It's really time for Chris Mason to stop
disgracing the open source community and tarnishing Oracle's name. 

Re: Honest timeline for btrfsck

[Chris Samuel]

On 07/10/11 10:20, Yalonda Gishtaka wrote:

> Couldn't have put it better.  It's really time for Chris Mason
> to stop disgracing the open source community and tarnishing
> Oracle's name. 

Oh come on - he's working *for* Oracle to do this and we are
getting the benefits for free.  We can hardly complain when
he's trying to deal with LKML, doing btrfs devel for Oracle
and having a life as well (i.e. his recent vacation).  I've
known too many people burn out in IT due to overcommitment
and I don't want to see that happen to Chris.

If you wish to direct his priorities then I suggest that you
should be paying Oracle to do so, or else attempt to employ
him yourself.

All the best,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: Honest timeline for btrfsck

[Chester]

On Thu, Oct 6, 2011 at 10:31 AM, Jeff Putney <jeffrey.putney@gmail.com>=
 wrote:
>> No, in this case it means we're confident it will get rolled out.
>
> On Aug 18th confidence was high enough to declare a possible release
> that very day. =A0This confidence turned into 7 weeks of silence
> followed by another 2 week estimate.
>
> These confident declarations are why things like mniederle's
> btrfs_rescue are considered 'interim' and not worth building on. =A0H=
ad
> this confidence of imminent release not been the prevalent message fo=
r
> the last year, others would have stepped in to fill the void.
>
>> I've given a number of hard dates recently and I'd prefer to show up
>> with the code instead. =A0I don't think it makes sense to put a part=
ial
>> implementation out there, we'll just have a bunch of people reportin=
g
>> problems that I know exist.
>>
>> -chris
>>
>
> This strategy of 'Lone Wolfing it' has delayed the release by a year.
> Either you are flying solo because you think that you can make more
> meaningful progress without the involvement of the btrfs community, o=
r
> you are willing to forfeit the contributions of the community in orde=
r
> to not have to listen to any complaints.
>
> The other problem of this flying solo plan, is that you are making th=
e
> assumption that the problems you know about are more significant than
> the problems you are unaware of and could be flushed out with more
> eyes on the code. =A0The longer you delay the release of the source, =
the
> longer it will be until confidence can be generated that major issues
> have been resolved.
>
> http://en.wikipedia.org/wiki/Release_early,_release_often
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>

The problem with this is that people naturally look for a fsck tool
when something bad goes wrong. Something as important as a fsck
utility shouldn't be released (unofficially or officially) half baked.
It can irreparably destroy a filesystem which could've otherwise been
repaired with a fully functional fsck.

I think Chris is trying to prevent that from happening.

Perhaps Chris can set up a private developer repo and ask for help
from redhat, fujitsu, etc..?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Chris Mason]

On Thu, Oct 06, 2011 at 10:31:41AM -0500, Jeff Putney wrote:
> > No, in this case it means we're confident it will get rolled out.
>=20
> On Aug 18th confidence was high enough to declare a possible release
> that very day.  This confidence turned into 7 weeks of silence
> followed by another 2 week estimate.
>=20
> These confident declarations are why things like mniederle's
> btrfs_rescue are considered 'interim' and not worth building on.  Had
> this confidence of imminent release not been the prevalent message fo=
r
> the last year, others would have stepped in to fill the void.
>=20
> > I've given a number of hard dates recently and I'd prefer to show u=
p
> > with the code instead. =A0I don't think it makes sense to put a par=
tial
> > implementation out there, we'll just have a bunch of people reporti=
ng
> > problems that I know exist.
> >
> > -chris
> >
>=20
> This strategy of 'Lone Wolfing it' has delayed the release by a year.
> Either you are flying solo because you think that you can make more
> meaningful progress without the involvement of the btrfs community, o=
r
> you are willing to forfeit the contributions of the community in orde=
r
> to not have to listen to any complaints.
>=20
> The other problem of this flying solo plan, is that you are making th=
e
> assumption that the problems you know about are more significant than
> the problems you are unaware of and could be flushed out with more
> eyes on the code.  The longer you delay the release of the source, th=
e
> longer it will be until confidence can be generated that major issues
> have been resolved.
>=20
> http://en.wikipedia.org/wiki/Release_early,_release_often

[ Thanks for everyone's comments! ]

Keep in mind that btrfs was released and ran for a long time while
intentionally crashing when we ran out of space.   This was a really
important part of our development because we attracted a huge number of
contributors, and some very brave users.

=46or fsck, even the stuff I have here does have a way to go before it =
is
at the level of an e2fsck or xfs_repair.  But I do want to make sure
that I'm surprised by any bugs before I send it out, and that's just no=
t
the case today.  The release has been delayed because I've alternated
between a few different ways of repairing, and because I got distracted
by some important features in the kernel.

That's how software goes sometimes, and I'll take the criticism because
it hasn't gone as well as it should have.  But, I can't stress enough h=
ow
much I appreciate everyone's contributions and interest in btrfs.

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

On Thu, 6 Oct 2011 23:20:45 +0000 (UTC)
Yalonda Gishtaka <yalonda.gishtaka@gmail.com> wrote:

> and tarnishing Oracle's name. 

Thank you sir you just made my day.

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Honest timeline for btrfsck

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/06/2011 10:50 PM, Chris Mason wrote:
> On Thu, Oct 06, 2011 at 10:31:41AM -0500, Jeff Putney wrote:
>>> No, in this case it means we're confident it will get rolled
>>> out.
>> 
>> On Aug 18th confidence was high enough to declare a possible
>> release that very day.  This confidence turned into 7 weeks of
>> silence followed by another 2 week estimate.
>> 
>> These confident declarations are why things like mniederle's 
>> btrfs_rescue are considered 'interim' and not worth building on.
>> Had this confidence of imminent release not been the prevalent
>> message for the last year, others would have stepped in to fill
>> the void.
>> 
>>> I've given a number of hard dates recently and I'd prefer to
>>> show up with the code instead.  I don't think it makes sense to
>>> put a partial implementation out there, we'll just have a bunch
>>> of people reporting problems that I know exist.
>>> 
>>> -chris
>>> 
>> 
>> This strategy of 'Lone Wolfing it' has delayed the release by a
>> year. Either you are flying solo because you think that you can
>> make more meaningful progress without the involvement of the
>> btrfs community, or you are willing to forfeit the contributions
>> of the community in order to not have to listen to any
>> complaints.
>> 
>> The other problem of this flying solo plan, is that you are
>> making the assumption that the problems you know about are more
>> significant than the problems you are unaware of and could be
>> flushed out with more eyes on the code.  The longer you delay the
>> release of the source, the longer it will be until confidence can
>> be generated that major issues have been resolved.
>> 
>> http://en.wikipedia.org/wiki/Release_early,_release_often
> 
> [ Thanks for everyone's comments! ]
> 
> Keep in mind that btrfs was released and ran for a long time while 
> intentionally crashing when we ran out of space.   This was a
> really important part of our development because we attracted a
> huge number of contributors, and some very brave users.
> 
> For fsck, even the stuff I have here does have a way to go before
> it is at the level of an e2fsck or xfs_repair.  But I do want to
> make sure that I'm surprised by any bugs before I send it out, and
> that's just not the case today.  The release has been delayed
> because I've alternated between a few different ways of repairing,
> and because I got distracted by some important features in the
> kernel.

Yes. The single biggest rule of file system recovery tools is that you
never leave the file system more broken than when you found it. Beta
testing fsck, when the author him/herself isn't comfortable releasing
the code, is insane when you have data you care about. If you
disagree, I'll hit the pause button until you learn some very hard
lessons.

- -Jeff

- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6Og+4ACgkQLPWxlyuTD7KcywCeNmC9N5pwuHaLu1++YhoSQYWC
+Y0An0wgtv3dxsH6ZZCdPy2JihJWOe14
=g/pv
-----END PGP SIGNATURE-----

Re: Honest timeline for btrfsck

[Jeff Putney]

> For fsck, even the stuff I have here does have a way to go before it =
is
> at the level of an e2fsck or xfs_repair. =A0But I do want to make sur=
e
> that I'm surprised by any bugs before I send it out, and that's just =
not
> the case today. =A0The release has been delayed because I've alternat=
ed
> between a few different ways of repairing, and because I got distract=
ed
> by some important features in the kernel.

I think there is a major difference between touching up a few bugs
before you release the code, and experimenting with a bunch of
different ways of repairing before you release the code.  I know I for
one would get as much value out of reading the code for the strategies
you abandoned as I would get from reading the final code, but I don't
know if having those branches in the main repo would have any value
for the project as a whole.

As the current solution goes, I'd just like to see a stake in the
ground for some sort of process to move away from the one man army
approach, should distractions etc continue.  Given the latest 2 week
estimate, something along the lines of, in 4 or 6 weeks, if it still
isn't 'ready', code will start to be released.


> That's how software goes sometimes, and I'll take the criticism becau=
se
> it hasn't gone as well as it should have. =A0But, I can't stress enou=
gh how
> much I appreciate everyone's contributions and interest in btrfs.
>
> -chris

I'm only criticizing the decision to not release the source, in
particular given the delays.  We all have our priorities and
distractions, and s**t happens.  (Part of why I'd argue against the
flying solo strategy.)  But, I really do appreciate all the stuff
you've built, which is part of why I am so keen on reading it. :-) .

Thanks for all the effort
--jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Josef Bacik]

On 10/07/2011 09:40 AM, Jeff Putney wrote:
>> For fsck, even the stuff I have here does have a way to go before it is
>> at the level of an e2fsck or xfs_repair.  But I do want to make sure
>> that I'm surprised by any bugs before I send it out, and that's just not
>> the case today.  The release has been delayed because I've alternated
>> between a few different ways of repairing, and because I got distracted
>> by some important features in the kernel.
> 
> I think there is a major difference between touching up a few bugs
> before you release the code, and experimenting with a bunch of
> different ways of repairing before you release the code.  I know I for
> one would get as much value out of reading the code for the strategies
> you abandoned as I would get from reading the final code, but I don't
> know if having those branches in the main repo would have any value
> for the project as a whole.
> 
> As the current solution goes, I'd just like to see a stake in the
> ground for some sort of process to move away from the one man army
> approach, should distractions etc continue.  Given the latest 2 week
> estimate, something along the lines of, in 4 or 6 weeks, if it still
> isn't 'ready', code will start to be released.
> 
> 
>> That's how software goes sometimes, and I'll take the criticism because
>> it hasn't gone as well as it should have.  But, I can't stress enough how
>> much I appreciate everyone's contributions and interest in btrfs.
>>
>> -chris
> 
> I'm only criticizing the decision to not release the source, in
> particular given the delays.  We all have our priorities and
> distractions, and s**t happens.  (Part of why I'd argue against the
> flying solo strategy.)  But, I really do appreciate all the stuff
> you've built, which is part of why I am so keen on reading it. :-) .
> 

The problem is people won't just read it, they will use it.  I wrote a
basic repair tool for a user in Fedora who seemed to have a very
specific kind of corruption, and earlier this week almost a month after
my initial repair tool I had to write another tool to go in and just
pull all his data off his disk because a bug in my repair tool made his
fs _WORSE_, to the point I'm not sure I can fix it.  Fsck has the
potential to make any users problems worse, and given the increasing
number of people putting production systems on btrfs with no backups the
idea of releasing a unpolished and not fully tested fsck into the world
is terrifying, and would likely cause long term "I heard that file
system's fsck tool eats babies" sort of reputation.

Release early and release often is nice for web browsers and desktop
environments, it's not so nice with things that could result in data
loss, especially when our user base is growing in leaps and bounds and
aren't necessarily informed enough on the dangers of using an file
system that's still under heavy development.  Thanks,

Josef

Re: Honest timeline for btrfsck

[Josef Bacik]

On 10/06/2011 04:56 PM, Francesco Riosa wrote:
> 2011/10/6 Andi Kleen <andi@firstfloor.org>:
>> Jeff Putney <jeffrey.putney@gmail.com> writes:
>>>
>>> http://en.wikipedia.org/wiki/Release_early,_release_often
>>
>> Well the other principle in free software you're forgetting
>> is:
>>
>> "It will be released when it's ready"
>>
>> If you don't like Chris' ways to do releases you're free to write
>> something on your own or pay someone to do so. Otherwise
>> you just have to deal with his time frames, as shifty
>> as they may be.
> 
> I did a different thing, I've offered Chris money to help rescue an
> hosed btrfs or to point to someone who could do, we ended in doing
> some tests (for free) but nothing else materialized.
> While the time passed has diminished the value of the data to be
> rescued I'm more on the "show us some code we can start from" than "it
> will be released when ready" vagon.
> 

If you still need that data, clone this repo

git://github.com/josefbacik/btrfs-progs.git

run make, and then run

./restore /dev/whatever /some/dir

and it will try and suck all of your data off the disk and dump it in
that directory.  If you have snapshots it will skip them by default, so
if you have snapshots that have useful data in them you'll want to use
the -s option.  If you run into random errors that you think are
recoverable, or if you don't care about the file that's being recovered,
you can run with -i which will ignore errors and keep trying to recover
your files.  Thanks,

Josef

Re: Honest timeline for btrfsck

[Dave]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Oct 07, 2011 at 10:50:04AM -0400, Josef Bacik wrote:
> If you still need that data, clone this repo
> 
> git://github.com/josefbacik/btrfs-progs.git
> 
> run make, and then run
> 
> ./restore /dev/whatever /some/dir
> 
> and it will try and suck all of your data off the disk and dump it in
> that directory.  If you have snapshots it will skip them by default, so
> if you have snapshots that have useful data in them you'll want to use
> the -s option.  If you run into random errors that you think are
> recoverable, or if you don't care about the file that's being recovered,
> you can run with -i which will ignore errors and keep trying to recover
> your files.  Thanks,

I'm actually MUCH more comfortable with this solution.  Rather than having an
untested fsck making changes to the filesystem, a way to get data off a broken
btrfs volume would be sufficient for the time being.  I've had three cases in
the past two months where I've had to resort to hacking the kernel to get at
data on broken btrfs volumes.  Since no one is using btrfs in production,
recovering a volume quickly is not a concern.  For people who haven't backed up
since yesterday, but want to recover the 8 hours of work they did today, the
above solution is great.

(Mind you I've never actually tried the solution above)

- -- 
- -=[dave]=-

Entropy isn't what it used to be.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iF4EAREIAAYFAk6PGTUACgkQXM0u5ajNnChDJQD7BqDiiMk0KZL0HBaveFIolYc4
VFaQpiyZoPkkmL9i/e4A/2c+t+w/xrmOMu5+245DoRhKMOsQ0bNPps9GSiDNJwW5
=0lnT
-----END PGP SIGNATURE-----

Re: Honest timeline for btrfsck

[Mike]

On 11-10-06 07:50 PM, Chris Mason wrote:
> That's how software goes sometimes, and I'll take the criticism because
> it hasn't gone as well as it should have.  But, I can't stress enough how
> much I appreciate everyone's contributions and interest in btrfs.

With all due respect Chris, your actions and your words seem to 
contradict each other. It would appear that people are wanting to help 
contribute, but without showing them the code, you're preventing any 
contributions from happening. As you know, contributing is more than 
just code, just as important is proper testing, especially with a fsck tool.

I also don't think you are giving people enough credit. e2fsck will 
cause corruption pretty much everytime its run on a mounted file system, 
but a nice big nasty warning message seems to handle that quite well and 
anyone who ignores it, well thats their own fault, not the developers:

e2fsck 1.41.14 (22-Dec-2010)
/dev/sdb1 is mounted.

WARNING!!!  The filesystem is mounted.   If you continue you ***WILL***
cause ***SEVERE*** filesystem damage.

Do you really want to continue (y/n)? cancelled!

You could easily place the same warning in btrfs fsck even for normal 
use and recommend/require that it be run on a loopback image rather than 
the actual data itself or something. Heck, even have it run in "make no 
changes" mode by default and require recompiling to enable "fix my 
filesystem" mode.

In fact, when its first released that would probably be a good idea to 
do this anyways. The reality is, it doesn't matter how long you work on 
the fsck tool, its pretty much guaranteed to be a few bugs that corrupt 
some peoples data even more than it was before, thats the price you pay 
for being on the bleeding edge.

Don't get me wrong, I definitely appreciate all your work, I just wish I 
could appreciate it even more with a fsck tool. ;)

-- Mike

Re: Honest timeline for btrfsck

[Jeff Putney]

The rescue tool may have a lot of the logic I personally am most
interested in reading.  Thank you for that!

> The problem is people won't just read it, they will use it.

I've read every last line of the current btrfsck, even though it
doesn't do anything.  I am interested in the source specifically to
read it.

> I wrote a
> basic repair tool for a user in Fedora who seemed to have a very
> specific kind of corruption, and earlier this week almost a month after
> my initial repair tool I had to write another tool to go in and just
> pull all his data off his disk because a bug in my repair tool made his
> fs _WORSE_, to the point I'm not sure I can fix it.

These are specifically the types of one off solutions that are having
to be done because the source hasn't been released for the community
to finish up.

> Fsck has the
> potential to make any users problems worse, and given the increasing
> number of people putting production systems on btrfs with no backups the
> idea of releasing a unpolished and not fully tested fsck into the world
> is terrifying, and would likely cause long term "I heard that file
> system's fsck tool eats babies" sort of reputation.

I fail to see the distinction between releasing an unpolished fsck vs
releasing an unpolished fs driver.  They are collaborating parts of
the same system.  Whether they say btrfsck eats babies or btrfs eats
babies, they are still saying that the babies are getting eaten.

> Release early and release often is nice for web browsers and desktop
> environments, it's not so nice with things that could result in data
> loss, especially when our user base is growing in leaps and bounds and
> aren't necessarily informed enough on the dangers of using an file
> system that's still under heavy development.

What you are saying is that the specter of increased data loss somehow
outweighs the combined benefit that the community has to offer.
Unless the current state of the project IS due solely to the work of
Chris and Josef, and you don't feel that the community at large has
provided meaningful contributions, than I think that's a pretty
skeptical and unappreciative attitude towards all of the people who
HAVE contributed to this project.

The 'specialness' of an fsck tool is highly suspect.  Current
development versions of all the other fsck tools are available in
their respective repositories, and yet headlines of their eating
babies are still pretty scarce.
I'm glad that Linus didn't use your logic when it came to releasing
the linux kernel.  Do you think the entire linux kernel is more like a
web browser, or a desktop environment?

This smells more like post hoc justification of being territorial over
a pet project than it does actual reasons for keeping the source a
state secret of oracle.  Unless their is no intention of releasing the
source, and Oracle intends to keep it a closed source product for
their own linux distributions alone.

Re: Honest timeline for btrfsck

[Josef Bacik]

On 10/07/2011 11:58 AM, Jeff Putney wrote:
> The rescue tool may have a lot of the logic I personally am most
> interested in reading.  Thank you for that!
> 
>> The problem is people won't just read it, they will use it.
> 
> I've read every last line of the current btrfsck, even though it
> doesn't do anything.  I am interested in the source specifically to
> read it.
> 
>> I wrote a
>> basic repair tool for a user in Fedora who seemed to have a very
>> specific kind of corruption, and earlier this week almost a month after
>> my initial repair tool I had to write another tool to go in and just
>> pull all his data off his disk because a bug in my repair tool made his
>> fs _WORSE_, to the point I'm not sure I can fix it.
> 
> These are specifically the types of one off solutions that are having
> to be done because the source hasn't been released for the community
> to finish up.
> 
>> Fsck has the
>> potential to make any users problems worse, and given the increasing
>> number of people putting production systems on btrfs with no backups the
>> idea of releasing a unpolished and not fully tested fsck into the world
>> is terrifying, and would likely cause long term "I heard that file
>> system's fsck tool eats babies" sort of reputation.
> 
> I fail to see the distinction between releasing an unpolished fsck vs
> releasing an unpolished fs driver.  They are collaborating parts of
> the same system.  Whether they say btrfsck eats babies or btrfs eats
> babies, they are still saying that the babies are getting eaten.
> 
>> Release early and release often is nice for web browsers and desktop
>> environments, it's not so nice with things that could result in data
>> loss, especially when our user base is growing in leaps and bounds and
>> aren't necessarily informed enough on the dangers of using an file
>> system that's still under heavy development.
> 
> What you are saying is that the specter of increased data loss somehow
> outweighs the combined benefit that the community has to offer.
> Unless the current state of the project IS due solely to the work of
> Chris and Josef, and you don't feel that the community at large has
> provided meaningful contributions, than I think that's a pretty
> skeptical and unappreciative attitude towards all of the people who
> HAVE contributed to this project.
> 
> The 'specialness' of an fsck tool is highly suspect.  Current
> development versions of all the other fsck tools are available in
> their respective repositories, and yet headlines of their eating
> babies are still pretty scarce.
> I'm glad that Linus didn't use your logic when it came to releasing
> the linux kernel.  Do you think the entire linux kernel is more like a
> web browser, or a desktop environment?
> 
> This smells more like post hoc justification of being territorial over
> a pet project than it does actual reasons for keeping the source a
> state secret of oracle.  Unless their is no intention of releasing the
> source, and Oracle intends to keep it a closed source product for
> their own linux distributions alone.

Well you've caught us, this is all just a conspiracy to keep the users
under our thumbs and to block out any contributions.  Sorry Chris, we
kept it up for a good year tho!

Josef

Re: Honest timeline for btrfsck

[Jeff Putney]

You jest, but in fact that is the result you've achieved, through
conspiring or not.

Do you honestly believe that had the source been public from the
start, that after a year there would still be no quality fsck tool?
Contributions are, de facto, blocked.
Had there not been repeated promise of an fsck utility for the last
year, do you honestly believe no one else would have begun
development?  Call it under your thumb if you'd like, but you'll argue
these declarations didn't have a stifling effect in vain.


On Fri, Oct 7, 2011 at 11:08 AM, Josef Bacik <josef@redhat.com> wrote:
> On 10/07/2011 11:58 AM, Jeff Putney wrote:
>> The rescue tool may have a lot of the logic I personally am most
>> interested in reading. =A0Thank you for that!
>>
>>> The problem is people won't just read it, they will use it.
>>
>> I've read every last line of the current btrfsck, even though it
>> doesn't do anything. =A0I am interested in the source specifically t=
o
>> read it.
>>
>>> I wrote a
>>> basic repair tool for a user in Fedora who seemed to have a very
>>> specific kind of corruption, and earlier this week almost a month a=
fter
>>> my initial repair tool I had to write another tool to go in and jus=
t
>>> pull all his data off his disk because a bug in my repair tool made=
 his
>>> fs _WORSE_, to the point I'm not sure I can fix it.
>>
>> These are specifically the types of one off solutions that are havin=
g
>> to be done because the source hasn't been released for the community
>> to finish up.
>>
>>> Fsck has the
>>> potential to make any users problems worse, and given the increasin=
g
>>> number of people putting production systems on btrfs with no backup=
s the
>>> idea of releasing a unpolished and not fully tested fsck into the w=
orld
>>> is terrifying, and would likely cause long term "I heard that file
>>> system's fsck tool eats babies" sort of reputation.
>>
>> I fail to see the distinction between releasing an unpolished fsck v=
s
>> releasing an unpolished fs driver. =A0They are collaborating parts o=
f
>> the same system. =A0Whether they say btrfsck eats babies or btrfs ea=
ts
>> babies, they are still saying that the babies are getting eaten.
>>
>>> Release early and release often is nice for web browsers and deskto=
p
>>> environments, it's not so nice with things that could result in dat=
a
>>> loss, especially when our user base is growing in leaps and bounds =
and
>>> aren't necessarily informed enough on the dangers of using an file
>>> system that's still under heavy development.
>>
>> What you are saying is that the specter of increased data loss someh=
ow
>> outweighs the combined benefit that the community has to offer.
>> Unless the current state of the project IS due solely to the work of
>> Chris and Josef, and you don't feel that the community at large has
>> provided meaningful contributions, than I think that's a pretty
>> skeptical and unappreciative attitude towards all of the people who
>> HAVE contributed to this project.
>>
>> The 'specialness' of an fsck tool is highly suspect. =A0Current
>> development versions of all the other fsck tools are available in
>> their respective repositories, and yet headlines of their eating
>> babies are still pretty scarce.
>> I'm glad that Linus didn't use your logic when it came to releasing
>> the linux kernel. =A0Do you think the entire linux kernel is more li=
ke a
>> web browser, or a desktop environment?
>>
>> This smells more like post hoc justification of being territorial ov=
er
>> a pet project than it does actual reasons for keeping the source a
>> state secret of oracle. =A0Unless their is no intention of releasing=
 the
>> source, and Oracle intends to keep it a closed source product for
>> their own linux distributions alone.
>
> Well you've caught us, this is all just a conspiracy to keep the user=
s
> under our thumbs and to block out any contributions. =A0Sorry Chris, =
we
> kept it up for a good year tho!
>
> Josef
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Gour-Gadadhara Dasa]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

On Fri, 07 Oct 2011 08:39:36 -0700
Mike <ipso@snappymail.ca> wrote:

> I also don't think you are giving people enough credit. e2fsck will 
> cause corruption pretty much everytime its run on a mounted file
> system, but a nice big nasty warning message seems to handle that
> quite well and anyone who ignores it, well thats their own fault, not
> the developers:

+1


-- 
“In the material world, conceptions of good and bad are
all mental speculations…” (Sri Caitanya Mahaprabhu)

http://atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Honest timeline for btrfsck

[cwillu]

On Fri, Oct 7, 2011 at 11:07 AM, Jeff Putney <jeffrey.putney@gmail.com>=
 wrote:
> You jest, but in fact that is the result you've achieved, through
> conspiring or not.
>
> Do you honestly believe that had the source been public from the
> start, that after a year there would still be no quality fsck tool?
> Contributions are, de facto, blocked.
> Had there not been repeated promise of an fsck utility for the last
> year, do you honestly believe no one else would have begun
> development? =C2=A0Call it under your thumb if you'd like, but you'll=
 argue
> these declarations didn't have a stifling effect in vain.

Heh, what sort of "quality" are you thinking would develop?  A
recovery tool by its nature is picking up the pieces where those
pieces are inconsistent.  The nature of those inconsistencies will
change with every patch that's more than a cleanup.

Combined with the well-known tendencies of users to not report errors
that are trivial to work around, and I find myself quite content with
the status quo:  a few general recovery techniques that can be found
with some digging, inconvenient enough that the reports don't get
lost, with enough context that the appropriate warnings and
alternatives can be given.

Yes, a deliberately broken-by-makefile version of what he's looking at
would be interesting, but I suspect it's not much past what any
competent programmer would put together given a couple weeks going
over the disk format, and we already have a couple of those.  What we
want is still in Chris' head, otherwise we _would_ have something.

--cwillu

Warning: while cwillu is never "wrong", he may not correspond to
reality.  cwillu should not be taken with caffeine or alcohol.
Contact a doctor immediately if cwillu submits patches, rants, or
directives.  Do not leave cwillu within reach of children or
ubuntuforums users.  Always make sufficient backups before taking
cwillu.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Asdo]

On 10/07/11 04:25, Chester wrote:
> The problem with this is that people naturally look for a fsck tool
> when something bad goes wrong. Something as important as a fsck
> utility shouldn't be released (unofficially or officially) half baked.
> It can irreparably destroy a filesystem which could've otherwise been
> repaired with a fully functional fsck.
>
> I think Chris is trying to prevent that from happening.

What I would like to know instead, is WHY we need an btrfsck when ZFS 
does not. Zfs also has this kind of problems especially on power 
failures, but you can always mount by rolling back to a previous 
uberblock, showing an earlier view of the filesystem, which would be 
consistent.

It wasn't always like this with ZFS, but this "feature" has been added 
after ther numerous requests of users who lost their complete 
filesystems after a power failure or similar events. And there was no 
zfsck (there still isn't, and it's apparently not needed).

This should be the fix I'm talking about (my original link to Sun site 
doesn't work any more)
http://wesunsolve.net/bugid/id/6667683
You can look up the internet with something like "zfs roll back txg" or 
"zfs mount old uberblock"...

I don't recall any other  "I have hosed my FS" request for help after 
the date this feature was implemented/provided.

Why isn't this approach dead-easy to implement with btrfs?

Re: Honest timeline for btrfsck

[cwillu]

> What I would like to know instead, is WHY we need an btrfsck when ZFS does
> not. Zfs also has this kind of problems especially on power failures, but
> you can always mount by rolling back to a previous uberblock, showing an
> earlier view of the filesystem, which would be consistent.
>
> It wasn't always like this with ZFS, but this "feature" has been added after
> ther numerous requests of users who lost their complete filesystems after a
> power failure or similar events. And there was no zfsck (there still isn't,
> and it's apparently not needed).

There's a couple different categories of tool that are occasionally
referred to as "fsck".  Consistency checking (which we already have in
the current btrfsck), data recovery (which we have a couple rough
tools for, but which improved tools are desired and probably part of
cmason's plans), transparent healing making the filesystem always
mountable (the big selling point of both zfs and btrfs, although
obviously btrfs is still immature in this regard; again, something
that progress is expected on), in-place "rebuild the world" repair
tools (which again I believe is part of cmason's plans), online scrubs
(which we have), and so on.

You'll note that zfs now has all of these tools (and took a remarkable
amount of time to acquire some of them), they just never call any
combination of them "zfsck".  Really, zfs doesn't require fsck in
exactly the same sense that ext3 doesn't:  given hardware that doesn't
lie to the filesystem, there's no massive "search and rescue"
operation required after an otherwise unclean shutdown.  That's it.
And we mostly have that too, modulo the usual and expected bumps of a
very young filesystem.

--cwillu

Warning:  cwillu may cause excessive warnings.

Re: Honest timeline for btrfsck

[Diego Calleja]

On Viernes, 7 de Octubre de 2011 21:10:33 Asdo escribi=F3:
> failures, but you can always mount by rolling back to a previous=20
> uberblock, showing an earlier view of the filesystem, which would be=20
> consistent.

This is already available in Btrfs, command btrfsck -s.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Helmut Hullen]

Hallo, Asdo,

Du meintest am 07.10.11:

> What I would like to know instead, is WHY we need an btrfsck when ZFS
> does not.

I need at least some tool which can "hide" defect sectors - I just have  
such a problem.

Viele Gruesse!
Helmut

Re: Honest timeline for btrfsck

[Jeff Putney]

> Heh, what sort of "quality" are you thinking would develop? =A0A
> recovery tool by its nature is picking up the pieces where those
> pieces are inconsistent. =A0The nature of those inconsistencies will
> change with every patch that's more than a cleanup.
>

Seriously?  You want to delay the solving of the problems we have
today, because we are going to have different problems in the future?
I hate to be the bearer of bad news, but for the most part, we are
going to have new problems in addition to the ones we already have.
We are not going to get a whole brand new batch of problems that are
somehow going to magically make all our old problems obsolete.  You
also make the assumption that the solution to these new problems is
going to have absolutely no similarity to the current problems,
otherwise it would be beneficial to have a quality solution to similar
problems to build off of.

> Combined with the well-known tendencies of users to not report errors
> that are trivial to work around, and I find myself quite content with
> the status quo: =A0a few general recovery techniques that can be foun=
d
> with some digging, inconvenient enough that the reports don't get
> lost, with enough context that the appropriate warnings and
> alternatives can be given.

If the status quo is an acceptable condition, then you must not see
the need for any fsck utility.  If you see a need for an fsck utility,
then certainly you must see the problem in committing to 'eminently'
deliver that utility repeatedly for a year or so, and never delivering
it.  If you don't see that as a fundamental wrench in the works, I
don't know what would be.


> Yes, a deliberately broken-by-makefile version of what he's looking a=
t
> would be interesting, but I suspect it's not much past what any
> competent programmer would put together given a couple weeks going
> over the disk format, and we already have a couple of those.

Yeah, I am also suspicious that that is all that exists, and I suspect
that may have something to do with the resistance to releasing the
source.  If that is the case, why not just come clean about it, and
let others start contributing, so we can get somewhere with it.

> What we want is still in Chris' head, otherwise we _would_ have
> something.

If it is still only exists in his head, after a year, how long should
we wait before someone else takes the reigns?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Asdo]

On 10/07/11 22:19, Diego Calleja wrote:
> On Viernes, 7 de Octubre de 2011 21:10:33 Asdo escribi=F3:
>> failures, but you can always mount by rolling back to a previous
>> uberblock, showing an earlier view of the filesystem, which would be
>> consistent.
> This is already available in Btrfs, command btrfsck -s.

Whops!? Then I am wondering what causes these corrupted unmountable=20
filesystems.
I think that in Btrfs wiki (which is now down) there was written that=20
btrfs was substantially stable, with the only exception that a power=20
loss combined with drives not honoring barriers could result in an=20
unmountable filesystems.
This is misleading information then.
If btrfsck -s is already available, power loss + not honoring barriers=20
cannot be the reason for unmountable filesystems. The wiki should be=20
changed...
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Fajar A. Nugraha]

On Sun, Oct 9, 2011 at 4:13 AM, Asdo <asdo@shiftmail.org> wrote:
> On 10/07/11 22:19, Diego Calleja wrote:
>>
>> On Viernes, 7 de Octubre de 2011 21:10:33 Asdo escribi=F3:
>>>
>>> failures, but you can always mount by rolling back to a previous
>>> uberblock, showing an earlier view of the filesystem, which would b=
e
>>> consistent.
>>
>> This is already available in Btrfs, command btrfsck -s.
>
> Whops!? Then I am wondering what causes these corrupted unmountable
> filesystems.

"-s" does not select previous uberblock. It selects alternate uberblock=
=2E

> I think that in Btrfs wiki (which is now down) there was written that=
 btrfs
> was substantially stable, with the only exception that a power loss c=
ombined
> with drives not honoring barriers could result in an unmountable
> filesystems.

for this condition btrfs-zero-log will be more useful

--=20
=46ajar
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Chris Mason]

Excerpts from Jeff Putney's message of 2011-10-07 11:58:55 -0400:
> 
> The 'specialness' of an fsck tool is highly suspect.  Current
> development versions of all the other fsck tools are available in
> their respective repositories, and yet headlines of their eating
> babies are still pretty scarce.
> I'm glad that Linus didn't use your logic when it came to releasing
> the linux kernel.  Do you think the entire linux kernel is more like a
> web browser, or a desktop environment?
> 
> This smells more like post hoc justification of being territorial over
> a pet project than it does actual reasons for keeping the source a
> state secret of oracle.  Unless their is no intention of releasing the
> source, and Oracle intends to keep it a closed source product for
> their own linux distributions alone.

I wish there were bigger forces at play here than me just not having it
finished yet, but that's all there is to it.

Btrfs is GPL, and Oracle is not holding back a massive set of tools around
btrfs.  The project has always stood on contributions from a number of
companies and closed tools aren't going to happen.

-chris

Re: Honest timeline for btrfsck

[Chris Mason]

Excerpts from Asdo's message of 2011-10-07 15:10:33 -0400:
> On 10/07/11 04:25, Chester wrote:
> > The problem with this is that people naturally look for a fsck tool
> > when something bad goes wrong. Something as important as a fsck
> > utility shouldn't be released (unofficially or officially) half baked.
> > It can irreparably destroy a filesystem which could've otherwise been
> > repaired with a fully functional fsck.
> >
> > I think Chris is trying to prevent that from happening.
> 
> What I would like to know instead, is WHY we need an btrfsck when ZFS 
> does not. Zfs also has this kind of problems especially on power 
> failures, but you can always mount by rolling back to a previous 
> uberblock, showing an earlier view of the filesystem, which would be 
> consistent.
> 
> It wasn't always like this with ZFS, but this "feature" has been added 
> after ther numerous requests of users who lost their complete 
> filesystems after a power failure or similar events. And there was no 
> zfsck (there still isn't, and it's apparently not needed).

One of my patches here adds a circular ring of past roots for a similar
feature.  The problem is that blocks get reused after a commit, so we
have to either slow that down (making enospc even more complex) or we'd
have to just use the past super and hope.

It gets even worse because things past supers aren't always enough.  The
drive can just plain corrupt an important block, or write-back cache
problems can make corruptions that last much longer than the last 5 or
10 commits.  Bad things do happen to good data, so you just plain need a
real fsck.

-chris

Re: Honest timeline for btrfsck

[Francesco Riosa]

2011/10/7 Josef Bacik <josef@redhat.com>:
> On 10/06/2011 04:56 PM, Francesco Riosa wrote:
>> 2011/10/6 Andi Kleen <andi@firstfloor.org>:
>>> Jeff Putney <jeffrey.putney@gmail.com> writes:
>>>>
>>>> http://en.wikipedia.org/wiki/Release_early,_release_often
>>>
>>> Well the other principle in free software you're forgetting
>>> is:
>>>
>>> "It will be released when it's ready"
>>>
>>> If you don't like Chris' ways to do releases you're free to write
>>> something on your own or pay someone to do so. Otherwise
>>> you just have to deal with his time frames, as shifty
>>> as they may be.
>>
>> I did a different thing, I've offered Chris money to help rescue an
>> hosed btrfs or to point to someone who could do, we ended in doing
>> some tests (for free) but nothing else materialized.
>> While the time passed has diminished the value of the data to be
>> rescued I'm more on the "show us some code we can start from" than "=
it
>> will be released when ready" vagon.
>>
>
> If you still need that data, clone this repo
>
> git://github.com/josefbacik/btrfs-progs.git
>
> run make, and then run
>
> ./restore /dev/whatever /some/dir
>
> and it will try and suck all of your data off the disk and dump it in
> that directory. =A0If you have snapshots it will skip them by default=
, so
> if you have snapshots that have useful data in them you'll want to us=
e
> the -s option. =A0If you run into random errors that you think are
> recoverable, or if you don't care about the file that's being recover=
ed,
> you can run with -i which will ignore errors and keep trying to recov=
er
> your files. =A0Thanks,
>
> Josef
>

I've tried, w/o luck

explanation come from 2011-06-21 thread;
http://thread.gmane.org/gmane.comp.file-systems.btrfs/11435
the following refer to a copy of that system

Label: space02  uuid: f752def1-1abc-48c7-8ebb-47ba37b8ffa6
        Total devices 7 FS bytes used 173.12GB
        devid    6 size 488.94GB used 60.25GB path /dev/sdd7
        devid    2 size 487.65GB used 58.76GB path /dev/sdd8
        devid    7 size 487.65GB used 0.00 path    /dev/sdf7
        devid    3 size 487.65GB used 60.26GB path /dev/sdf8
        devid    7 size 487.65GB used 1.50GB path  /dev/sdg7
        devid    5 size 488.94GB used 58.75GB path /dev/sdb7
        devid    4 size 487.65GB used 60.26GB path /dev/sdb8

# ./restore /dev/sdd7 /tmp/restore
failed to read /dev/sr0
failed to read /dev/sr0
restore: volumes.c:1367: btrfs_read_sys_array: Assertion `!(ret)' faile=
d.
Aborted

this is the output after filling volumes.c of printk at every function =
start,
there is also a btrfs_print_tree() call at the failing read_one_chunk()

please let me know if I can do anything

int btrfs_scan_one_device(int fd=3D3, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D1, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D134816, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/md127, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdd, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdc, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdb, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdg, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sdf, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sde, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
failed to read /dev/sr0
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/md128, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram9, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram14, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram13, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram15, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram0, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram12, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram11, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram10, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/ram1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/sda, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop0, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D3, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D139870728973573,
u64 super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D15306741171905079,
u64 super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/md127, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdd, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdc, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdb, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdg, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sdf, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sde, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D7, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
failed to read /dev/sr0
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/md128, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram9, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram14, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram13, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram15, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram0, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram12, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram11, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram10, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop6, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/ram1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop4, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/sda, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
static int device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)
static struct btrfs_fs_devices *find_fsid(u8 *fsid)
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop7, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop0, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop3, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop2, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_scan_one_device(int fd=3D5, *path=3D/dev/loop1, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)
int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, int flags)
int btrfs_read_sys_array(struct btrfs_root *root)
static int read_one_chunk(struct btrfs_root *root, struct btrfs_key *ke=
y,...
  leaf 65536 items 0 free space 3995 generation 604315930624 owner 2097=
9712
  fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6
  chunk uuid 5f424852-6653-5f4d-318e-0b0000000000
struct btrfs_device *btrfs_find_device(struct btrfs_root *root, u64 dev=
id,
static struct btrfs_device *__find_device(struct list_head *head, u64
devid, u8 *uuid
restore: volumes.c:1397: btrfs_read_sys_array: Assertion `!(ret)' faile=
d.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Josef Bacik]

On Tue, Oct 11, 2011 at 11:21:45PM +0200, Francesco Riosa wrote:
> 2011/10/7 Josef Bacik <josef@redhat.com>:
> > On 10/06/2011 04:56 PM, Francesco Riosa wrote:
> >> 2011/10/6 Andi Kleen <andi@firstfloor.org>:
> >>> Jeff Putney <jeffrey.putney@gmail.com> writes:
> >>>>
> >>>> http://en.wikipedia.org/wiki/Release_early,_release_often
> >>>
> >>> Well the other principle in free software you're forgetting
> >>> is:
> >>>
> >>> "It will be released when it's ready"
> >>>
> >>> If you don't like Chris' ways to do releases you're free to write
> >>> something on your own or pay someone to do so. Otherwise
> >>> you just have to deal with his time frames, as shifty
> >>> as they may be.
> >>
> >> I did a different thing, I've offered Chris money to help rescue a=
n
> >> hosed btrfs or to point to someone who could do, we ended in doing
> >> some tests (for free) but nothing else materialized.
> >> While the time passed has diminished the value of the data to be
> >> rescued I'm more on the "show us some code we can start from" than=
 "it
> >> will be released when ready" vagon.
> >>
> >
> > If you still need that data, clone this repo
> >
> > git://github.com/josefbacik/btrfs-progs.git
> >
> > run make, and then run
> >
> > ./restore /dev/whatever /some/dir
> >
> > and it will try and suck all of your data off the disk and dump it =
in
> > that directory. =A0If you have snapshots it will skip them by defau=
lt, so
> > if you have snapshots that have useful data in them you'll want to =
use
> > the -s option. =A0If you run into random errors that you think are
> > recoverable, or if you don't care about the file that's being recov=
ered,
> > you can run with -i which will ignore errors and keep trying to rec=
over
> > your files. =A0Thanks,
> >
> > Josef
> >
>=20
> I've tried, w/o luck
>=20
> explanation come from 2011-06-21 thread;
> http://thread.gmane.org/gmane.comp.file-systems.btrfs/11435
> the following refer to a copy of that system
>=20
> Label: space02  uuid: f752def1-1abc-48c7-8ebb-47ba37b8ffa6
>         Total devices 7 FS bytes used 173.12GB
>         devid    6 size 488.94GB used 60.25GB path /dev/sdd7
>         devid    2 size 487.65GB used 58.76GB path /dev/sdd8
>         devid    7 size 487.65GB used 0.00 path    /dev/sdf7
>         devid    3 size 487.65GB used 60.26GB path /dev/sdf8
>         devid    7 size 487.65GB used 1.50GB path  /dev/sdg7
>         devid    5 size 488.94GB used 58.75GB path /dev/sdb7
>         devid    4 size 487.65GB used 60.26GB path /dev/sdb8
>=20
> # ./restore /dev/sdd7 /tmp/restore
> failed to read /dev/sr0
> failed to read /dev/sr0
> restore: volumes.c:1367: btrfs_read_sys_array: Assertion `!(ret)' fai=
led.
> Aborted
>=20

So this is kind of a problem since you have multiple disks.  We maybe c=
ould get
away with ignoring a failure, but the problem is if you have data on a =
disk
where we couldn't read the chunk then the chances are we won't be able =
to map
that file and read the data off.  That being said, theres no harm in tr=
ying ;).
Can you make btrfs_read_sys_array complain about failing, but not actua=
lly BUG?
See if that helps you.  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Martin Steigerwald]

Am Freitag, 7. Oktober 2011 schrieb Gour-Gadadhara Dasa:
> On Fri, 07 Oct 2011 08:39:36 -0700
>=20
> Mike <ipso@snappymail.ca> wrote:
> > I also don't think you are giving people enough credit. e2fsck will
> > cause corruption pretty much everytime its run on a mounted file
> > system, but a nice big nasty warning message seems to handle that
> > quite well and anyone who ignores it, well thats their own fault, n=
ot
>=20
> > the developers:
> +1

Even if its a thousand +1 following, it seems to me that its perfectly=20
Chris Masons decision...

Chris seems to have some ideas on when to release the fsck. So what do =
you=20
think you achieve by asking for its release again and again? It won=B4t=
=20
happen anytime sooner unless you happen to find the holy mantra that=20
convinces Chris to release it now. I bet thats unlikely.

So its either do an fsck for yourself or wait...

BTRFS is still experimental software...

I do not argue that having a nice fsck sooner than later is fine, but I=
=20
question the usefulness of repeating reminders. Chris Mason and other=20
developers possibly working on the fsck should know by now, that you wa=
nt=20
it. So its unlikely that "I want it too" is going to change anything.

Ciao,
--=20
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

> Even if its a thousand +1 following, it seems to me that its perfectl=
y
> Chris Masons decision...

Obviously.

> Chris seems to have some ideas on when to release the fsck.

Yes, and that idea of when has been drifting in the couple week range
for about a year.

> So what do you
> think you achieve by asking for its release again and again?

Best case scenario, everything just gets released in about a week, as
per the last estimate.  Barring that, a decoupling of the source
release from the tools completion, opening up the possibility of no
longer having a single point of failure (there is only so much one
person can do).  If he doesn't have any code that he feels is worth
building on, and releasing, then he should come clean about that and
open the door for someone else to step in.

> It won=B4t
> happen anytime sooner unless you happen to find the holy mantra that
> convinces Chris to release it now. I bet thats unlikely.

Or if the general consensus is that it is never going to get done, and
someone else should take the reigns.

> So its either do an fsck for yourself or wait...

This would be an appealing option, and one that at least 2 people have
pursued to some extent so far, but who wants to invest their time in
something like this if all of their efforts are just going to be
considered disposable?  This is in fact the single biggest problem
with Chris promising a tool, and perpetually not delivering it, nor
letting anyone see his source.  He is guaranteeing that any effort you
put into helping out with this would be ultimately wasted.


> BTRFS is still experimental software...

And yet Chris and Oracle are moving it into production use.

> I do not argue that having a nice fsck sooner than later is fine, but=
 I
> question the usefulness of repeating reminders. Chris Mason and other
> developers possibly working on the fsck should know by now, that you =
want
> it. So its unlikely that "I want it too" is going to change anything.

I haven't gotten the impression that Chris is quite that tone deaf, or
inconsiderate of the opinions of the community at large.  And the
discussion you are commenting on was about releasing the source code,
not the completed tool.  I don't think anyone is saying that Chris
needs to work harder and finish it faster.  What we are saying is that
it would be better for the progress of btrfs in general if the
development of fsck were done in an open way, and available for others
to contribute to.  The main problem with your statement of "Chris
Mason and other developers..." is that there does not appear to be any
other developers at all.  That's what we'd most like to see changed.

> Ciao,
> --
> Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
> GPG: 03B0 0D6C 0040 0710 4AFA =A0B82F 991B EAAC A599 84C7
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Martin Steigerwald]

Am Mittwoch, 12. Oktober 2011 schrieb Jeff Putney:
> > I do not argue that having a nice fsck sooner than later is fine, but
> > I question the usefulness of repeating reminders. Chris Mason and
> > other developers possibly working on the fsck should know by now,
> > that you want it. So its unlikely that "I want it too" is going to
> > change anything.
> 
> I haven't gotten the impression that Chris is quite that tone deaf, or
> inconsiderate of the opinions of the community at large.  And the
> discussion you are commenting on was about releasing the source code,
> not the completed tool.  I don't think anyone is saying that Chris
> needs to work harder and finish it faster.  What we are saying is that
> it would be better for the progress of btrfs in general if the
> development of fsck were done in an open way, and available for others
> to contribute to.  The main problem with your statement of "Chris
> Mason and other developers..." is that there does not appear to be any
> other developers at all.  That's what we'd most like to see changed.

Fair enough, but once again you are repeating that wish.

I pretty much bet that Chris Mason is aware of this wish of yours and 
other people already and the reasons for that wish. I has been at least 
expressed for a dozen times on this thread.

So again what is another dozen of times trying to achieve?

This all reminds my of childs that ask their parents when they will arrive 
every 10 seconds.

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: Honest timeline for btrfsck

[Jeff Putney]

On Wed, Oct 12, 2011 at 2:53 PM, Martin Steigerwald <Martin@lichtvoll.d=
e> wrote:
> Am Mittwoch, 12. Oktober 2011 schrieb Jeff Putney:
>> > I do not argue that having a nice fsck sooner than later is fine, =
but
>> > I question the usefulness of repeating reminders. Chris Mason and
>> > other developers possibly working on the fsck should know by now,
>> > that you want it. So its unlikely that "I want it too" is going to
>> > change anything.
>>
>> I haven't gotten the impression that Chris is quite that tone deaf, =
or
>> inconsiderate of the opinions of the community at large. =A0And the
>> discussion you are commenting on was about releasing the source code=
,
>> not the completed tool. =A0I don't think anyone is saying that Chris
>> needs to work harder and finish it faster. =A0What we are saying is =
that
>> it would be better for the progress of btrfs in general if the
>> development of fsck were done in an open way, and available for othe=
rs
>> to contribute to. =A0The main problem with your statement of "Chris
>> Mason and other developers..." is that there does not appear to be a=
ny
>> other developers at all. =A0That's what we'd most like to see change=
d.
>
> Fair enough, but once again you are repeating that wish.
>
> I pretty much bet that Chris Mason is aware of this wish of yours and
> other people already and the reasons for that wish. I has been at lea=
st
> expressed for a dozen times on this thread.
>
> So again what is another dozen of times trying to achieve?
>
> This all reminds my of childs that ask their parents when they will a=
rrive
> every 10 seconds.


If your driver keeps telling you that you're going to arrive in 10
seconds, and it takes a child to start asking questions, maybe you
should pay more attention and realize you just might be gettin
shanghaied.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/12/2011 06:47 PM, Jeff Putney wrote:
> On Wed, Oct 12, 2011 at 2:53 PM, Martin Steigerwald
> <Martin@lichtvoll.de> wrote:
>> Am Mittwoch, 12. Oktober 2011 schrieb Jeff Putney:
>>>> I do not argue that having a nice fsck sooner than later is
>>>> fine, but I question the usefulness of repeating reminders.
>>>> Chris Mason and other developers possibly working on the fsck
>>>> should know by now, that you want it. So its unlikely that "I
>>>> want it too" is going to change anything.
>>> 
>>> I haven't gotten the impression that Chris is quite that tone
>>> deaf, or inconsiderate of the opinions of the community at
>>> large.  And the discussion you are commenting on was about
>>> releasing the source code, not the completed tool.  I don't
>>> think anyone is saying that Chris needs to work harder and
>>> finish it faster.  What we are saying is that it would be
>>> better for the progress of btrfs in general if the development
>>> of fsck were done in an open way, and available for others to
>>> contribute to.  The main problem with your statement of "Chris 
>>> Mason and other developers..." is that there does not appear to
>>> be any other developers at all.  That's what we'd most like to
>>> see changed.
>> 
>> Fair enough, but once again you are repeating that wish.
>> 
>> I pretty much bet that Chris Mason is aware of this wish of yours
>> and other people already and the reasons for that wish. I has
>> been at least expressed for a dozen times on this thread.
>> 
>> So again what is another dozen of times trying to achieve?
>> 
>> This all reminds my of childs that ask their parents when they
>> will arrive every 10 seconds.
> 
> 
> If your driver keeps telling you that you're going to arrive in 10 
> seconds, and it takes a child to start asking questions, maybe you 
> should pay more attention and realize you just might be gettin 
> shanghaied.


Are you serious? How much are you paying for that ride?

The first rule of open source software is that those who write the
code pick the features and set the schedule. Unless you're the one
writing the paycheck, your recourse is to write a competitive solution.

I'm with you in wanting an fsck tool, but as an open source developer,
I'm offended by the assertion that you think you have the right to
demand that Chris do anything.

The fact that you view Chris writing one as a deterrent for others
writing a fsck assumes two things: that it is forthcoming and that it
will be of better quality than your fsck solution.

If you think one or both of those things are true, then by all means,
write your own. Write it or stop acting like you have any right to
tell Chris what to do. Seriously.

- -Jeff

- -- 
Jeff Mahoney
SuSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6WfZIACgkQLPWxlyuTD7KdAgCgii9e2AVQ+5MJ4cfKnI7Bumnx
wpIAoJHYyavF27mrNu4Bb+V6kkrgouiq
=Lhzp
-----END PGP SIGNATURE-----

Re: Honest timeline for btrfsck

[Chris Samuel]

[-- Attachment #1: Type: Text/Plain, Size: 1191 bytes --]

On Saturday 08 October 2011 01:48:08 Josef Bacik wrote:

> [...]  Fsck has the
> potential to make any users problems worse, and given the
> increasing number of people putting production systems on btrfs
> with no backups the idea of releasing a unpolished and not fully
> tested fsck into the world is terrifying, and would likely cause
> long term "I heard that file system's fsck tool eats babies" sort
> of reputation.

...and, for instance, it was one of the criticisms levelled at
reiserfs (v3) that its fsck implementation was rather brain dead
in some ways - it couldn't distinguish files contained in an image
of a reiserfs filesystem from those in the reiserfs filesystem that
the image file was in, plus it could discover files from a previous 
reiserfs filesystem after it had been reformatted (as reiserfs).

http://en.wikipedia.org/wiki/ReiserFS#fsck

Chris worked on reiserfs, so he probably doesn't really want to
go through all that again..

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: Honest timeline for btrfsck

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2037 bytes --]

On Thu, Oct 13, 2011 at 10:28:01PM +1100, Chris Samuel wrote:
> On Saturday 08 October 2011 01:48:08 Josef Bacik wrote:
> 
> > [...]  Fsck has the
> > potential to make any users problems worse, and given the
> > increasing number of people putting production systems on btrfs
> > with no backups the idea of releasing a unpolished and not fully
> > tested fsck into the world is terrifying, and would likely cause
> > long term "I heard that file system's fsck tool eats babies" sort
> > of reputation.
> 
> ...and, for instance, it was one of the criticisms levelled at
> reiserfs (v3) that its fsck implementation was rather brain dead
> in some ways - it couldn't distinguish files contained in an image
> of a reiserfs filesystem from those in the reiserfs filesystem that
> the image file was in, plus it could discover files from a previous 
> reiserfs filesystem after it had been reformatted (as reiserfs).
> 
> http://en.wikipedia.org/wiki/ReiserFS#fsck
> 
> Chris worked on reiserfs, so he probably doesn't really want to
> go through all that again..

   Fortunately, that particular problem is not an issue with btrfs, as
it has the UUID of the filesystem in every metadata block. Only if you
manage to take a partial image of the FS and copy it into the FS it's
an image of would you get that particular issue -- and even then, the
clone would have older generation numbers in its blocks than the FS
containing it.

   On the other hand, btrfs does have some new complications of its
own, which lead to additional difficulties. Cloning a btrfs filesystem
with dd, for example, doesn't change the UUID, which can confuse some
of the tools -- but btrfs isn't unique in that (see the recent "Linux
Plumbers" thread).

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- "He's a nutcase, you know. There's no getting away from it -- ---  
                     he'll end up with a knighthood"                     

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Honest timeline for btrfsck

[Francesco Riosa]

2011/10/12 Josef Bacik <josef@redhat.com>:
> On Tue, Oct 11, 2011 at 11:21:45PM +0200, Francesco Riosa wrote:
>> 2011/10/7 Josef Bacik <josef@redhat.com>:
>> > On 10/06/2011 04:56 PM, Francesco Riosa wrote:
>> >> 2011/10/6 Andi Kleen <andi@firstfloor.org>:
>> >>> Jeff Putney <jeffrey.putney@gmail.com> writes:
>> >>>>
>> >>>> http://en.wikipedia.org/wiki/Release_early,_release_often
>> >>>
>> >>> Well the other principle in free software you're forgetting
>> >>> is:
>> >>>
>> >>> "It will be released when it's ready"
>> >>>
>> >>> If you don't like Chris' ways to do releases you're free to writ=
e
>> >>> something on your own or pay someone to do so. Otherwise
>> >>> you just have to deal with his time frames, as shifty
>> >>> as they may be.
>> >>
>> >> I did a different thing, I've offered Chris money to help rescue =
an
>> >> hosed btrfs or to point to someone who could do, we ended in doin=
g
>> >> some tests (for free) but nothing else materialized.
>> >> While the time passed has diminished the value of the data to be
>> >> rescued I'm more on the "show us some code we can start from" tha=
n "it
>> >> will be released when ready" vagon.
>> >>
>> >
>> > If you still need that data, clone this repo
>> >
>> > git://github.com/josefbacik/btrfs-progs.git
>> >
>> > run make, and then run
>> >
>> > ./restore /dev/whatever /some/dir
>> >
>> > and it will try and suck all of your data off the disk and dump it=
 in
>> > that directory. =A0If you have snapshots it will skip them by defa=
ult, so
>> > if you have snapshots that have useful data in them you'll want to=
 use
>> > the -s option. =A0If you run into random errors that you think are
>> > recoverable, or if you don't care about the file that's being reco=
vered,
>> > you can run with -i which will ignore errors and keep trying to re=
cover
>> > your files. =A0Thanks,
>> >
>> > Josef
>> >
>>
>> I've tried, w/o luck
>>
>> explanation come from 2011-06-21 thread;
>> http://thread.gmane.org/gmane.comp.file-systems.btrfs/11435
>> the following refer to a copy of that system
>>
>> Label: space02 =A0uuid: f752def1-1abc-48c7-8ebb-47ba37b8ffa6
>> =A0 =A0 =A0 =A0 Total devices 7 FS bytes used 173.12GB
>> =A0 =A0 =A0 =A0 devid =A0 =A06 size 488.94GB used 60.25GB path /dev/=
sdd7
>> =A0 =A0 =A0 =A0 devid =A0 =A02 size 487.65GB used 58.76GB path /dev/=
sdd8
>> =A0 =A0 =A0 =A0 devid =A0 =A07 size 487.65GB used 0.00 path =A0 =A0/=
dev/sdf7
>> =A0 =A0 =A0 =A0 devid =A0 =A03 size 487.65GB used 60.26GB path /dev/=
sdf8
>> =A0 =A0 =A0 =A0 devid =A0 =A07 size 487.65GB used 1.50GB path =A0/de=
v/sdg7
>> =A0 =A0 =A0 =A0 devid =A0 =A05 size 488.94GB used 58.75GB path /dev/=
sdb7
>> =A0 =A0 =A0 =A0 devid =A0 =A04 size 487.65GB used 60.26GB path /dev/=
sdb8
>>
>> # ./restore /dev/sdd7 /tmp/restore
>> failed to read /dev/sr0
>> failed to read /dev/sr0
>> restore: volumes.c:1367: btrfs_read_sys_array: Assertion `!(ret)' fa=
iled.
>> Aborted
>>
>
> So this is kind of a problem since you have multiple disks. =A0We may=
be could get
> away with ignoring a failure, but the problem is if you have data on =
a disk
> where we couldn't read the chunk then the chances are we won't be abl=
e to map
> that file and read the data off. =A0That being said, theres no harm i=
n trying ;).
> Can you make btrfs_read_sys_array complain about failing, but not act=
ually BUG?
> See if that helps you. =A0Thanks,
>
> Josef
>

I've tried replacing the "BUG_ON(ret);" to printk("FAILED!!! %d\n", ret=
);
the diff of the result is reported at the bottom.

diff -u5  btrfs-progs.log btrfs-progs2.log
--- btrfs-progs.log =A0 =A0 2011-10-11 23:01:56.985577874 +0200+++
btrfs-progs2.log =A0 =A02011-10-13 14:29:48.498033330 +0200@@ -115,11
+98,11 @@
=A0int btrfs_scan_one_device(int fd=3D4, *path=3D/dev/loop5, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D2, u64
super_offset=3D65536)=A0int btrfs_scan_one_device(int fd=3D4,
*path=3D/dev/loop0, struct btrfs_fs_devices **fs_devices_ret, u64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop3, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop2, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop1, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)-int btrfs_scan_one_device(in=
t
fd=3D3, *path=3D/dev/sdd8, struct btrfs_fs_devices **fs_devices_ret, u6=
4
*total_devs=3D139870728973573, u64 super_offset=3D65536)

 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0+int
btrfs_scan_one_device(int fd=3D3, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D139871521856773,
u64 super_offset=3D65536)

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0static int device_list_ad=
d(const char *path,
struct btrfs_super_block *disk_super, u64 devid, struct
btrfs_fs_devices **fs_devices_ret)=A0static struct btrfs_fs_devices
*find_fsid(u8 *fsid)=A0static struct btrfs_device *__find_device(struct
list_head *head, u64 devid, u8 *uuid=A0int btrfs_scan_one_device(int
fd=3D5, *path=3D/dev/sdb8, struct btrfs_fs_devices **fs_devices_ret, u6=
4
*total_devs=3D15306741171905079, u64 super_offset=3D65536)=A0static int
device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)@@
-222,11 +205,40 @@=A0int btrfs_scan_one_device(int fd=3D5,
*path=3D/dev/loop2, struct btrfs_fs_devices **fs_devices_ret, u64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D5, *path=3D/dev/loop1, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_open_devices(str=
uct
btrfs_fs_devices *fs_devices, int flags)=A0int
btrfs_read_sys_array(struct btrfs_root *root)=A0static int
read_one_chunk(struct btrfs_root *root, struct btrfs_key *key,...-
leaf 65536 items 0 free space 3995 generation 604315930624 owner
20979712- =A0fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6- =A0chunk uui=
d
5f424852-6653-5f4d-318e-0b0000000000+--- btrfs_print_tree(root, leaf,
0);+leaf 65536 items 0 free space 3995 generation 604315930624 owner
20979712+fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+FAILED!!! -5+static int read_one_chunk(struct btrfs_root *root,
struct btrfs_key *key,...+--- btrfs_print_tree(root, leaf, 0);+leaf
65536 items 0 free space 3995 generation 604315930624 owner
20979712+fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid+FAILED!!! -5+static int
read_one_chunk(struct btrfs_root *root, struct btrfs_key *key,...+---
btrfs_print_tree(root, leaf, 0);+leaf 65536 items 0 free space 3995
generation 604315930624 owner 20979712+fs uuid
f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid=A0struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,=A0static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid-restore: volumes.c:1397: btrfs_read_sys_array: Assertion
`!(ret)' failed.+FAILED!!! -5+int btrfs_map_block(struct
btrfs_mapping_tree *map_tree, int rw,...+restore: volumes.c:988:
btrfs_map_block: Assertion `!(!ce)' failed.>hom>viv>tmp colordiff -u5
btrfs-progs.log btrfs-progs2.log--- btrfs-progs.log =A0 =A0 2011-10-11
23:01:56.985577874 +0200+++ btrfs-progs2.log =A0 =A02011-10-13
14:29:48.498033330 +0200@@ -1,22 +1,5 @@-Label: space02 =A0uuid:
f752def1-1abc-48c7-8ebb-47ba37b8ffa6- =A0 =A0 =A0 =A0Total devices 7 FS=
 bytes
used 173.12GB- =A0 =A0 =A0 =A0devid =A0 =A06 size 488.94GB used 60.25GB=
 path
/dev/sdd7- =A0 =A0 =A0 =A0devid =A0 =A02 size 487.65GB used 58.76GB pat=
h
/dev/sdd8- =A0 =A0 =A0 =A0devid =A0 =A07 size 487.65GB used 0.00 path
/dev/sdf7- =A0 =A0 =A0 =A0devid =A0 =A03 size 487.65GB used 60.26GB pat=
h
/dev/sdf8- =A0 =A0 =A0 =A0devid =A0 =A07 size 487.65GB used 1.50GB path
/dev/sdg7- =A0 =A0 =A0 =A0devid =A0 =A05 size 488.94GB used 58.75GB pat=
h
/dev/sdb7- =A0 =A0 =A0 =A0devid =A0 =A04 size 487.65GB used 60.26GB pat=
h
/dev/sdb8--# ./restore /dev/sdd7 /tmp/restore-failed to read
/dev/sr0-failed to read /dev/sr0-restore: volumes.c:1367:
btrfs_read_sys_array: Assertion `!(ret)' failed.-Aborted--=A0int
btrfs_scan_one_device(int fd=3D3, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D1, u64
super_offset=3D65536)=A0static int device_list_add(const char *path,
struct btrfs_super_block *disk_super, u64 devid, struct
btrfs_fs_devices **fs_devices_ret)=A0static struct btrfs_fs_devices
*find_fsid(u8 *fsid)=A0int btrfs_scan_one_device(int fd=3D4,
*path=3D/dev/sdb8, struct btrfs_fs_devices **fs_devices_ret, u64
*total_devs=3D134816, u64 super_offset=3D65536)=A0static int
device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)@@
-115,11 +98,11 @@=A0int btrfs_scan_one_device(int fd=3D4,
*path=3D/dev/loop5, struct btrfs_fs_devices **fs_devices_ret, u64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop0, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop3, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop2, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D4, *path=3D/dev/loop1, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)-int btrfs_scan_one_device(in=
t
fd=3D3, *path=3D/dev/sdd8, struct btrfs_fs_devices **fs_devices_ret, u6=
4
*total_devs=3D139870728973573, u64 super_offset=3D65536)

 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0+int
btrfs_scan_one_device(int fd=3D3, *path=3D/dev/sdd8, struct
btrfs_fs_devices **fs_devices_ret, u64 *total_devs=3D139871521856773,
u64 super_offset=3D65536)

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0static int device_list_ad=
d(const char *path,
struct btrfs_super_block *disk_super, u64 devid, struct
btrfs_fs_devices **fs_devices_ret)=A0static struct btrfs_fs_devices
*find_fsid(u8 *fsid)=A0static struct btrfs_device *__find_device(struct
list_head *head, u64 devid, u8 *uuid=A0int btrfs_scan_one_device(int
fd=3D5, *path=3D/dev/sdb8, struct btrfs_fs_devices **fs_devices_ret, u6=
4
*total_devs=3D15306741171905079, u64 super_offset=3D65536)=A0static int
device_list_add(const char *path, struct btrfs_super_block
*disk_super, u64 devid, struct btrfs_fs_devices **fs_devices_ret)@@
-222,11 +205,40 @@=A0int btrfs_scan_one_device(int fd=3D5,
*path=3D/dev/loop2, struct btrfs_fs_devices **fs_devices_ret, u64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_scan_one_device(=
int
fd=3D5, *path=3D/dev/loop1, struct btrfs_fs_devices **fs_devices_ret, u=
64
*total_devs=3D2, u64 super_offset=3D65536)=A0int btrfs_open_devices(str=
uct
btrfs_fs_devices *fs_devices, int flags)=A0int
btrfs_read_sys_array(struct btrfs_root *root)=A0static int
read_one_chunk(struct btrfs_root *root, struct btrfs_key *key,...-
leaf 65536 items 0 free space 3995 generation 604315930624 owner
20979712- =A0fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6- =A0chunk uui=
d
5f424852-6653-5f4d-318e-0b0000000000+--- btrfs_print_tree(root, leaf,
0);+leaf 65536 items 0 free space 3995 generation 604315930624 owner
20979712+fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+FAILED!!! -5+static int read_one_chunk(struct btrfs_root *root,
struct btrfs_key *key,...+--- btrfs_print_tree(root, leaf, 0);+leaf
65536 items 0 free space 3995 generation 604315930624 owner
20979712+fs uuid f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid+FAILED!!! -5+static int
read_one_chunk(struct btrfs_root *root, struct btrfs_key *key,...+---
btrfs_print_tree(root, leaf, 0);+leaf 65536 items 0 free space 3995
generation 604315930624 owner 20979712+fs uuid
f752def1-1abc-48c7-8ebb-47ba37b8ffa6+chunk uuid
5f424852-6653-5f4d-318e-0b0000000000+struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,+static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid+struct btrfs_device *btrfs_find_device(struct btrfs_root *root,
u64 devid,+static struct btrfs_device *__find_device(struct list_head
*head, u64 devid, u8 *uuid=A0struct btrfs_device
*btrfs_find_device(struct btrfs_root *root, u64 devid,=A0static struct
btrfs_device *__find_device(struct list_head *head, u64 devid, u8
*uuid-restore: volumes.c:1397: btrfs_read_sys_array: Assertion
`!(ret)' failed.+FAILED!!! -5+int btrfs_map_block(struct
btrfs_mapping_tree *map_tree, int rw,...+restore: volumes.c:988:
btrfs_map_block: Assertion `!(!ce)' failed.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Josef Bacik]

On Thu, Oct 13, 2011 at 02:57:01PM +0200, Francesco Riosa wrote:
> 2011/10/12 Josef Bacik <josef@redhat.com>:
> > On Tue, Oct 11, 2011 at 11:21:45PM +0200, Francesco Riosa wrote:
> >> 2011/10/7 Josef Bacik <josef@redhat.com>:
> >> > On 10/06/2011 04:56 PM, Francesco Riosa wrote:
> >> >> 2011/10/6 Andi Kleen <andi@firstfloor.org>:
> >> >>> Jeff Putney <jeffrey.putney@gmail.com> writes:
> >> >>>>
> >> >>>> http://en.wikipedia.org/wiki/Release_early,_release_often
> >> >>>
> >> >>> Well the other principle in free software you're forgetting
> >> >>> is:
> >> >>>
> >> >>> "It will be released when it's ready"
> >> >>>
> >> >>> If you don't like Chris' ways to do releases you're free to wr=
ite
> >> >>> something on your own or pay someone to do so. Otherwise
> >> >>> you just have to deal with his time frames, as shifty
> >> >>> as they may be.
> >> >>
> >> >> I did a different thing, I've offered Chris money to help rescu=
e an
> >> >> hosed btrfs or to point to someone who could do, we ended in do=
ing
> >> >> some tests (for free) but nothing else materialized.
> >> >> While the time passed has diminished the value of the data to b=
e
> >> >> rescued I'm more on the "show us some code we can start from" t=
han "it
> >> >> will be released when ready" vagon.
> >> >>
> >> >
> >> > If you still need that data, clone this repo
> >> >
> >> > git://github.com/josefbacik/btrfs-progs.git
> >> >
> >> > run make, and then run
> >> >
> >> > ./restore /dev/whatever /some/dir
> >> >
> >> > and it will try and suck all of your data off the disk and dump =
it in
> >> > that directory. =A0If you have snapshots it will skip them by de=
fault, so
> >> > if you have snapshots that have useful data in them you'll want =
to use
> >> > the -s option. =A0If you run into random errors that you think a=
re
> >> > recoverable, or if you don't care about the file that's being re=
covered,
> >> > you can run with -i which will ignore errors and keep trying to =
recover
> >> > your files. =A0Thanks,
> >> >
> >> > Josef
> >> >
> >>
> >> I've tried, w/o luck
> >>
> >> explanation come from 2011-06-21 thread;
> >> http://thread.gmane.org/gmane.comp.file-systems.btrfs/11435
> >> the following refer to a copy of that system
> >>
> >> Label: space02 =A0uuid: f752def1-1abc-48c7-8ebb-47ba37b8ffa6
> >> =A0 =A0 =A0 =A0 Total devices 7 FS bytes used 173.12GB
> >> =A0 =A0 =A0 =A0 devid =A0 =A06 size 488.94GB used 60.25GB path /de=
v/sdd7
> >> =A0 =A0 =A0 =A0 devid =A0 =A02 size 487.65GB used 58.76GB path /de=
v/sdd8
> >> =A0 =A0 =A0 =A0 devid =A0 =A07 size 487.65GB used 0.00 path =A0 =A0=
/dev/sdf7
> >> =A0 =A0 =A0 =A0 devid =A0 =A03 size 487.65GB used 60.26GB path /de=
v/sdf8
> >> =A0 =A0 =A0 =A0 devid =A0 =A07 size 487.65GB used 1.50GB path =A0/=
dev/sdg7
> >> =A0 =A0 =A0 =A0 devid =A0 =A05 size 488.94GB used 58.75GB path /de=
v/sdb7
> >> =A0 =A0 =A0 =A0 devid =A0 =A04 size 487.65GB used 60.26GB path /de=
v/sdb8
> >>
> >> # ./restore /dev/sdd7 /tmp/restore
> >> failed to read /dev/sr0
> >> failed to read /dev/sr0
> >> restore: volumes.c:1367: btrfs_read_sys_array: Assertion `!(ret)' =
failed.
> >> Aborted
> >>
> >
> > So this is kind of a problem since you have multiple disks. =A0We m=
aybe could get
> > away with ignoring a failure, but the problem is if you have data o=
n a disk
> > where we couldn't read the chunk then the chances are we won't be a=
ble to map
> > that file and read the data off. =A0That being said, theres no harm=
 in trying ;).
> > Can you make btrfs_read_sys_array complain about failing, but not a=
ctually BUG?
> > See if that helps you. =A0Thanks,
> >
> > Josef
> >
>=20
> I've tried replacing the "BUG_ON(ret);" to printk("FAILED!!! %d\n", r=
et);
> the diff of the result is reported at the bottom.
>=20

Ok so this is a little trickier, your chunk tree is screwed up.  We nee=
d that to
be intact so we can translate the logical block addresses to physical a=
ddresses,
without that we're screwed because we have no way of knowing where anyt=
hing is.
I'm working on a tool to try and find root items, but currently it also=
 requires
having a working chunk tree.  Once I get finished making it work on a f=
ile
system with an intact chunk tree I'll try and figure out something for
rebuilding a chunk tree.  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

>> If your driver keeps telling you that you're going to arrive in 10
>> seconds, and it takes a child to start asking questions, maybe you
>> should pay more attention and realize you just might be gettin
>> shanghaied.
>
>
> Are you serious? How much are you paying for that ride?
>

Not really, I was trying to deflect a little ad hominem with some
levity, and pointing out some of the shortcomings of the metaphor.

> The first rule of open source software is that those who write the
> code pick the features and set the schedule.

We are talking about presently closed source and trying to get it to
be open source.  The first rule of closed source is those who own the
code get to pick who is going to write the code who pick the features
and set the schedule.

Anyone can make up an axiom and call it the first rule of something.
It doesn't make it so.

> Unless you're the one
> writing the paycheck, your recourse is to write a competitive solution.
>
> I'm with you in wanting an fsck tool, but as an open source developer,
> I'm offended by the assertion that you think you have the right to
> demand that Chris do anything.

Be  offended all you want, but characterizing people discussing how to
best change the status quo, as some sort of demand is your
contribution to the discussion, and doesn't reflect the tenor of the
conversation.

> The fact that you view Chris writing one as a deterrent for others
> writing a fsck assumes two things: that it is forthcoming and that it
> will be of better quality than your fsck solution.

No, it is the *belief* that the tool will be forthcoming that has a
deterring effect.  The tool will arrive, or not, regardless of this
belief.  It is precisely this belief that I think needs further
consideration given the past year of repeated estimates.

> If you think one or both of those things are true, then by all means,
> write your own. Write it or stop acting like you have any right to
> tell Chris what to do. Seriously.

You are the first person I recall telling anyone what to do, or making
demands in this thread.  Seriously.

Should the tool, and or source not be forthcoming, then this is
exactly what I am proposing be done.  There are very few people that
could write such a tool by themselves, and we'd still be in the same
boat of having a single point of failure.  So, I think the best
approach would be a multiple people contributing to a real open source
effort.  This type of effort is not going to be able to get underway
so long as their is a pervasive belief in the eminent release of
Chris' tool.


--jeff

> - -Jeff

>
> - --
> Jeff Mahoney
> SuSE Labs
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.18 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk6WfZIACgkQLPWxlyuTD7KdAgCgii9e2AVQ+5MJ4cfKnI7Bumnx
> wpIAoJHYyavF27mrNu4Bb+V6kkrgouiq
> =Lhzp
> -----END PGP SIGNATURE-----
>

Re: Honest timeline for btrfsck

[Chris Samuel]

[-- Attachment #1: Type: Text/Plain, Size: 767 bytes --]

On Fri, 14 Oct 2011 02:51:26 AM Jeff Putney wrote:

> Should the tool, and or source not be forthcoming, then this is
> exactly what I am proposing be done.

I'd suggest that you don't wait and instead make a start on it,
if for nothing else than so when Chris's version does appear there
is a way to merge the best of both together.

Even if all your tool has is a friendlier interface or better
help then that'll still be a useful contribution to the effort,
and if it's got more than that then you'll have made even more
of a contribution.

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: Honest timeline for btrfsck

[David Summers]

On 05/10/11 07:16, Chris Mason wrote:
>
> So over the next two weeks I'm juggling the merge window and the fsck
> release.  My goal is to demo fsck at linuxcon europe.  Thanks again for
> all of your patience and help with Btrfs!
>
Any chance of a copy of your talk at linuxcon? ;)

David.

Re: Honest timeline for btrfsck

[Clemens Eisserer]

Any update on the state of btrfschk?

Thanks, Clemens

2011/10/31 David Summers <btrfs@summers5913.freeserve.co.uk>:
> On 05/10/11 07:16, Chris Mason wrote:
>>
>>
>> So over the next two weeks I'm juggling the merge window and the fsc=
k
>> release. =C2=A0My goal is to demo fsck at linuxcon europe. =C2=A0Tha=
nks again for
>> all of your patience and help with Btrfs!
>>
> Any chance of a copy of your talk at linuxcon? ;)
>
> David.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht=
ml
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Jeff Putney]

Or, better yet, how about just releasing the source code already.


On Wed, Nov 30, 2011 at 4:19 AM, Clemens Eisserer <linuxhippy@gmail.com=
> wrot
> Any update on the state of btrfschk?
>
> Thanks, Clemens
>
> 2011/10/31 David Summers <btrfs@summers5913.freeserve.co.uk>:
>> On 05/10/11 07:16, Chris Mason wrote:
>>>
>>>
>>> So over the next two weeks I'm juggling the merge window and the fs=
ck
>>> release. =A0My goal is to demo fsck at linuxcon europe. =A0Thanks a=
gain for
>>> all of your patience and help with Btrfs!
>>>
>> Any chance of a copy of your talk at linuxcon? ;)
>>
>> David.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrf=
s" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[Danny Piccirillo]

Chris Mason <chris.mason <at> oracle.com> writes:

>=20
> So over the next two weeks I'm juggling the merge window and the fsck
> release.  My goal is to demo fsck at linuxcon europe.  Thanks again f=
or
> all of your patience and help with Btrfs!
>=20

So we have a lot of new features which is awesome but still not enough =
for
production use (I'm hoping that Fedora will actually be able to ship wi=
th btrfs
this time).

What's new? What are the next goals and hopeful deadlines?=20

Best,
=2Edanny

--=20
=E2=98=AE =E2=99=A5 =E2=92=B6
     .danny

This email is: [ x ] bloggable   [  ] shareable with consent   [  ] let=
hal if
repeated or forwarded

[=F0=9D=84=BD#] The Silent Number - http://thesilentnumber.me/

=C2=B5Blog: http://identi.ca/dpic

--------------------------------------------
Q: Why is this email five sentences or less?
A: http://five.sentenc.es

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Honest timeline for btrfsck

[David Summers]

On 18/08/11 21:50, Chris Mason wrote:
> Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400:
>> Chris Mason<chris.mason<at>  oracle.com>  writes:
>>
>>>
>>> Aside from making sure the kernel code is stable, btrfsck is all I'm
>>> working on right now.  I do expect a release in the next two weeks that
>>> can recover your data (and many others).
>>>
>>> Thanks,
>>> Chris
>>> --
>>
>>
>> Chris,
>>
>> We're all on the edge of our seats.  Can you provide an updated ETA on the
>> release of the first functional btrfsck tool?  No pressure or anything ;)
>
> Hi everyone,
>
> I've been working non-stop on this.  Currently fsck has four parts:
>
> 1) mount -o recovery mode.  I've posted smaller forms of these patches
> in the past that bypass log tree replay.  The new versions have code to
> create stub roots for trees that can't be read (like the extent
> allocation tree) and will allow the mount to proceed.
>
> 2) fsck that scans for older roots.  This takes advantage of older
> copies of metadata to look for consistent tree roots on disk.  The
> downside is that it is currently very slow.  I'm trying to speed it up
> by limiting the search to only the metadata block groups and a few other
> tricks.
>
> 3) fsck that fixes the extent allocation tree and the chunk tree.  This
> is where I've been spending most of my time.  The problem is that it
> tends to recover some filesystems and badly break others.  While I'm
> fixing up the corner cases that work poorly, I'm adding an undo log to
> the fsck code so that you can get the FS back into its original state if
> you don't like the result of the fsck.
>
> 4) The rest of the corruptions can be dealt with fairly well from the
> kernel.  I have a series of patches to make the extent allocation tree
> less strict about reference counts and other rules, basically allowing
> the FS to limp along instead of crash.
>
> These four things together are basically my minimal set of features
> required for fedora and our own internal projects at Oracle to start
> treating us as production filesystem.
>
> There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> hoped to get #1 out the door before I left on vacation and I still might
> post it tonight.
>
> -chris
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Just checking my reading on where we are is correct.

1&2 have been done?

Whats the progress on 3&4 - is Chris the only one working on these, or 
are others active?

David.

Re: Honest timeline for btrfsck

[Chris Mason]

On Tue, Jan 17, 2012 at 03:07:16PM +0000, David Summers wrote:
> On 18/08/11 21:50, Chris Mason wrote:
> >Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400:
> >>Chris Mason<chris.mason<at>  oracle.com>  writes:
> >>
> >>>
> >>>Aside from making sure the kernel code is stable, btrfsck is all I'm
> >>>working on right now.  I do expect a release in the next two weeks that
> >>>can recover your data (and many others).
> >>>
> >>>Thanks,
> >>>Chris
> >>>--
> >>
> >>
> >>Chris,
> >>
> >>We're all on the edge of our seats.  Can you provide an updated ETA on the
> >>release of the first functional btrfsck tool?  No pressure or anything ;)
> >
> >Hi everyone,
> >
> >I've been working non-stop on this.  Currently fsck has four parts:
> >
> >1) mount -o recovery mode.  I've posted smaller forms of these patches
> >in the past that bypass log tree replay.  The new versions have code to
> >create stub roots for trees that can't be read (like the extent
> >allocation tree) and will allow the mount to proceed.
> >
> >2) fsck that scans for older roots.  This takes advantage of older
> >copies of metadata to look for consistent tree roots on disk.  The
> >downside is that it is currently very slow.  I'm trying to speed it up
> >by limiting the search to only the metadata block groups and a few other
> >tricks.
> >
> >3) fsck that fixes the extent allocation tree and the chunk tree.  This
> >is where I've been spending most of my time.  The problem is that it
> >tends to recover some filesystems and badly break others.  While I'm
> >fixing up the corner cases that work poorly, I'm adding an undo log to
> >the fsck code so that you can get the FS back into its original state if
> >you don't like the result of the fsck.
> >
> >4) The rest of the corruptions can be dealt with fairly well from the
> >kernel.  I have a series of patches to make the extent allocation tree
> >less strict about reference counts and other rules, basically allowing
> >the FS to limp along instead of crash.
> >
> >These four things together are basically my minimal set of features
> >required for fedora and our own internal projects at Oracle to start
> >treating us as production filesystem.
> >
> >There are always bugs to fix, and I have #1 and #2 mostly ready.  I had
> >hoped to get #1 out the door before I left on vacation and I still might
> >post it tonight.
> >
> 
> Just checking my reading on where we are is correct.
> 
> 1&2 have been done?
> 
> Whats the progress on 3&4 - is Chris the only one working on these,
> or are others active?

People have already started picking up #4, fujitsu had some patches in
this direction that we'll keep developing with.

I stepped back to add some directory metadata fixups as well to the
basic fsck tool.  I had thought I could easily do it all from the
kernel, but sometimes the userland side really is easier.

-chris

Re: Honest timeline for btrfsck

[Danny Piccirillo]

Chris Mason <chris.mason <at> oracle.com> writes:
> 
> People have already started picking up #4, fujitsu had some patches in
> this direction that we'll keep developing with.
> 
> I stepped back to add some directory metadata fixups as well to the
> basic fsck tool.  I had thought I could easily do it all from the
> kernel, but sometimes the userland side really is easier.


Phoronix reported on your talk at the SCALE 10x conference here 
http://www.phoronix.com/scan.php?page=news_item&px=MTA0Njk


What happened to the February 14th deadline? Is another rewrite 
underway? I think the biggest thing that can be done in lieu of the release
is really good communication. There's no feed to get up-to-date info on 
what's being worked on, what's left, etc. 


Looking forward to this! 
.danny

Re: Honest timeline for btrfsck

[Duncan]

Danny Piccirillo posted on Wed, 28 Mar 2012 06:15:41 +0000 as excerpted:

> Chris Mason <chris.mason <at> oracle.com> writes:
>> 
>> People have already started picking up #4, fujitsu had some patches in
>> this direction that we'll keep developing with.
>> 
>> I stepped back to add some directory metadata fixups as well to the
>> basic fsck tool.  I had thought I could easily do it all from the
>> kernel, but sometimes the userland side really is easier.
> 
> 
> Phoronix reported on your talk at the SCALE 10x conference here
> http://www.phoronix.com/scan.php?page=news_item&px=MTA0Njk
> 
> 
> What happened to the February 14th deadline? Is another rewrite
> underway? I think the biggest thing that can be done in lieu of the
> release is really good communication. There's no feed to get up-to-date
> info on what's being worked on, what's left, etc.

I too read the phoronix article, and in fact mentioned it here back about 
Feb 10 or so...

Actually, a somewhat-working repair-writing btrfsck does exist and is in 
fact "public"... for some value of "public", at least.  It's still sort 
of like the plans for the pan-galactic bypass in hitchhiker's guide to 
the galaxy, down several flights of stairs behind a door that says 
"beware of leopard", in that it's only available in Chris's 
"dangerdonteveruse" git branch, but it *IS* publicly available... as I 
said for /some/ value of "public" anyway.

Here's the issue, tho.  At present, it's pretty much a "well, it may fix 
some problems, but then again, it might wreak further havoc on an already 
damaged filesystem, destroying any reasonable chance of retrieving valid 
data off it, too!" type of situation.  That's why it's still stuck in 
"dangerdonteveruse".

But if you read the article well, that's all that was really promised 
anyway, at this point.  That was a deadline for Oracle getting it for 
testing and QA, etc.  It was *NOT* a "release quality" deadline, or even 
a "this matches the experimental state of the filesystem" deadline, in 
any shape or form.

So Oracle and others are doing some seriously focused QA and bug fixes on 
the thing at present.  The idea is that they can release it along with a 
kernel with their own patches, and support that.  It's worth noting, 
however, that if they're only supporting their own kernel, they can, if 
necessary, disable certain already available features of the filesystem 
in their kernel, AND in the btrfsck, so as to be able to support what's 
left.  After all, the filesystem itself is still officially experimental 
and in disk-format-flux, as well as lacking various targeted features 
ATM, and it's only a small stretch to go from there to disabling a 
feature or two already in mainline, if necessary to be able to be 
comfortable supporting the rest.


Bottom line:  Purely as a simple Linux user following this list for 
perhaps a couple months now, I'd expect btrfs to continue maturing and 
that it'll probably getting toward somewhat stable toward the end of the 
year... depending on how conservative you are, of course (there's many 
that are barely warming up to ext4 at this point).  Which would put it in 
line for the spring of next year (2013) distro releases.  Until then, I'd 
expect the current label, experimental, fit only for testing, not for 
storage of data you expect to be able to reliably retrieve, to continue 
to apply, tho to a lessor degree as it gradually matures.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman