About btrfs chunk tree backups

About btrfs chunk tree backups

[Miao Xie]

Hi, Chris and Josef

Recently, I am interested in chunk tree backups, which is described in
Project ideas web. But I found it is hard to be implemented.

The description said "make the mappings discoverable via a block device
scan so that we can recover from corrupted chunk trees", I think it is
hard to find the boundary of each stripe, which is the constituent of
the chunks, via a block device scan according to the current on-disk data
structures. And besides that, even though we find the stripes of each
chunks, we still don't know the order of the stripes in one chunk. So
I think chunk tree backups is hard to be implemented.

Maybe I made some mistakes in understanding this project idea?

Thanks
Miao

Re: About btrfs chunk tree backups

[Josef Bacik]

On Thu, Mar 03, 2011 at 05:01:00PM +0800, Miao Xie wrote:
> Hi, Chris and Josef
> 
> Recently, I am interested in chunk tree backups, which is described in
> Project ideas web. But I found it is hard to be implemented.
> 
> The description said "make the mappings discoverable via a block device
> scan so that we can recover from corrupted chunk trees", I think it is
> hard to find the boundary of each stripe, which is the constituent of
> the chunks, via a block device scan according to the current on-disk data
> structures. And besides that, even though we find the stripes of each
> chunks, we still don't know the order of the stripes in one chunk. So
> I think chunk tree backups is hard to be implemented.
> 
> Maybe I made some mistakes in understanding this project idea?
>

We talked about this some earlier this week.  This will require a format change,
basically we need to put down some metadata at the start of every chunk so that
we can scan the disk and find the chunks.  This could probably be done in a
backwards compatible way, we just need to make sure the space for the block
thats used is taken up in the extent tree so old kernels don't write over the
block.  Thanks,

Josef 

Re: About btrfs chunk tree backups

[Miao Xie]

On Thu, 3 Mar 2011 07:37:25 -0500, Josef Bacik wrote:
> On Thu, Mar 03, 2011 at 05:01:00PM +0800, Miao Xie wrote:
>> Hi, Chris and Josef
>>
>> Recently, I am interested in chunk tree backups, which is described in
>> Project ideas web. But I found it is hard to be implemented.
>>
>> The description said "make the mappings discoverable via a block device
>> scan so that we can recover from corrupted chunk trees", I think it is
>> hard to find the boundary of each stripe, which is the constituent of
>> the chunks, via a block device scan according to the current on-disk data
>> structures. And besides that, even though we find the stripes of each
>> chunks, we still don't know the order of the stripes in one chunk. So
>> I think chunk tree backups is hard to be implemented.
>>
>> Maybe I made some mistakes in understanding this project idea?
>>
> 
> We talked about this some earlier this week.  This will require a format change,
> basically we need to put down some metadata at the start of every chunk so that
> we can scan the disk and find the chunks.  This could probably be done in a
> backwards compatible way, we just need to make sure the space for the block
> thats used is taken up in the extent tree so old kernels don't write over the
> block.  Thanks,

Hi, Josef

Is anyone implementing it now? Maybe we can help him to do some work.

Thanks
Miao

> 
> Josef 
> 

Re: About btrfs chunk tree backups

[Josef Bacik]

On Tue, Mar 08, 2011 at 12:14:05PM +0800, Miao Xie wrote:
> On Thu, 3 Mar 2011 07:37:25 -0500, Josef Bacik wrote:
> > On Thu, Mar 03, 2011 at 05:01:00PM +0800, Miao Xie wrote:
> >> Hi, Chris and Josef
> >>
> >> Recently, I am interested in chunk tree backups, which is described in
> >> Project ideas web. But I found it is hard to be implemented.
> >>
> >> The description said "make the mappings discoverable via a block device
> >> scan so that we can recover from corrupted chunk trees", I think it is
> >> hard to find the boundary of each stripe, which is the constituent of
> >> the chunks, via a block device scan according to the current on-disk data
> >> structures. And besides that, even though we find the stripes of each
> >> chunks, we still don't know the order of the stripes in one chunk. So
> >> I think chunk tree backups is hard to be implemented.
> >>
> >> Maybe I made some mistakes in understanding this project idea?
> >>
> > 
> > We talked about this some earlier this week.  This will require a format change,
> > basically we need to put down some metadata at the start of every chunk so that
> > we can scan the disk and find the chunks.  This could probably be done in a
> > backwards compatible way, we just need to make sure the space for the block
> > thats used is taken up in the extent tree so old kernels don't write over the
> > block.  Thanks,
> 
> Hi, Josef
> 
> Is anyone implementing it now? Maybe we can help him to do some work.
> 

Not that I know of.  Thanks,

Josef