[PATCH 1/2] Btrfs: add datacow flag in inode flag

[PATCH 1/2] Btrfs: add datacow flag in inode flag

[liubo]

For datacow control, the corresponding inode flags are needed.
This is for the following patch.

Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
---
 include/linux/fs.h |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 63d069b..bef47ff 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -353,6 +353,8 @@ struct inodes_stat_t {
 #define FS_TOPDIR_FL			0x00020000 /* Top of directory hierarchies*/
 #define FS_EXTENT_FL			0x00080000 /* Extents */
 #define FS_DIRECTIO_FL			0x00100000 /* Use direct i/o */
+#define FS_NOCOW_FL			0x00800000 /* Do not cow file */
+#define FS_COW_FL			0x01000000 /* Cow file */
 #define FS_RESERVED_FL			0x80000000 /* reserved for ext2 lib */
 
 #define FS_FL_USER_VISIBLE		0x0003DFFF /* User visible flags */
-- 
1.6.5.2

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Chris Mason]

Excerpts from liubo's message of 2011-03-03 03:35:42 -0500:
> 
> For datacow control, the corresponding inode flags are needed.
> This is for the following patch.
> 
> Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
> ---
>  include/linux/fs.h |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 63d069b..bef47ff 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -353,6 +353,8 @@ struct inodes_stat_t {
>  #define FS_TOPDIR_FL            0x00020000 /* Top of directory hierarchies*/
>  #define FS_EXTENT_FL            0x00080000 /* Extents */
>  #define FS_DIRECTIO_FL            0x00100000 /* Use direct i/o */
> +#define FS_NOCOW_FL            0x00800000 /* Do not cow file */
> +#define FS_COW_FL            0x01000000 /* Cow file */
>  #define FS_RESERVED_FL            0x80000000 /* reserved for ext2 lib */
>  
>  #define FS_FL_USER_VISIBLE        0x0003DFFF /* User visible flags */

Hi everyone,

I'd like to go ahead and include these for btrfs to use.  Are there
objections or a different preferred interface?

-chris

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Christoph Hellwig]

On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
> Excerpts from liubo's message of 2011-03-03 03:35:42 -0500:

I'm fine with it.  I'll defer the check for conflicts with extN-specific flags
to Ted, though.

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Andreas Dilger]

On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
> On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
>>  #define FS_EXTENT_FL         0x00080000 /* Extents */
>>  #define FS_DIRECTIO_FL       0x00100000 /* Use direct i/o */
>> +#define FS_NOCOW_FL          0x00800000 /* Do not cow file */
>> +#define FS_COW_FL            0x01000000 /* Cow file */
>>  #define FS_RESERVED_FL       0x80000000 /* reserved for ext2 lib */
> 
> I'm fine with it.  I'll defer the check for conflicts with extN-specific flags
> to Ted, though.

Looking at the upstream e2fsprogs I see in that range:

> #define EXT4_EXTENTS_FL           0x00080000 /* Inode uses extents */
> #define EXT4_EA_INODE_FL          0x00200000 /* Inode used for large EA */
> #define EXT4_EOFBLOCKS_FL         0x00400000 /* Blocks allocated beyond EOF */
> #define EXT4_SNAPFILE_FL          0x01000000 /* Inode is a snapshot */
> #define EXT4_SNAPFILE_DELETED_FL  0x04000000 /* Snapshot is being deleted */
> #define EXT4_SNAPFILE_SHRUNK_FL   0x08000000 /* Snapshot shrink has completed */
> #define EXT2_RESERVED_FL          0x80000000 /* reserved for ext2 lib */
> 
> #define EXT2_FL_USER_VISIBLE      0x004BDFFF /* User visible flags */

so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL.  I don't know the semantics of those two flags enough to say for sure whether it is reasonable that they alias to each other, but at first glance "COW" and "SNAPSHOT" don't seem completely unrelated.

It also isn't clear to me whether the SNAPFILE_DELETED_FL and SNAPFILE_SHRUNK_FL really need to be persistent flags that are stored on disk, or if they are state flags that should only be stored in the in-memory inode?  Not having them in the limited remaining inode flag space would be better...  Amir?

Cheers, Andreas

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Chris Mason]

Excerpts from Andreas Dilger's message of 2011-03-15 18:06:49 -0400:
> On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
> > On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
> >>  #define FS_EXTENT_FL         0x00080000 /* Extents */
> >>  #define FS_DIRECTIO_FL       0x00100000 /* Use direct i/o */
> >> +#define FS_NOCOW_FL          0x00800000 /* Do not cow file */
> >> +#define FS_COW_FL            0x01000000 /* Cow file */
> >>  #define FS_RESERVED_FL       0x80000000 /* reserved for ext2 lib */
> > 
> > I'm fine with it.  I'll defer the check for conflicts with extN-specific flags
> > to Ted, though.
> 
> Looking at the upstream e2fsprogs I see in that range:
> 
> > #define EXT4_EXTENTS_FL           0x00080000 /* Inode uses extents */
> > #define EXT4_EA_INODE_FL          0x00200000 /* Inode used for large EA */
> > #define EXT4_EOFBLOCKS_FL         0x00400000 /* Blocks allocated beyond EOF */
> > #define EXT4_SNAPFILE_FL          0x01000000 /* Inode is a snapshot */
> > #define EXT4_SNAPFILE_DELETED_FL  0x04000000 /* Snapshot is being deleted */
> > #define EXT4_SNAPFILE_SHRUNK_FL   0x08000000 /* Snapshot shrink has completed */
> > #define EXT2_RESERVED_FL          0x80000000 /* reserved for ext2 lib */
> > 
> > #define EXT2_FL_USER_VISIBLE      0x004BDFFF /* User visible flags */
> 
> so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL.  I don't know the semantics of those two flags enough to say for sure whether it is reasonable that they alias to each other, but at first glance "COW" and "SNAPSHOT" don't seem completely unrelated.

In the btrfs case FS_COW_FL means to do COW even when there are no
snapshots.  FS_NOCOW_FL means to do cow only when there are snapshots.

-chris

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Amir Goldstein]

On Wed, Mar 16, 2011 at 1:35 AM, Chris Mason <chris.mason@oracle.com> w=
rote:
> Excerpts from Andreas Dilger's message of 2011-03-15 18:06:49 -0400:
>> On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
>> > On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
>> >> =A0#define FS_EXTENT_FL =A0 =A0 =A0 =A0 0x00080000 /* Extents */
>> >> =A0#define FS_DIRECTIO_FL =A0 =A0 =A0 0x00100000 /* Use direct i/=
o */
>> >> +#define FS_NOCOW_FL =A0 =A0 =A0 =A0 =A00x00800000 /* Do not cow =
file */
>> >> +#define FS_COW_FL =A0 =A0 =A0 =A0 =A0 =A00x01000000 /* Cow file =
*/
>> >> =A0#define FS_RESERVED_FL =A0 =A0 =A0 0x80000000 /* reserved for =
ext2 lib */
>> >
>> > I'm fine with it. =A0I'll defer the check for conflicts with extN-=
specific flags
>> > to Ted, though.
>>
>> Looking at the upstream e2fsprogs I see in that range:
>>
>> > #define EXT4_EXTENTS_FL =A0 =A0 =A0 =A0 =A0 0x00080000 /* Inode us=
es extents */
>> > #define EXT4_EA_INODE_FL =A0 =A0 =A0 =A0 =A00x00200000 /* Inode us=
ed for large EA */
>> > #define EXT4_EOFBLOCKS_FL =A0 =A0 =A0 =A0 0x00400000 /* Blocks all=
ocated beyond EOF */
>> > #define EXT4_SNAPFILE_FL =A0 =A0 =A0 =A0 =A00x01000000 /* Inode is=
 a snapshot */
>> > #define EXT4_SNAPFILE_DELETED_FL =A00x04000000 /* Snapshot is bein=
g deleted */
>> > #define EXT4_SNAPFILE_SHRUNK_FL =A0 0x08000000 /* Snapshot shrink =
has completed */
>> > #define EXT2_RESERVED_FL =A0 =A0 =A0 =A0 =A00x80000000 /* reserved=
 for ext2 lib */
>> >
>> > #define EXT2_FL_USER_VISIBLE =A0 =A0 =A00x004BDFFF /* User visible=
 flags */
>>
>> so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL. =A0I don=
't know the semantics of those two flags enough to say for sure whether=
 it is reasonable that they alias to each other, but at first glance "C=
OW" and "SNAPSHOT" don't seem completely unrelated.

EXT4_SNAPFILE_FL indicates a special system snapshot file, so it has
no equivalence relation with FS_COW_FL.
Please use 0x02000000 for FS_COW_FL.

EXT4_SNAPFILE_DELETED_FL is a persistent state of a snapshot file,
which is no longer
available as a mountable device, but cannot be unlinked because it
holds changed data sets
needed by older snapshots.

EXT4_SNAPFILE_SHRUNK_FL is a persistent state of a (deleted) snapshot
file, which has
undergone a "shrink" process to free all change sets not needed by
older snapshots.
The persistence of the flag is needed to avoid tedious shrinking when
it is not needed.


>
> In the btrfs case FS_COW_FL means to do COW even when there are no
> snapshots. =A0FS_NOCOW_FL means to do cow only when there are snapsho=
ts.
>

I am interested in FS_NOCOW_FL as well, but for my implementation it wo=
uld mean
do not do COW on rewrites even when there are snapshots, so a user can
create a pre-allocated
"island of blocks", which are pinned to a physical location, for raw
VM image for example.


Thanks,
Amir.
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel=
" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[liubo]

On 03/16/2011 05:06 PM, Amir Goldstein wrote:
> On Wed, Mar 16, 2011 at 1:35 AM, Chris Mason <chris.mason@oracle.com> wrote:
>> Excerpts from Andreas Dilger's message of 2011-03-15 18:06:49 -0400:
>>> On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
>>>> On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
>>>>>  #define FS_EXTENT_FL         0x00080000 /* Extents */
>>>>>  #define FS_DIRECTIO_FL       0x00100000 /* Use direct i/o */
>>>>> +#define FS_NOCOW_FL          0x00800000 /* Do not cow file */
>>>>> +#define FS_COW_FL            0x01000000 /* Cow file */
>>>>>  #define FS_RESERVED_FL       0x80000000 /* reserved for ext2 lib */
>>>> I'm fine with it.  I'll defer the check for conflicts with extN-specific flags
>>>> to Ted, though.
>>> Looking at the upstream e2fsprogs I see in that range:
>>>
>>>> #define EXT4_EXTENTS_FL           0x00080000 /* Inode uses extents */
>>>> #define EXT4_EA_INODE_FL          0x00200000 /* Inode used for large EA */
>>>> #define EXT4_EOFBLOCKS_FL         0x00400000 /* Blocks allocated beyond EOF */
>>>> #define EXT4_SNAPFILE_FL          0x01000000 /* Inode is a snapshot */
>>>> #define EXT4_SNAPFILE_DELETED_FL  0x04000000 /* Snapshot is being deleted */
>>>> #define EXT4_SNAPFILE_SHRUNK_FL   0x08000000 /* Snapshot shrink has completed */
>>>> #define EXT2_RESERVED_FL          0x80000000 /* reserved for ext2 lib */
>>>>
>>>> #define EXT2_FL_USER_VISIBLE      0x004BDFFF /* User visible flags */
>>> so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL.  I don't know the semantics of those two flags enough to say for sure whether it is reasonable that they alias to each other, but at first glance "COW" and "SNAPSHOT" don't seem completely unrelated.
> 
> EXT4_SNAPFILE_FL indicates a special system snapshot file, so it has
> no equivalence relation with FS_COW_FL.
> Please use 0x02000000 for FS_COW_FL.

Fine with that, but it's up to Chris. :)

thanks,
liubo

> 
> EXT4_SNAPFILE_DELETED_FL is a persistent state of a snapshot file,
> which is no longer
> available as a mountable device, but cannot be unlinked because it
> holds changed data sets
> needed by older snapshots.
> 
> EXT4_SNAPFILE_SHRUNK_FL is a persistent state of a (deleted) snapshot
> file, which has
> undergone a "shrink" process to free all change sets not needed by
> older snapshots.
> The persistence of the flag is needed to avoid tedious shrinking when
> it is not needed.
> 
> 
>> In the btrfs case FS_COW_FL means to do COW even when there are no
>> snapshots.  FS_NOCOW_FL means to do cow only when there are snapshots.
>>
> 
> I am interested in FS_NOCOW_FL as well, but for my implementation it would mean
> do not do COW on rewrites even when there are snapshots, so a user can
> create a pre-allocated
> "island of blocks", which are pinned to a physical location, for raw
> VM image for example.
> 
> 
> Thanks,
> Amir.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Chris Mason]

Excerpts from liubo's message of 2011-03-16 22:10:09 -0400:
> On 03/16/2011 05:06 PM, Amir Goldstein wrote:
> > On Wed, Mar 16, 2011 at 1:35 AM, Chris Mason <chris.mason@oracle.com> wrote:
> >> Excerpts from Andreas Dilger's message of 2011-03-15 18:06:49 -0400:
> >>> On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
> >>>> On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
> >>>>>  #define FS_EXTENT_FL         0x00080000 /* Extents */
> >>>>>  #define FS_DIRECTIO_FL       0x00100000 /* Use direct i/o */
> >>>>> +#define FS_NOCOW_FL          0x00800000 /* Do not cow file */
> >>>>> +#define FS_COW_FL            0x01000000 /* Cow file */
> >>>>>  #define FS_RESERVED_FL       0x80000000 /* reserved for ext2 lib */
> >>>> I'm fine with it.  I'll defer the check for conflicts with extN-specific flags
> >>>> to Ted, though.
> >>> Looking at the upstream e2fsprogs I see in that range:
> >>>
> >>>> #define EXT4_EXTENTS_FL           0x00080000 /* Inode uses extents */
> >>>> #define EXT4_EA_INODE_FL          0x00200000 /* Inode used for large EA */
> >>>> #define EXT4_EOFBLOCKS_FL         0x00400000 /* Blocks allocated beyond EOF */
> >>>> #define EXT4_SNAPFILE_FL          0x01000000 /* Inode is a snapshot */
> >>>> #define EXT4_SNAPFILE_DELETED_FL  0x04000000 /* Snapshot is being deleted */
> >>>> #define EXT4_SNAPFILE_SHRUNK_FL   0x08000000 /* Snapshot shrink has completed */
> >>>> #define EXT2_RESERVED_FL          0x80000000 /* reserved for ext2 lib */
> >>>>
> >>>> #define EXT2_FL_USER_VISIBLE      0x004BDFFF /* User visible flags */
> >>> so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL.  I don't know the semantics of those two flags enough to say for sure whether it is reasonable that they alias to each other, but at first glance "COW" and "SNAPSHOT" don't seem completely unrelated.
> > 
> > EXT4_SNAPFILE_FL indicates a special system snapshot file, so it has
> > no equivalence relation with FS_COW_FL.
> > Please use 0x02000000 for FS_COW_FL.
> 
> Fine with that, but it's up to Chris. :)

I'd rather not conflict unless we're critically short on space.

> > 
> > EXT4_SNAPFILE_DELETED_FL is a persistent state of a snapshot file,
> > which is no longer
> > available as a mountable device, but cannot be unlinked because it
> > holds changed data sets
> > needed by older snapshots.
> > 
> > EXT4_SNAPFILE_SHRUNK_FL is a persistent state of a (deleted) snapshot
> > file, which has
> > undergone a "shrink" process to free all change sets not needed by
> > older snapshots.
> > The persistence of the flag is needed to avoid tedious shrinking when
> > it is not needed.
> > 
> > 
> >> In the btrfs case FS_COW_FL means to do COW even when there are no
> >> snapshots.  FS_NOCOW_FL means to do cow only when there are snapshots.
> >>
> > 
> > I am interested in FS_NOCOW_FL as well, but for my implementation it would mean
> > do not do COW on rewrites even when there are snapshots, so a user can
> > create a pre-allocated
> > "island of blocks", which are pinned to a physical location, for raw
> > VM image for example.

I'm not sure how the island of blocks idea can work with snapshots?
Wouldn't the snapshot corrupt if anything in the island were changed?

-chris

Re: [PATCH 1/2] Btrfs: add datacow flag in inode flag

[Amir Goldstein]

On Thu, Mar 17, 2011 at 4:21 PM, Chris Mason <chris.mason@oracle.com> w=
rote:
> Excerpts from liubo's message of 2011-03-16 22:10:09 -0400:
>> On 03/16/2011 05:06 PM, Amir Goldstein wrote:
>> > On Wed, Mar 16, 2011 at 1:35 AM, Chris Mason <chris.mason@oracle.c=
om> wrote:
>> >> Excerpts from Andreas Dilger's message of 2011-03-15 18:06:49 -04=
00:
>> >>> On 2011-03-15, at 2:57 PM, Christoph Hellwig wrote:
>> >>>> On Tue, Mar 15, 2011 at 04:26:50PM -0400, Chris Mason wrote:
>> >>>>> =A0#define FS_EXTENT_FL =A0 =A0 =A0 =A0 0x00080000 /* Extents =
*/
>> >>>>> =A0#define FS_DIRECTIO_FL =A0 =A0 =A0 0x00100000 /* Use direct=
 i/o */
>> >>>>> +#define FS_NOCOW_FL =A0 =A0 =A0 =A0 =A00x00800000 /* Do not c=
ow file */
>> >>>>> +#define FS_COW_FL =A0 =A0 =A0 =A0 =A0 =A00x01000000 /* Cow fi=
le */
>> >>>>> =A0#define FS_RESERVED_FL =A0 =A0 =A0 0x80000000 /* reserved f=
or ext2 lib */
>> >>>> I'm fine with it. =A0I'll defer the check for conflicts with ex=
tN-specific flags
>> >>>> to Ted, though.
>> >>> Looking at the upstream e2fsprogs I see in that range:
>> >>>
>> >>>> #define EXT4_EXTENTS_FL =A0 =A0 =A0 =A0 =A0 0x00080000 /* Inode=
 uses extents */
>> >>>> #define EXT4_EA_INODE_FL =A0 =A0 =A0 =A0 =A00x00200000 /* Inode=
 used for large EA */
>> >>>> #define EXT4_EOFBLOCKS_FL =A0 =A0 =A0 =A0 0x00400000 /* Blocks =
allocated beyond EOF */
>> >>>> #define EXT4_SNAPFILE_FL =A0 =A0 =A0 =A0 =A00x01000000 /* Inode=
 is a snapshot */
>> >>>> #define EXT4_SNAPFILE_DELETED_FL =A00x04000000 /* Snapshot is b=
eing deleted */
>> >>>> #define EXT4_SNAPFILE_SHRUNK_FL =A0 0x08000000 /* Snapshot shri=
nk has completed */
>> >>>> #define EXT2_RESERVED_FL =A0 =A0 =A0 =A0 =A00x80000000 /* reser=
ved for ext2 lib */
>> >>>>
>> >>>> #define EXT2_FL_USER_VISIBLE =A0 =A0 =A00x004BDFFF /* User visi=
ble flags */
>> >>> so there is a conflict with FS_COW_FL and EXT4_SNAPFILE_FL. =A0I=
 don't know the semantics of those two flags enough to say for sure whe=
ther it is reasonable that they alias to each other, but at first glanc=
e "COW" and "SNAPSHOT" don't seem completely unrelated.
>> >
>> > EXT4_SNAPFILE_FL indicates a special system snapshot file, so it h=
as
>> > no equivalence relation with FS_COW_FL.
>> > Please use 0x02000000 for FS_COW_FL.
>>
>> Fine with that, but it's up to Chris. :)
>
> I'd rather not conflict unless we're critically short on space.
>
>> >
>> > EXT4_SNAPFILE_DELETED_FL is a persistent state of a snapshot file,
>> > which is no longer
>> > available as a mountable device, but cannot be unlinked because it
>> > holds changed data sets
>> > needed by older snapshots.
>> >
>> > EXT4_SNAPFILE_SHRUNK_FL is a persistent state of a (deleted) snaps=
hot
>> > file, which has
>> > undergone a "shrink" process to free all change sets not needed by
>> > older snapshots.
>> > The persistence of the flag is needed to avoid tedious shrinking w=
hen
>> > it is not needed.
>> >
>> >
>> >> In the btrfs case FS_COW_FL means to do COW even when there are n=
o
>> >> snapshots. =A0FS_NOCOW_FL means to do cow only when there are sna=
pshots.
>> >>
>> >
>> > I am interested in FS_NOCOW_FL as well, but for my implementation =
it would mean
>> > do not do COW on rewrites even when there are snapshots, so a user=
 can
>> > create a pre-allocated
>> > "island of blocks", which are pinned to a physical location, for r=
aw
>> > VM image for example.
>
> I'm not sure how the island of blocks idea can work with snapshots?
> Wouldn't the snapshot corrupt if anything in the island were changed?
>

It would corrupt, but only to the extent that the file to which you req=
uested
NOCOW may contain newer data. It cannot contain uninitialized data,
because truncating the file would leave it's blocks referenced by the s=
napshot.

Think of a large database file, which is already replicated and hot bac=
ked up
regularly. An arbitrary snapshot of that file will give you a copy for
disaster recovery
at best. Not sure this is worth the effort of COWing it and
fragmenting it beyond
recognition.

Amir.
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel=
" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html