[Ocfs2-devel] [PATCH 0/1] ocfs2: return non-zero st_blocks for inline data [resend2]

[Ocfs2-devel] [PATCH 0/1] ocfs2: return non-zero st_blocks for inline data [resend2]

[John Haxby]

Hello All,

[Really sorry about this and I hope you're not getting fed up of
 multiple copies of this message but the list on oss.oracle.com really
 doesn't like me.]

Some programs, and programmers, assume that if a file is occupying
zero blocks (st_blocks == 0) then it contains no data and there's no
point in reading it.  Posix doesn't actually say anything about this,
but it seems to be something a lot of people expect. Indeed, ext4,
btrfs and ntfs-3d all seem to behave this way so that no one[1] has
any unpleasant surprises.

This patch is almost exactly the same as commit 9206c561554c ("ext4:
return non-zero st_blocks for inline data") although I couldn't bring
myself to include the typo in the comment :)

jch

[resend because rejected by list the first time.]
[1] tar, I'm looking at you, but you're not the only one.


John Haxby (1):
  ocfs2: return non-zero st_blocks for inline data

 fs/ocfs2/file.c | 8 ++++++++
 1 file changed, 8 insertions(+)

-- 
2.5.0

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[John Haxby]

Some versions of tar assume that files with st_blocks == 0 do not
contain any data and will skip reading them entirely. See also
commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").

Signed-off-by: John Haxby <john.haxby@oracle.com>
---
 fs/ocfs2/file.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
index 0e5b451..d631279 100644
--- a/fs/ocfs2/file.c
+++ b/fs/ocfs2/file.c
@@ -1302,6 +1302,14 @@ int ocfs2_getattr(struct vfsmount *mnt,
 	}
 
 	generic_fillattr(inode, stat);
+	/*
+	 * If there is inline data in the inode, the inode will normally not
+	 * have data blocks allocated (it may have an external xattr block).
+	 * Report at least one sector for such files, so tools like tar, rsync,
+	 * others don't incorrectly think the file is completely sparse.
+	 */
+	if (unlikely(OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL))
+		stat->blocks += (stat->size + 511)>>9;
 
 	/* We set the blksize from the cluster size for performance */
 	stat->blksize = osb->s_clustersize;
-- 
2.5.0

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[Gang He]

Looks good for me.

Thanks
Gang


>>> 
> Some versions of tar assume that files with st_blocks == 0 do not
> contain any data and will skip reading them entirely. See also
> commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").
> 
> Signed-off-by: John Haxby <john.haxby@oracle.com>
> ---
>  fs/ocfs2/file.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
> index 0e5b451..d631279 100644
> --- a/fs/ocfs2/file.c
> +++ b/fs/ocfs2/file.c
> @@ -1302,6 +1302,14 @@ int ocfs2_getattr(struct vfsmount *mnt,
>  	}
>  
>  	generic_fillattr(inode, stat);
> +	/*
> +	 * If there is inline data in the inode, the inode will normally not
> +	 * have data blocks allocated (it may have an external xattr block).
> +	 * Report at least one sector for such files, so tools like tar, rsync,
> +	 * others don't incorrectly think the file is completely sparse.
> +	 */
> +	if (unlikely(OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL))
> +		stat->blocks += (stat->size + 511)>>9;
>  
>  	/* We set the blksize from the cluster size for performance */
>  	stat->blksize = osb->s_clustersize;
> -- 
> 2.5.0
> 
> 
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com 
> https://oss.oracle.com/mailman/listinfo/ocfs2-devel

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[Junxiao Bi]

On 11/25/2015 05:07 AM, John Haxby wrote:
> Some versions of tar assume that files with st_blocks == 0 do not
> contain any data and will skip reading them entirely. See also
> commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").
> 
> Signed-off-by: John Haxby <john.haxby@oracle.com>
> ---
>  fs/ocfs2/file.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
> index 0e5b451..d631279 100644
> --- a/fs/ocfs2/file.c
> +++ b/fs/ocfs2/file.c
> @@ -1302,6 +1302,14 @@ int ocfs2_getattr(struct vfsmount *mnt,
>  	}
>  
>  	generic_fillattr(inode, stat);
> +	/*
> +	 * If there is inline data in the inode, the inode will normally not
> +	 * have data blocks allocated (it may have an external xattr block).
> +	 * Report at least one sector for such files, so tools like tar, rsync,
> +	 * others don't incorrectly think the file is completely sparse.
> +	 */
> +	if (unlikely(OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL))
> +		stat->blocks += (stat->size + 511)>>9;
From filesystem side, looks reasonable that data block is 0 for
inlined-data file. This is like a hack to filesystem to fix tools issue.
Indeed tar-1.26-27 have been fixed to not think file with st_blocks == 0
empty. But I am not sure why ext4 merge that fix.

Thanks,
Junxiao.

>  
>  	/* We set the blksize from the cluster size for performance */
>  	stat->blksize = osb->s_clustersize;
> 

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[John Haxby]

> On 1 Dec 2015, at 07:08, Junxiao Bi <junxiao.bi@oracle.com> wrote:
> 
> On 11/25/2015 05:07 AM, John Haxby wrote:
>> Some versions of tar assume that files with st_blocks == 0 do not
>> contain any data and will skip reading them entirely. See also
>> commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").
>> 
>> Signed-off-by: John Haxby <john.haxby@oracle.com>
>> ---
>> fs/ocfs2/file.c | 8 ++++++++
>> 1 file changed, 8 insertions(+)
>> 
>> diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
>> index 0e5b451..d631279 100644
>> --- a/fs/ocfs2/file.c
>> +++ b/fs/ocfs2/file.c
>> @@ -1302,6 +1302,14 @@ int ocfs2_getattr(struct vfsmount *mnt,
>> 	}
>> 
>> 	generic_fillattr(inode, stat);
>> +	/*
>> +	 * If there is inline data in the inode, the inode will normally not
>> +	 * have data blocks allocated (it may have an external xattr block).
>> +	 * Report at least one sector for such files, so tools like tar, rsync,
>> +	 * others don't incorrectly think the file is completely sparse.
>> +	 */
>> +	if (unlikely(OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL))
>> +		stat->blocks += (stat->size + 511)>>9;
> From filesystem side, looks reasonable that data block is 0 for
> inlined-data file. This is like a hack to filesystem to fix tools issue.
> Indeed tar-1.26-27 have been fixed to not think file with st_blocks == 0
> empty. But I am not sure why ext4 merge that fix.

It?s not just tar and it?s not just ext4.   Programmers not unreasonably assume that a file occupying zero blocks contains no data (where would you put it?)

ext4, btrfs and ntfs-3g all give inlined files a non-zero block size to avoid surprising programmers.   There?s nothing in Posix that says what stat?s st_blocks so in this case it?s right for the file systems in question to stick to the principle of least surprise.  In this case, it would be surprising if some small files suddenly started occupying no space while being non-empty.   It?s not as though it would be consistent: some small files would occupy space and some would not.  We want to present a consistent view of files to the user.  It?s not as though we?re breaking du either: it already tells lies :)

Does that make sense now?

jch


> 
> Thanks,
> Junxiao.
> 
>> 
>> 	/* We set the blksize from the cluster size for performance */
>> 	stat->blksize = osb->s_clustersize;

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-devel/attachments/20151201/327439b3/attachment.html 

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[Junxiao Bi]

On 12/02/2015 06:33 AM, John Haxby wrote:
> 
>> On 1 Dec 2015, at 07:08, Junxiao Bi <junxiao.bi@oracle.com
>> <mailto:junxiao.bi@oracle.com>> wrote:
>>
>> On 11/25/2015 05:07 AM, John Haxby wrote:
>>> Some versions of tar assume that files with st_blocks == 0 do not
>>> contain any data and will skip reading them entirely. See also
>>> commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").
>>>
>>> Signed-off-by: John Haxby <john.haxby@oracle.com
>>> <mailto:john.haxby@oracle.com>>
>>> ---
>>> fs/ocfs2/file.c | 8 ++++++++
>>> 1 file changed, 8 insertions(+)
>>>
>>> diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
>>> index 0e5b451..d631279 100644
>>> --- a/fs/ocfs2/file.c
>>> +++ b/fs/ocfs2/file.c
>>> @@ -1302,6 +1302,14 @@ int ocfs2_getattr(struct vfsmount *mnt,
>>> }
>>>
>>> generic_fillattr(inode, stat);
>>> +/*
>>> + * If there is inline data in the inode, the inode will normally not
>>> + * have data blocks allocated (it may have an external xattr block).
>>> + * Report at least one sector for such files, so tools like tar, rsync,
>>> + * others don't incorrectly think the file is completely sparse.
>>> + */
>>> +if (unlikely(OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL))
>>> +stat->blocks += (stat->size + 511)>>9;
>> From filesystem side, looks reasonable that data block is 0 for
>> inlined-data file. This is like a hack to filesystem to fix tools issue.
>> Indeed tar-1.26-27 have been fixed to not think file with st_blocks == 0
>> empty. But I am not sure why ext4 merge that fix.
> 
> It?s not just tar and it?s not just ext4.   Programmers not unreasonably
> assume that a file occupying zero blocks contains no data (where would
> you put it?)
> 
> ext4, btrfs and ntfs-3g all give inlined files a non-zero block size to
> avoid surprising programmers.   There?s nothing in Posix that says what
> stat?s st_blocks so in this case it?s right for the file systems in
> question to stick to the principle of least surprise.  In this case, it
> would be surprising if some small files suddenly started occupying no
> space while being non-empty.   It?s not as though it would be
> consistent: some small files would occupy space and some would not.  We
> want to present a consistent view of files to the user.  It?s not as
> though we?re breaking du either: it already tells lies :)
> 
> Does that make sense now?
OK. Thanks you for the explanation. We'd better not surprise programmers
and keep align with other fs. So

Reviewed-by: Junxiao Bi <junxiao.bi@oracle.com>
> 
> jch
> 
> 
>>
>> Thanks,
>> Junxiao.
>>
>>>
>>> /* We set the blksize from the cluster size for performance */
>>> stat->blksize = osb->s_clustersize;
> 

[Ocfs2-devel] [PATCH 1/1] ocfs2: return non-zero st_blocks for inline data

[Mark Fasheh]

On Tue, Nov 24, 2015 at 09:07:01PM +0000, John Haxby wrote:
> Some versions of tar assume that files with st_blocks == 0 do not
> contain any data and will skip reading them entirely. See also
> commit 9206c561554c ("ext4: return non-zero st_blocks for inline data").
> 
> Signed-off-by: John Haxby <john.haxby@oracle.com>
Reviewed-by: Mark Fasheh <mfasheh@suse.de>

--
Mark Fasheh