XFS Preallocate using ALLOCSP

XFS Preallocate using ALLOCSP

[Smit Shah]

Hi, 

I wanted to preallocate the file using the ALLOCSP so that the preallocated
space is zeroed out. Hence i used the xfsctl but the problem is that i
cannot execute  it as a non-root user.  So i tried using fallocate but when
i saw the xfs implementation of kernel it uses the RESVP cmd and  since the
unwritten flag is set i am assuming that the performance of writing to the
preallocated space will suffer because of the metadata updates. So is there
a way to preallocate using ALLOCSP as a non-root user or else i would be
required to modify the kernel ??

TIA, 
Smit 
-- 
View this message in context: http://www.nabble.com/XFS-Preallocate-using-ALLOCSP-tp24042506p24042506.html
Sent from the linux-xfs mailing list archive at Nabble.com.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

Ok i did the first part i.e preallocation using xfsctl() by using
setresgid().
But still is there some way this option could be incorporated in the
fallocate system call.

Smit Shah wrote:
> 
> Hi, 
> 
> I wanted to preallocate the file using the ALLOCSP so that the
> preallocated space is zeroed out. Hence i used the xfsctl but the problem
> is that i cannot execute  it as a non-root user.  So i tried using
> fallocate but when i saw the xfs implementation of kernel it uses the
> RESVP cmd and  since the unwritten flag is set i am assuming that the
> performance of writing to the preallocated space will suffer because of
> the metadata updates. So is there a way to preallocate using ALLOCSP as a
> non-root user or else i would be required to modify the kernel ??
> 
> TIA, 
> Smit 
> 

-- 
View this message in context: http://www.nabble.com/XFS-Preallocate-using-ALLOCSP-tp24042506p24046118.html
Sent from the linux-xfs mailing list archive at Nabble.com.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Eric Sandeen]

Smit Shah wrote:
> Hi, 
> 
> I wanted to preallocate the file using the ALLOCSP so that the preallocated
> space is zeroed out. Hence i used the xfsctl but the problem is that i
> cannot execute  it as a non-root user.  So i tried using fallocate but when
> i saw the xfs implementation of kernel it uses the RESVP cmd and  since the
> unwritten flag is set i am assuming that the performance of writing to the
> preallocated space will suffer because of the metadata updates. So is there
> a way to preallocate using ALLOCSP as a non-root user or else i would be
> required to modify the kernel ??
> 
> TIA, 
> Smit 

ALLOCSP simply writes 0s to a file.... you may as well do it using posix
calls, as a normal user, I think.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

[-- Attachment #1.1: Type: text/plain, Size: 1698 bytes --]

On Mon, Jun 15, 2009 at 8:34 PM, Eric Sandeen <sandeen@sandeen.net> wrote:

> Smit Shah wrote:
> > Hi,
> >
> > I wanted to preallocate the file using the ALLOCSP so that the
> preallocated
> > space is zeroed out. Hence i used the xfsctl but the problem is that i
> > cannot execute  it as a non-root user.  So i tried using fallocate but
> when
> > i saw the xfs implementation of kernel it uses the RESVP cmd and  since
> the
> > unwritten flag is set i am assuming that the performance of writing to
> the
> > preallocated space will suffer because of the metadata updates. So is
> there
> > a way to preallocate using ALLOCSP as a non-root user or else i would be
> > required to modify the kernel ??
> >
> > TIA,
> > Smit
>
> ALLOCSP simply writes 0s to a file.... you may as well do it using posix
> calls, as a normal user, I think.
>

Even the man page of fallocate says that it allocates and initializes to
zero the disk space allocated but when i saw the code i did found out that
it does not zero it out.  Hence was a kindof confused. So posix_allocate is
similar to ALLOCSP when falloc is not supported by the underlying filesystem
 that is to ftruncate the file  and zero it out. So all of them try to
allocate contiguous blocks but the only difference is when we use the
fallocate in ext4/xfs it does not zero out the preallocated space. Am i
right ?
But  when i fallocate in ext4 i can see the write performance improvement
but not in xfs and reason i found out in one of your previous comments is
 because of the unwritten flag set in xfs. So how do we see if the unwritten
flag is set or not ? I did use xfs_info but it didnt show any such
information.

Thanks,
Smit


>
> -Eric
>

[-- Attachment #1.2: Type: text/html, Size: 2285 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

[-- Attachment #1.1: Type: text/plain, Size: 2337 bytes --]

On Mon, Jun 15, 2009 at 11:44 PM, Smit Shah <getsmit@gmail.com> wrote:

>
>
> On Mon, Jun 15, 2009 at 8:34 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
>
>> Smit Shah wrote:
>> > Hi,
>> >
>> > I wanted to preallocate the file using the ALLOCSP so that the
>> preallocated
>> > space is zeroed out. Hence i used the xfsctl but the problem is that i
>> > cannot execute  it as a non-root user.  So i tried using fallocate but
>> when
>> > i saw the xfs implementation of kernel it uses the RESVP cmd and  since
>> the
>> > unwritten flag is set i am assuming that the performance of writing to
>> the
>> > preallocated space will suffer because of the metadata updates. So is
>> there
>> > a way to preallocate using ALLOCSP as a non-root user or else i would be
>> > required to modify the kernel ??
>> >
>> > TIA,
>> > Smit
>>
>> ALLOCSP simply writes 0s to a file.... you may as well do it using posix
>> calls, as a normal user, I think.
>>
>
> Even the man page of fallocate says that it allocates and initializes to
> zero the disk space allocated but when i saw the code i did found out that
> it does not zero it out.  Hence was a kindof confused. So posix_allocate is
> similar to ALLOCSP when falloc is not supported by the underlying filesystem
>  that is to ftruncate the file  and zero it out. So all of them try to
> allocate contiguous blocks but the only difference is when we use the
> fallocate in ext4/xfs it does not zero out the preallocated space. Am i
> right ?
>
I guess i am not right here ftruncate simply does a lseek and wirtes to it
which might not be contiguous whereas fallocate tries to allocate contiguous
block so as to reduce fragmentation and hence i thought to reduce
fragmentation and for security reasons its better to use ALLOCSP rather than
something like ftruncate /posix_fallocate or RSEVSP which kindof performs
bad for writes with unwritten flag set and now there being a no direct way
while creating the fs to disable unwritten.


> But  when i fallocate in ext4 i can see the write performance improvement
> but not in xfs and reason i found out in one of your previous comments is
>  because of the unwritten flag set in xfs. So how do we see if the unwritten
> flag is set or not ? I did use xfs_info but it didnt show any such
> information.
>


>
> Thanks,
> Smit
>
>
>>
>> -Eric
>>
>
>

[-- Attachment #1.2: Type: text/html, Size: 3503 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Eric Sandeen]

Smit Shah wrote:

> Even the man page of fallocate says that it allocates and initializes
> to zero the disk space allocated

Bleah, so it does:

       FALLOC_FL_KEEP_SIZE
              This flag allocates and initializes to zero the disk space

well, that's misleading and/or wrong.

> but when i saw the code i did found out that it does not zero it out.
> Hence was a kindof confused. So posix_allocate is similar to ALLOCSP
> when falloc is not supported by the underlying filesystem  that is to
> ftruncate the file  and zero it out. So all of them try to allocate
> contiguous blocks but the only difference is when we use the
> fallocate in ext4/xfs it does not zero out the preallocated space. Am
> i right ?

fallocate / sys_fallocate marks the region as uninitialized so that you
get back 0s when you read.  It's implemented on xfs, ext4, ocfs2, and btrfs.

posix_fallocate manages to reach sys_fallocate when all the stars align:
kernel,  glibc, and filesystgem.  Otherwise it writes 0s.

> But  when i fallocate in ext4 i can see the write performance 
> improvement but not in xfs

Testing how?

> and reason i found out in one of your previous comments is  because
> of the unwritten flag set in xfs. So how do we see if the unwritten
> flag is set or not ? I did use xfs_info but it didnt show any such
> information.

ext4 & xfs are doing the same basic thing, they must maintain the
unwritten state on the preallocated extents, and manage that as it
changes when portions are written with real data.

xfs_bmap -v -v -p on a file will show you extent state for xfs.

> I guess i am not right here ftruncate simply does a lseek and wirtes

ftruncate simply sets i_size, it does no data IO.

> to it which might not be contiguous whereas fallocate tries to
> allocate contiguous block so as to reduce fragmentation

Actually fallocate's only official job is to reserve blocks so you don't
get ENOSPC later.  Because the request comes in all at once, you are
very likely to get an optimal allocation, and that's a nice side effect,
but it's not actually required by the interface.

> and hence i
> thought to reduce fragmentation and for security reasons 

None of these normal interfaces poses any security risk.  If you build
xfs without the unwritten extent feature you could allocate w/o flagging
uninitialized and expose stale data, but that's not a normal mode of
operation.

> its better
> to use ALLOCSP rather than something like ftruncate /posix_fallocate
> or RSEVSP which kindof performs bad for writes with unwritten flag
> set and now there being a no direct way while creating the fs to
> disable unwritten.

In the end, there are only 2 ways to preallocate blocks: explicitly
write 0s, or flag regions as unwritten (as xfs/ext4/... can do).  (Ok,
or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's what
you're looking for, but that's deprecated or not really available at
this point).

Maybe I should ask what the end goal is here.  :)

-Eric

> Thanks, Smit

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Felix Blyakher]

On Jun 16, 2009, at 10:16 AM, Eric Sandeen wrote:

> Smit Shah wrote:
>
>> Even the man page of fallocate says that it allocates and initializes
>> to zero the disk space allocated
>
> Bleah, so it does:
>
>       FALLOC_FL_KEEP_SIZE
>              This flag allocates and initializes to zero the disk  
> space
>
> well, that's misleading and/or wrong.
>
>> but when i saw the code i did found out that it does not zero it out.
>> Hence was a kindof confused. So posix_allocate is similar to ALLOCSP
>> when falloc is not supported by the underlying filesystem  that is to
>> ftruncate the file  and zero it out. So all of them try to allocate
>> contiguous blocks but the only difference is when we use the
>> fallocate in ext4/xfs it does not zero out the preallocated space. Am
>> i right ?
>
> fallocate / sys_fallocate marks the region as uninitialized so that  
> you
> get back 0s when you read.  It's implemented on xfs, ext4, ocfs2,  
> and btrfs.
>
> posix_fallocate manages to reach sys_fallocate when all the stars  
> align:
> kernel,  glibc, and filesystgem.  Otherwise it writes 0s.
>
>> But  when i fallocate in ext4 i can see the write performance
>> improvement but not in xfs
>
> Testing how?
>
>> and reason i found out in one of your previous comments is  because
>> of the unwritten flag set in xfs. So how do we see if the unwritten
>> flag is set or not ? I did use xfs_info but it didnt show any such
>> information.
>
> ext4 & xfs are doing the same basic thing, they must maintain the
> unwritten state on the preallocated extents, and manage that as it
> changes when portions are written with real data.

Well, the difference in managing the unwritten state can
theoretically result in different performance. Not that I'd
expect ext4 being better than xfs in this respect.
More data is needed here.

> xfs_bmap -v -v -p on a file will show you extent state for xfs.
>
>> I guess i am not right here ftruncate simply does a lseek and wirtes
>
> ftruncate simply sets i_size, it does no data IO.

... and no block reservation/allocation either.

>> to it which might not be contiguous whereas fallocate tries to
>> allocate contiguous block so as to reduce fragmentation
>
> Actually fallocate's only official job is to reserve blocks so you  
> don't
> get ENOSPC later.  Because the request comes in all at once, you are
> very likely to get an optimal allocation, and that's a nice side  
> effect,
> but it's not actually required by the interface.
>
>> and hence i
>> thought to reduce fragmentation and for security reasons
>
> None of these normal interfaces poses any security risk.  If you build
> xfs without the unwritten extent feature

I don't think, it's possible. Not in any configurable way,
at least.

> you could allocate w/o flagging
> uninitialized and expose stale data, but that's not a normal mode of
> operation.

That's was possible with mount option unwritten=0, but
AFAIK, it's been recently completely removed from code.

>
>
>> its better
>> to use ALLOCSP rather than something like ftruncate /posix_fallocate
>> or RSEVSP which kindof performs bad for writes with unwritten flag
>> set and now there being a no direct way while creating the fs to
>> disable unwritten.
>
> In the end, there are only 2 ways to preallocate blocks: explicitly
> write 0s, or flag regions as unwritten (as xfs/ext4/... can do).

Exactly.
That's a trade-off between spending time on setup or at the
write time. And if explicit zeros are desirable for the former
approach, it can be driven from the user space (after
preallocation) rather than from the kernel with the exactly the
same outcome.

Just restating the same what Eric already said :)

Felix

>  (Ok,
> or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's  
> what
> you're looking for, but that's deprecated or not really available at
> this point).
>
> Maybe I should ask what the end goal is here.  :)
>
> -Eric
>
>> Thanks, Smit
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>
> On Jun 16, 2009, at 10:16 AM, Eric Sandeen wrote:
>
>> Smit Shah wrote:
>>
>>> Even the man page of fallocate says that it allocates and initializes
>>> to zero the disk space allocated
>>
>> Bleah, so it does:
>>
>>       FALLOC_FL_KEEP_SIZE
>>              This flag allocates and initializes to zero the disk
>> space
>>
>> well, that's misleading and/or wrong.
>>
>>> but when i saw the code i did found out that it does not zero it out.
>>> Hence was a kindof confused. So posix_allocate is similar to ALLOCSP
>>> when falloc is not supported by the underlying filesystem  that is to
>>> ftruncate the file  and zero it out. So all of them try to allocate
>>> contiguous blocks but the only difference is when we use the
>>> fallocate in ext4/xfs it does not zero out the preallocated space. Am
>>> i right ?
>>
>> fallocate / sys_fallocate marks the region as uninitialized so that
>> you
>> get back 0s when you read.  It's implemented on xfs, ext4, ocfs2,
>> and btrfs.
>>
>> posix_fallocate manages to reach sys_fallocate when all the stars
>> align:
>> kernel,  glibc, and filesystgem.  Otherwise it writes 0s.
>>
>>> But  when i fallocate in ext4 i can see the write performance
>>> improvement but not in xfs
>>
>> Testing how?

I use IOmeter to test it.

>>
>>> and reason i found out in one of your previous comments is  because
>>> of the unwritten flag set in xfs. So how do we see if the unwritten
>>> flag is set or not ? I did use xfs_info but it didnt show any such
>>> information.
>>
>> ext4 & xfs are doing the same basic thing, they must maintain the
>> unwritten state on the preallocated extents, and manage that as it
>> changes when portions are written with real data.
>
> Well, the difference in managing the unwritten state can
> theoretically result in different performance. Not that I'd
> expect ext4 being better than xfs in this respect.
> More data is needed here.

Like when i do preallocation i see the thorughput for seq writes being
the same in ext4 but for xfs its goes down by say 10Mps or so.

>
>> xfs_bmap -v -v -p on a file will show you extent state for xfs.
>>
Thanks a lot.
>>> I guess i am not right here ftruncate simply does a lseek and wirtes
>>
>> ftruncate simply sets i_size, it does no data IO.
>
> ... and no block reservation/allocation either.
>
Yeah rite what i meant to say was that posix_fallocate uses ftruncate
which in turn just updates the i_size and then posix_fallocate zeros
out the whole thing

>>> to it which might not be contiguous whereas fallocate tries to
>>> allocate contiguous block so as to reduce fragmentation
>>
>> Actually fallocate's only official job is to reserve blocks so you
>> don't
>> get ENOSPC later.  Because the request comes in all at once, you are
>> very likely to get an optimal allocation, and that's a nice side
>> effect,
>> but it's not actually required by the interface.
>>
>>> and hence i
>>> thought to reduce fragmentation and for security reasons
>>
>> None of these normal interfaces poses any security risk.  If you build
>> xfs without the unwritten extent feature
>
> I don't think, it's possible. Not in any configurable way,
> at least.
>
>> you could allocate w/o flagging
>> uninitialized and expose stale data, but that's not a normal mode of
>> operation.
>
> That's was possible with mount option unwritten=0, but
> AFAIK, it's been recently completely removed from code.
>
>>
>>
>>> its better
>>> to use ALLOCSP rather than something like ftruncate /posix_fallocate
>>> or RSEVSP which kindof performs bad for writes with unwritten flag
>>> set and now there being a no direct way while creating the fs to
>>> disable unwritten.
>>
>> In the end, there are only 2 ways to preallocate blocks: explicitly
>> write 0s, or flag regions as unwritten (as xfs/ext4/... can do).
>
> Exactly.
> That's a trade-off between spending time on setup or at the
> write time. And if explicit zeros are desirable for the former
> approach, it can be driven from the user space (after
> preallocation) rather than from the kernel with the exactly the
> same outcome.
>
> Just restating the same what Eric already said :)
>
> Felix
>
>>  (Ok,
>> or a 3rd sorta-way, which is to reserve w/o flagging, maybe that's
>> what
>> you're looking for, but that's deprecated or not really available at
>> this point).
Yes that is what i was looking for :) but i guess its no more
available through mkfs.xfs
and i will have to do it with xfs_db as stated in one of the Eric's
replies in the previous posts

>>
>> Maybe I should ask what the end goal is here.  :)
>>
Just to see if preallocation using fallocate helps reducing the
fragmentation and increases the throughput.  I guess it wll help
reduce the fragmentation but the write performance is going to suffer.

>> -Eric
>>
>>> Thanks, Smit
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs
>
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Eric Sandeen]

Smit Shah wrote:
> On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>> On Jun 16, 2009, at 10:16 AM, Eric Sandeen wrote:


>>> Maybe I should ask what the end goal is here.  :)
>>>
> Just to see if preallocation using fallocate helps reducing the
> fragmentation and increases the throughput.  I guess it wll help
> reduce the fragmentation but the write performance is going to suffer.

Depends on the IO you do, I think.  I'd be curious to know exactly how
you're running preallocation & iometer so I could give it a shot, to see
why xfs seems to be doing worse than ext4.

Thanks,
-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Felix Blyakher]

On Jun 16, 2009, at 12:28 PM, Smit Shah wrote:

> Yeah rite what i meant to say was that posix_fallocate uses ftruncate
> which in turn just updates the i_size and then posix_fallocate zeros
> out the whole thing

I didn't look at the libc implementation, but it seems
strange to use ftruncate here at all. Is it more efficient
to write zeros into a hole than appending to a file?

[snip]

> Just to see if preallocation using fallocate helps reducing the
> fragmentation and increases the throughput.  I guess it wll help
> reduce the fragmentation

... and as result read performance on such files.

> but the write performance is going to suffer.

It's not clear why it should. Not doing preallocation doesn't
mean that there is no inode updates with every write. Why
would extent conversion be more expensive that creating the
space (extent) and updating the inode size for every write?
It'd interesting to reproduce your results. Any details on
your tests and the iometer usage?

Thanks,
Felix

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

In the vmware sparse disk,  the chunks are allocated of the size of 64k,
which kindof leads to fragmentation. Hence the sequential read throughput
suffers. So thought it might be a good idea to preallocate some
amount memory say 50Meg when it asks for allocating the chunk thus helps
in reducing fragmentation and improves the throughput.
On linux i do preallocation using fallocate.

I have vmware workstation running Windows Vista and host is RHEL 5.3
with the guest having 1GB of memory and host having 2 Gig. And i have
separate dedicated SATA disk on which i perform the tests. For
testing, I create a new sparse vdisk and run a IOmeter sequential
write on it.

For ext4 i tend to get say around 5-6 Mbps of improvement in write
throughput sometimes but it never goes down. But for XFS i always see
atleast
8-10 Mbps of downfall.

-Smit

On 6/16/09, Eric Sandeen <sandeen@sandeen.net> wrote:
> Smit Shah wrote:
>> On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>>> On Jun 16, 2009, at 10:16 AM, Eric Sandeen wrote:
>
>
>>>> Maybe I should ask what the end goal is here.  :)
>>>>
>> Just to see if preallocation using fallocate helps reducing the
>> fragmentation and increases the throughput.  I guess it wll help
>> reduce the fragmentation but the write performance is going to suffer.
>
> Depends on the IO you do, I think.  I'd be curious to know exactly how
> you're running preallocation & iometer so I could give it a shot, to see
> why xfs seems to be doing worse than ext4.
>


> Thanks,
> -Eric
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>
> On Jun 16, 2009, at 12:28 PM, Smit Shah wrote:
>
>> Yeah rite what i meant to say was that posix_fallocate uses ftruncate
>> which in turn just updates the i_size and then posix_fallocate zeros
>> out the whole thing
>
> I didn't look at the libc implementation, but it seems
> strange to use ftruncate here at all. Is it more efficient
> to write zeros into a hole than appending to a file?
I havent seen in detail but i had just scanned through the
posix_fallocate code in libc sometime back and it seemed to be doing
that but i can confirm that later.

> [snip]
>
>> Just to see if preallocation using fallocate helps reducing the
>> fragmentation and increases the throughput.  I guess it wll help
>> reduce the fragmentation
>
> ... and as result read performance on such files.
>
>> but the write performance is going to suffer.
>
> It's not clear why it should. Not doing preallocation doesn't
> mean that there is no inode updates with every write. Why
> would extent conversion be more expensive that creating the
> space (extent) and updating the inode size for every write?
> It'd interesting to reproduce your results. Any details on
> your tests and the iometer usage?

Since fallocate uses the RESVSP cmd for xfs. And as given given for
RESVSP in man page for xfsctl
If the  XFS filesystem  is  configured to flag unwritten file extents,
performance will be negatively affected when writing to preallocated
space, since extra filesystem transactions are required to convert
extent  flags  on  the  range  of  the  file  written.

>
> Thanks,
> Felix
>
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Eric Sandeen]

Smit Shah wrote:
> On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:

>>> but the write performance is going to suffer.
>> It's not clear why it should. Not doing preallocation doesn't
>> mean that there is no inode updates with every write. Why
>> would extent conversion be more expensive that creating the
>> space (extent) and updating the inode size for every write?
>> It'd interesting to reproduce your results. Any details on
>> your tests and the iometer usage?
> 
> Since fallocate uses the RESVSP cmd for xfs. And as given given for
> RESVSP in man page for xfsctl
> If the  XFS filesystem  is  configured to flag unwritten file extents,
> performance will be negatively affected when writing to preallocated
> space, since extra filesystem transactions are required to convert
> extent  flags  on  the  range  of  the  file  written.

And ext4 must do basically the same thing, as would any fs that flags
unwritten extents.

ext4 may convert more at a time, though, rather than leaving
"fragmented" written/unwritten/written/unwritten regions.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Smit Shah]

On 6/16/09, Smit Shah <getsmit@gmail.com> wrote:
> On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>>
>> On Jun 16, 2009, at 12:28 PM, Smit Shah wrote:
>>
>>> Yeah rite what i meant to say was that posix_fallocate uses ftruncate
>>> which in turn just updates the i_size and then posix_fallocate zeros
>>> out the whole thing
>>
>> I didn't look at the libc implementation, but it seems
>> strange to use ftruncate here at all. Is it more efficient
>> to write zeros into a hole than appending to a file?
> I havent seen in detail but i had just scanned through the
> posix_fallocate code in libc sometime back and it seemed to be doing
> that but i can confirm that later.
>
Yeah it simply write's zero's, i had seen the ftruncate used in the
posix_fallocate implimentation and hence got confused. It is used when
the prealloc length is zero and total file size is less than offset
specified.
My bad.

>> [snip]
>>
>>> Just to see if preallocation using fallocate helps reducing the
>>> fragmentation and increases the throughput.  I guess it wll help
>>> reduce the fragmentation
>>
>> ... and as result read performance on such files.
>>
>>> but the write performance is going to suffer.
>>
>> It's not clear why it should. Not doing preallocation doesn't
>> mean that there is no inode updates with every write. Why
>> would extent conversion be more expensive that creating the
>> space (extent) and updating the inode size for every write?
>> It'd interesting to reproduce your results. Any details on
>> your tests and the iometer usage?
>
> Since fallocate uses the RESVSP cmd for xfs. And as given given for
> RESVSP in man page for xfsctl
> If the  XFS filesystem  is  configured to flag unwritten file extents,
> performance will be negatively affected when writing to preallocated
> space, since extra filesystem transactions are required to convert
> extent  flags  on  the  range  of  the  file  written.
>
>>
>> Thanks,
>> Felix
>>
>>
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Felix Blyakher]

On Jun 16, 2009, at 4:42 PM, Smit Shah wrote:

> On 6/16/09, Felix Blyakher <felixb@sgi.com> wrote:
>>
>> On Jun 16, 2009, at 12:28 PM, Smit Shah wrote:
>>> but the write performance is going to suffer.
>>
>> It's not clear why it should. Not doing preallocation doesn't
>> mean that there is no inode updates with every write. Why
>> would extent conversion be more expensive that creating the
>> space (extent) and updating the inode size for every write?
>> It'd interesting to reproduce your results. Any details on
>> your tests and the iometer usage?
>
> Since fallocate uses the RESVSP cmd for xfs. And as given given for
> RESVSP in man page for xfsctl
> If the  XFS filesystem  is  configured to flag unwritten file extents,

Good point, we should update the man page, there is no 'if'
any more.

>
> performance will be negatively affected when writing to preallocated
> space, since extra filesystem transactions are required to convert
> extent  flags  on  the  range  of  the  file  written.

Yes, absolutely, there is overhead to handle unwritten
extents, no doubt. My remark was wrt comparison between
not doing preallocation at all and preallocation with
unwritten extents. Both cases would incur the metadata
updates with every write. And I doubt that write performance
would suffer more in the preallocation case.
Though, if you prefer to trade extra time spent at every
write to setup time, you can/should manually fill the
file with zeros.

Felix

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Felix Blyakher]

On Jun 16, 2009, at 5:05 PM, Eric Sandeen wrote:

> Smit Shah wrote:
>>
>> Since fallocate uses the RESVSP cmd for xfs. And as given given for
>> RESVSP in man page for xfsctl
>> If the  XFS filesystem  is  configured to flag unwritten file  
>> extents,
>> performance will be negatively affected when writing to preallocated
>> space, since extra filesystem transactions are required to convert
>> extent  flags  on  the  range  of  the  file  written.
>
> And ext4 must do basically the same thing, as would any fs that flags
> unwritten extents.
>
> ext4 may convert more at a time, though, rather than leaving
> "fragmented" written/unwritten/written/unwritten regions.

Converting more at a time would require writing extra zeros
to extra space converted, but not covered with write. Again,
it's a trade off, and there is no clear winner. Well, at least
not in my mind, until proven that some operations are more
efficient than others.

Thanks,
Felix

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS Preallocate using ALLOCSP

[Michael Monnerie]

On Dienstag 16 Juni 2009 Smit Shah wrote:
> In the vmware sparse disk

Ah, virtualization again. Yes, that's no good for performance ;-) 
There's a "defragment disk" button in VMware which simply copies the 
flat file on the filesystem in order to remove fragmentation. I guess 
running xfs_fsr would help more in that case. Did you try with that?

> chunks are allocated of the size of 64k,

It's a pity that there's no option to define how large each chunk should 
be. Using 1-50MB of real disk space for each new chunk could help a lot 
to keep the file less fragmented. It get's really funny when you run 
"defrag" within the VM on a sparse vdisk.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc    -----      http://it-management.at
// Tel: 0660 / 415 65 31                      .network.your.ideas.
// PGP Key:         "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38  500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net                  Key-ID: 1C1209B4

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs