XFS defragmentation issue

XFS defragmentation issue

[John Lister]

  Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1 
I'm trying to defragment a drive. If I use xfs_db to check for 
fragmentation it shows for example that a file has 20k+ fragments:
inode 578505506 actual 23240 ideal 1

mapping the inode to the file and running xfs_frs states that the file 
is fully defragmented.

Also running xfs_frs says the drive is defragmented while xfs_db for 
example reports 300k fragments.

Which is right? and is it possible to defragment the file assuming it is 
fragmented so badly, if so how?

Thanks

John

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Stan Hoeppner]

John Lister put forth on 9/10/2010 9:41 AM:
>  Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1
> I'm trying to defragment a drive. If I use xfs_db to check for
> fragmentation it shows for example that a file has 20k+ fragments:
> inode 578505506 actual 23240 ideal 1
> 
> mapping the inode to the file and running xfs_frs states that the file
> is fully defragmented.
> 
> Also running xfs_frs says the drive is defragmented while xfs_db for
> example reports 300k fragments.
> 
> Which is right? and is it possible to defragment the file assuming it is
> fragmented so badly, if so how?

Try unmounting and remounting the filesystem, and see if the various
tools all report the same thing afterwards.  This solved the exact same
problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
2.9.8.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[John Lister]

  Stan Hoeppner wrote on 9/10/2010 14:00
 >>On 10/09/2010 15:41, John Lister wrote:
 >> Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog 
v2.10.2-1 I'm trying to defragment a drive. If I use xfs_db to check for 
fragmentation it shows for >>example that a file has 20k+ fragments:
 >>inode 578505506 actual 23240 ideal 1
 >>
 >>mapping the inode to the file and running xfs_frs states that the 
file is fully defragmented.
 >>
 >>Also running xfs_frs says the drive is defragmented while xfs_db for 
example reports 300k fragments.
 >>
 >>Which is right? and is it possible to defragment the file assuming it 
is fragmented so badly, if so how?

 >Try unmounting and remounting the filesystem, and see if the various
 >tools all report the same thing afterwards. This solved the exact same
 >problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
 >2.9.8.

Cheers, that got rid of most of it, there is still a slight discrepency  
(50 extra fragments) which I can live with.

John

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Dave Chinner]

On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote:
>  Stan Hoeppner wrote on 9/10/2010 14:00
> >>On 10/09/2010 15:41, John Lister wrote:
> >Try unmounting and remounting the filesystem, and see if the various
> >tools all report the same thing afterwards. This solved the exact same
> >problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
> >2.9.8.
> 
> Cheers, that got rid of most of it, there is still a slight
> discrepency  (50 extra fragments) which I can live with.

xfs_db used buffered IO on the block device, which is not coherent
with the filesystem. If you are using it on an active filesystem,
then running "echo 1 > /proc/sys/vm/drop_caches" before you run
xfs_db should make it read from disk at least once....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[John Lister]

On 11/09/2010 09:23, Dave Chinner wrote:
> On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote:
>>   Stan Hoeppner wrote on 9/10/2010 14:00
>>>> On 10/09/2010 15:41, John Lister wrote:
>>> Try unmounting and remounting the filesystem, and see if the various
>>> tools all report the same thing afterwards. This solved the exact same
>>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
>>> 2.9.8.
>> Cheers, that got rid of most of it, there is still a slight
>> discrepency  (50 extra fragments) which I can live with.
> xfs_db used buffered IO on the block device, which is not coherent
> with the filesystem. If you are using it on an active filesystem,
> then running "echo 1>  /proc/sys/vm/drop_caches" before you run
> xfs_db should make it read from disk at least once....
Ah, that makes more sense, I'll give it a go.

Thanks

John

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Stan Hoeppner]

Dave Chinner put forth on 9/11/2010 3:23 AM:
> On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote:
>>  Stan Hoeppner wrote on 9/10/2010 14:00
>>>> On 10/09/2010 15:41, John Lister wrote:
>>> Try unmounting and remounting the filesystem, and see if the various
>>> tools all report the same thing afterwards. This solved the exact same
>>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
>>> 2.9.8.
>>
>> Cheers, that got rid of most of it, there is still a slight
>> discrepency  (50 extra fragments) which I can live with.
> 
> xfs_db used buffered IO on the block device, which is not coherent
> with the filesystem. If you are using it on an active filesystem,
> then running "echo 1 > /proc/sys/vm/drop_caches" before you run
> xfs_db should make it read from disk at least once....

That's good to know Dave.  Thanks.  I created a short-name script a
while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could
"clear the baffles" now and then.  Looks like it now has multiple uses
(if I can just remember to use it in this context).

Out of curiosity, who here schedules automatic xfs_fsr runs, and with
what frequency?  Currently I just run it manually now and then when
performance starts to seem sluggish.

Thanks.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Iustin Pop]

On Sat, Sep 11, 2010 at 02:38:48PM -0500, Stan Hoeppner wrote:
> Dave Chinner put forth on 9/11/2010 3:23 AM:
> > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote:
> >>  Stan Hoeppner wrote on 9/10/2010 14:00
> >>>> On 10/09/2010 15:41, John Lister wrote:
> >>> Try unmounting and remounting the filesystem, and see if the various
> >>> tools all report the same thing afterwards. This solved the exact same
> >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
> >>> 2.9.8.
> >>
> >> Cheers, that got rid of most of it, there is still a slight
> >> discrepency  (50 extra fragments) which I can live with.
> > 
> > xfs_db used buffered IO on the block device, which is not coherent
> > with the filesystem. If you are using it on an active filesystem,
> > then running "echo 1 > /proc/sys/vm/drop_caches" before you run
> > xfs_db should make it read from disk at least once....

I wonder if xfs_db shouldn't use by default direct I/O, or at least take
a flag to allow it to do direct I/O only against the blocks it needs.
Dropping the entire caches on a big box is not nice :)

> That's good to know Dave.  Thanks.  I created a short-name script a
> while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could
> "clear the baffles" now and then.  Looks like it now has multiple uses
> (if I can just remember to use it in this context).
> 
> Out of curiosity, who here schedules automatic xfs_fsr runs, and with
> what frequency?  Currently I just run it manually now and then when
> performance starts to seem sluggish.

This is my setup, on a machine with multiple XFS filesystems:

$ cat /etc/cron.d/xfsfsr 
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

0   01  *   *   7   root    xfs_fsr -v

I've set it up a few years back and, except for a few rare bugs in 2.6.x
which are then fixed in 2.6.x.y, I've had no issues.

regards,
iustin

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Dave Chinner]

On Sat, Sep 11, 2010 at 09:57:02PM +0200, Iustin Pop wrote:
> On Sat, Sep 11, 2010 at 02:38:48PM -0500, Stan Hoeppner wrote:
> > Dave Chinner put forth on 9/11/2010 3:23 AM:
> > > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote:
> > >>  Stan Hoeppner wrote on 9/10/2010 14:00
> > >>>> On 10/09/2010 15:41, John Lister wrote:
> > >>> Try unmounting and remounting the filesystem, and see if the various
> > >>> tools all report the same thing afterwards. This solved the exact same
> > >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs
> > >>> 2.9.8.
> > >>
> > >> Cheers, that got rid of most of it, there is still a slight
> > >> discrepency  (50 extra fragments) which I can live with.
> > > 
> > > xfs_db used buffered IO on the block device, which is not coherent
> > > with the filesystem. If you are using it on an active filesystem,
> > > then running "echo 1 > /proc/sys/vm/drop_caches" before you run
> > > xfs_db should make it read from disk at least once....
> 
> I wonder if xfs_db shouldn't use by default direct I/O, or at least take
> a flag to allow it to do direct I/O only against the blocks it needs.
> Dropping the entire caches on a big box is not nice :)

True - I need to look at whether posix_fadvise(POSIX_FADV_DONTNEED)
will clear the bdev pages, and if so that is the easiest solution.
Rewriting xfs_db to use direct IO is a pretty major undertaking...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Emmanuel Florac]

Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez:

> That's good to know Dave.  Thanks.  I created a short-name script a
> while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could
> "clear the baffles" now and then. 

But isn't it somewhat risky? isn't there any risk of data loss by
dropping cache at any moment?

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Stan Hoeppner]

Emmanuel Florac put forth on 9/12/2010 3:43 AM:
> Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez:
> 
>> That's good to know Dave.  Thanks.  I created a short-name script a
>> while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could
>> "clear the baffles" now and then. 
> 
> But isn't it somewhat risky? isn't there any risk of data loss by
> dropping cache at any moment?

From:  http://linux-mm.org/Drop_Caches

This is a non-destructive operation and will only free things that are
completely unused. Dirty objects will continue to be in use until
written out to disk and are not freeable. If you run "sync" first to
flush them out to disk, these drop operations will tend to free more
memory.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS defragmentation issue

[Roel van Meer]

Stan Hoeppner writes:

> Out of curiosity, who here schedules automatic xfs_fsr runs, and with
> what frequency?  Currently I just run it manually now and then when
> performance starts to seem sluggish.

I run it daily during the week, at a time when load is practically nil, on 
two boxes with 24TB storage each, divided into six parts of about 4GB. A 
10-pass xfs_fsr run usually completes in under one hour, except on monday, 
when it takes three to four hours.

I don't have any stats on the net effect, though.

Regards,

roel

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs