* XFS defragmentation issue @ 2010-09-10 14:41 John Lister 2010-09-10 19:00 ` Stan Hoeppner 2010-09-11 6:55 ` John Lister 0 siblings, 2 replies; 11+ messages in thread From: John Lister @ 2010-09-10 14:41 UTC (permalink / raw) To: xfs Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1 I'm trying to defragment a drive. If I use xfs_db to check for fragmentation it shows for example that a file has 20k+ fragments: inode 578505506 actual 23240 ideal 1 mapping the inode to the file and running xfs_frs states that the file is fully defragmented. Also running xfs_frs says the drive is defragmented while xfs_db for example reports 300k fragments. Which is right? and is it possible to defragment the file assuming it is fragmented so badly, if so how? Thanks John _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-10 14:41 XFS defragmentation issue John Lister @ 2010-09-10 19:00 ` Stan Hoeppner 2010-09-11 6:55 ` John Lister 1 sibling, 0 replies; 11+ messages in thread From: Stan Hoeppner @ 2010-09-10 19:00 UTC (permalink / raw) To: xfs John Lister put forth on 9/10/2010 9:41 AM: > Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1 > I'm trying to defragment a drive. If I use xfs_db to check for > fragmentation it shows for example that a file has 20k+ fragments: > inode 578505506 actual 23240 ideal 1 > > mapping the inode to the file and running xfs_frs states that the file > is fully defragmented. > > Also running xfs_frs says the drive is defragmented while xfs_db for > example reports 300k fragments. > > Which is right? and is it possible to defragment the file assuming it is > fragmented so badly, if so how? Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8. -- Stan _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-10 14:41 XFS defragmentation issue John Lister 2010-09-10 19:00 ` Stan Hoeppner @ 2010-09-11 6:55 ` John Lister 2010-09-11 8:23 ` Dave Chinner 1 sibling, 1 reply; 11+ messages in thread From: John Lister @ 2010-09-11 6:55 UTC (permalink / raw) To: xfs Stan Hoeppner wrote on 9/10/2010 14:00 >>On 10/09/2010 15:41, John Lister wrote: >> Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1 I'm trying to defragment a drive. If I use xfs_db to check for fragmentation it shows for >>example that a file has 20k+ fragments: >>inode 578505506 actual 23240 ideal 1 >> >>mapping the inode to the file and running xfs_frs states that the file is fully defragmented. >> >>Also running xfs_frs says the drive is defragmented while xfs_db for example reports 300k fragments. >> >>Which is right? and is it possible to defragment the file assuming it is fragmented so badly, if so how? >Try unmounting and remounting the filesystem, and see if the various >tools all report the same thing afterwards. This solved the exact same >problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs >2.9.8. Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with. John _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 6:55 ` John Lister @ 2010-09-11 8:23 ` Dave Chinner 2010-09-11 8:25 ` John Lister 2010-09-11 19:38 ` Stan Hoeppner 0 siblings, 2 replies; 11+ messages in thread From: Dave Chinner @ 2010-09-11 8:23 UTC (permalink / raw) To: John Lister; +Cc: xfs On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote: > Stan Hoeppner wrote on 9/10/2010 14:00 > >>On 10/09/2010 15:41, John Lister wrote: > >Try unmounting and remounting the filesystem, and see if the various > >tools all report the same thing afterwards. This solved the exact same > >problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs > >2.9.8. > > Cheers, that got rid of most of it, there is still a slight > discrepency (50 extra fragments) which I can live with. xfs_db used buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1 > /proc/sys/vm/drop_caches" before you run xfs_db should make it read from disk at least once.... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 8:23 ` Dave Chinner @ 2010-09-11 8:25 ` John Lister 2010-09-11 19:38 ` Stan Hoeppner 1 sibling, 0 replies; 11+ messages in thread From: John Lister @ 2010-09-11 8:25 UTC (permalink / raw) To: Dave Chinner; +Cc: xfs On 11/09/2010 09:23, Dave Chinner wrote: > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote: >> Stan Hoeppner wrote on 9/10/2010 14:00 >>>> On 10/09/2010 15:41, John Lister wrote: >>> Try unmounting and remounting the filesystem, and see if the various >>> tools all report the same thing afterwards. This solved the exact same >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs >>> 2.9.8. >> Cheers, that got rid of most of it, there is still a slight >> discrepency (50 extra fragments) which I can live with. > xfs_db used buffered IO on the block device, which is not coherent > with the filesystem. If you are using it on an active filesystem, > then running "echo 1> /proc/sys/vm/drop_caches" before you run > xfs_db should make it read from disk at least once.... Ah, that makes more sense, I'll give it a go. Thanks John _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 8:23 ` Dave Chinner 2010-09-11 8:25 ` John Lister @ 2010-09-11 19:38 ` Stan Hoeppner 2010-09-11 19:57 ` Iustin Pop ` (2 more replies) 1 sibling, 3 replies; 11+ messages in thread From: Stan Hoeppner @ 2010-09-11 19:38 UTC (permalink / raw) To: xfs Dave Chinner put forth on 9/11/2010 3:23 AM: > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote: >> Stan Hoeppner wrote on 9/10/2010 14:00 >>>> On 10/09/2010 15:41, John Lister wrote: >>> Try unmounting and remounting the filesystem, and see if the various >>> tools all report the same thing afterwards. This solved the exact same >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs >>> 2.9.8. >> >> Cheers, that got rid of most of it, there is still a slight >> discrepency (50 extra fragments) which I can live with. > > xfs_db used buffered IO on the block device, which is not coherent > with the filesystem. If you are using it on an active filesystem, > then running "echo 1 > /proc/sys/vm/drop_caches" before you run > xfs_db should make it read from disk at least once.... That's good to know Dave. Thanks. I created a short-name script a while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could "clear the baffles" now and then. Looks like it now has multiple uses (if I can just remember to use it in this context). Out of curiosity, who here schedules automatic xfs_fsr runs, and with what frequency? Currently I just run it manually now and then when performance starts to seem sluggish. Thanks. -- Stan _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 19:38 ` Stan Hoeppner @ 2010-09-11 19:57 ` Iustin Pop 2010-09-12 1:01 ` Dave Chinner 2010-09-12 8:43 ` Emmanuel Florac 2010-09-13 8:05 ` Roel van Meer 2 siblings, 1 reply; 11+ messages in thread From: Iustin Pop @ 2010-09-11 19:57 UTC (permalink / raw) To: Stan Hoeppner; +Cc: xfs On Sat, Sep 11, 2010 at 02:38:48PM -0500, Stan Hoeppner wrote: > Dave Chinner put forth on 9/11/2010 3:23 AM: > > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote: > >> Stan Hoeppner wrote on 9/10/2010 14:00 > >>>> On 10/09/2010 15:41, John Lister wrote: > >>> Try unmounting and remounting the filesystem, and see if the various > >>> tools all report the same thing afterwards. This solved the exact same > >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs > >>> 2.9.8. > >> > >> Cheers, that got rid of most of it, there is still a slight > >> discrepency (50 extra fragments) which I can live with. > > > > xfs_db used buffered IO on the block device, which is not coherent > > with the filesystem. If you are using it on an active filesystem, > > then running "echo 1 > /proc/sys/vm/drop_caches" before you run > > xfs_db should make it read from disk at least once.... I wonder if xfs_db shouldn't use by default direct I/O, or at least take a flag to allow it to do direct I/O only against the blocks it needs. Dropping the entire caches on a big box is not nice :) > That's good to know Dave. Thanks. I created a short-name script a > while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could > "clear the baffles" now and then. Looks like it now has multiple uses > (if I can just remember to use it in this context). > > Out of curiosity, who here schedules automatic xfs_fsr runs, and with > what frequency? Currently I just run it manually now and then when > performance starts to seem sluggish. This is my setup, on a machine with multiple XFS filesystems: $ cat /etc/cron.d/xfsfsr PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin 0 01 * * 7 root xfs_fsr -v I've set it up a few years back and, except for a few rare bugs in 2.6.x which are then fixed in 2.6.x.y, I've had no issues. regards, iustin _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 19:57 ` Iustin Pop @ 2010-09-12 1:01 ` Dave Chinner 0 siblings, 0 replies; 11+ messages in thread From: Dave Chinner @ 2010-09-12 1:01 UTC (permalink / raw) To: Stan Hoeppner, xfs On Sat, Sep 11, 2010 at 09:57:02PM +0200, Iustin Pop wrote: > On Sat, Sep 11, 2010 at 02:38:48PM -0500, Stan Hoeppner wrote: > > Dave Chinner put forth on 9/11/2010 3:23 AM: > > > On Sat, Sep 11, 2010 at 07:55:32AM +0100, John Lister wrote: > > >> Stan Hoeppner wrote on 9/10/2010 14:00 > > >>>> On 10/09/2010 15:41, John Lister wrote: > > >>> Try unmounting and remounting the filesystem, and see if the various > > >>> tools all report the same thing afterwards. This solved the exact same > > >>> problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs > > >>> 2.9.8. > > >> > > >> Cheers, that got rid of most of it, there is still a slight > > >> discrepency (50 extra fragments) which I can live with. > > > > > > xfs_db used buffered IO on the block device, which is not coherent > > > with the filesystem. If you are using it on an active filesystem, > > > then running "echo 1 > /proc/sys/vm/drop_caches" before you run > > > xfs_db should make it read from disk at least once.... > > I wonder if xfs_db shouldn't use by default direct I/O, or at least take > a flag to allow it to do direct I/O only against the blocks it needs. > Dropping the entire caches on a big box is not nice :) True - I need to look at whether posix_fadvise(POSIX_FADV_DONTNEED) will clear the bdev pages, and if so that is the easiest solution. Rewriting xfs_db to use direct IO is a pretty major undertaking... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 19:38 ` Stan Hoeppner 2010-09-11 19:57 ` Iustin Pop @ 2010-09-12 8:43 ` Emmanuel Florac 2010-09-12 9:45 ` Stan Hoeppner 2010-09-13 8:05 ` Roel van Meer 2 siblings, 1 reply; 11+ messages in thread From: Emmanuel Florac @ 2010-09-12 8:43 UTC (permalink / raw) To: Stan Hoeppner; +Cc: xfs Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez: > That's good to know Dave. Thanks. I created a short-name script a > while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could > "clear the baffles" now and then. But isn't it somewhat risky? isn't there any risk of data loss by dropping cache at any moment? -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-12 8:43 ` Emmanuel Florac @ 2010-09-12 9:45 ` Stan Hoeppner 0 siblings, 0 replies; 11+ messages in thread From: Stan Hoeppner @ 2010-09-12 9:45 UTC (permalink / raw) To: xfs Emmanuel Florac put forth on 9/12/2010 3:43 AM: > Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez: > >> That's good to know Dave. Thanks. I created a short-name script a >> while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could >> "clear the baffles" now and then. > > But isn't it somewhat risky? isn't there any risk of data loss by > dropping cache at any moment? From: http://linux-mm.org/Drop_Caches This is a non-destructive operation and will only free things that are completely unused. Dirty objects will continue to be in use until written out to disk and are not freeable. If you run "sync" first to flush them out to disk, these drop operations will tend to free more memory. -- Stan _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: XFS defragmentation issue 2010-09-11 19:38 ` Stan Hoeppner 2010-09-11 19:57 ` Iustin Pop 2010-09-12 8:43 ` Emmanuel Florac @ 2010-09-13 8:05 ` Roel van Meer 2 siblings, 0 replies; 11+ messages in thread From: Roel van Meer @ 2010-09-13 8:05 UTC (permalink / raw) To: xfs Stan Hoeppner writes: > Out of curiosity, who here schedules automatic xfs_fsr runs, and with > what frequency? Currently I just run it manually now and then when > performance starts to seem sluggish. I run it daily during the week, at a time when load is practically nil, on two boxes with 24TB storage each, divided into six parts of about 4GB. A 10-pass xfs_fsr run usually completes in under one hour, except on monday, when it takes three to four hours. I don't have any stats on the net effect, though. Regards, roel _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2010-09-13 8:04 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-09-10 14:41 XFS defragmentation issue John Lister 2010-09-10 19:00 ` Stan Hoeppner 2010-09-11 6:55 ` John Lister 2010-09-11 8:23 ` Dave Chinner 2010-09-11 8:25 ` John Lister 2010-09-11 19:38 ` Stan Hoeppner 2010-09-11 19:57 ` Iustin Pop 2010-09-12 1:01 ` Dave Chinner 2010-09-12 8:43 ` Emmanuel Florac 2010-09-12 9:45 ` Stan Hoeppner 2010-09-13 8:05 ` Roel van Meer
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.