* [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? @ 2012-01-03 23:44 Dave Chinner 2012-01-04 17:18 ` Alex Elder 0 siblings, 1 reply; 3+ messages in thread From: Dave Chinner @ 2012-01-03 23:44 UTC (permalink / raw) To: lsf-pc; +Cc: linux-fsdevel, xfs Given that more people are using xfstests and developing tests, we need to consider how to make it friendlier to hack on. The current structure of the tree is difficult to work with, the way tests are organised and numbered make it difficult to co-ordinate new tests and results in patch conflicts, etc. We also see problems arising from people not really understanding how the xfstests harness is designed and how it really is supposed to work, so an overview of the underlying principles of operation would probably be helpful to a lot of people. It will also save review and rework time if we can avoid having people make the same mistakes the first time they submit tests.... I'd also like to discuss some potential infrastructure changes to make it easier to add new tests without conflicts with others developing new tests. Some of the ideas Christoph and I have previously tossed around include: - break tests up into groups in their own subdirectories. e.g. generic tests, xfs/ext4/btrfs specific tests, stress tests, performance tests, large FS tests, etc - change the way we define groups of tests so we don't have a single registry of tests and their groups - allow different naming of tests, such as desciptive text names rather than just plain numbers - allow duplicate test names in different groups I'm sure that other users of xfstests will have some ideas on how to improve it for the way they run it, so I'd like to gather and incorporate these ideas into any structural change we make to xfstests. Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? 2012-01-03 23:44 [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? Dave Chinner @ 2012-01-04 17:18 ` Alex Elder 2012-01-04 20:35 ` Dave Chinner 0 siblings, 1 reply; 3+ messages in thread From: Alex Elder @ 2012-01-04 17:18 UTC (permalink / raw) To: Dave Chinner; +Cc: lsf-pc, linux-fsdevel, xfs On Wed, 2012-01-04 at 10:44 +1100, Dave Chinner wrote: > Given that more people are using xfstests and developing tests, we > need to consider how to make it friendlier to hack on. The current > structure of the tree is difficult to work with, the way tests are > organised and numbered make it difficult to co-ordinate new tests > and results in patch conflicts, etc. Coordination of numbers is not a big deal, the test names/numbers can be easily fixed up at commit time. I also thought that the numbers--though meaningless on their own--also avoided having to decide where a particular test belongs. I.e., a test that exercises several categories of things (maybe preallocation, quota, and ENOSPC) won't be hidden in any sort of "enospc" test directory. I do think the growing number of tests is making it a bit unwieldy though, so I think some sort of reorganization is a good plan. > We also see problems arising from people not really understanding how > the xfstests harness is designed and how it really is supposed to > work, so an overview of the underlying principles of operation would > probably be helpful to a lot of people. It will also save > review and rework time if we can avoid having people make the same > mistakes the first time they submit tests.... This is very important. And the gist of it ought to be captured somewhere if it is not already. > I'd also like to discuss some potential infrastructure changes to > make it easier to add new tests without conflicts with others > developing new tests. Some of the ideas Christoph and I have > previously tossed around include: > > - break tests up into groups in their own subdirectories. > e.g. generic tests, xfs/ext4/btrfs specific tests, stress > tests, performance tests, large FS tests, etc > - change the way we define groups of tests so we don't have > a single registry of tests and their groups > - allow different naming of tests, such as desciptive text > names rather than just plain numbers > - allow duplicate test names in different groups Despite what I said above, I don't disagree with any of this. Perhaps the tests can be buried in one or more subdirectories, but each FSTYP defines its own groups file to drive testing. > I'm sure that other users of xfstests will have some ideas on how to > improve it for the way they run it, so I'd like to gather and > incorporate these ideas into any structural change we make to > xfstests. Should be a good discussion. It might be useful to have a proposal or two to work with as a starting point, or maybe an outline of the types of changes (naming, directory structure, etc.), to help keep things focused. -Alex > Cheers, > > Dave. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? 2012-01-04 17:18 ` Alex Elder @ 2012-01-04 20:35 ` Dave Chinner 0 siblings, 0 replies; 3+ messages in thread From: Dave Chinner @ 2012-01-04 20:35 UTC (permalink / raw) To: Alex Elder; +Cc: lsf-pc, linux-fsdevel, xfs On Wed, Jan 04, 2012 at 11:18:11AM -0600, Alex Elder wrote: > On Wed, 2012-01-04 at 10:44 +1100, Dave Chinner wrote: > > Given that more people are using xfstests and developing tests, we > > need to consider how to make it friendlier to hack on. The current > > structure of the tree is difficult to work with, the way tests are > > organised and numbered make it difficult to co-ordinate new tests > > and results in patch conflicts, etc. > > Coordination of numbers is not a big deal, the test names/numbers > can be easily fixed up at commit time. I also thought that the > numbers--though meaningless on their own--also avoided having to > decide where a particular test belongs. Instead, we decide that via the global group file and via definitions within the test itself. Yes, the global group file allows tests to exist in multiple logical test groups, but that's something that canbe maintained even with a broken up deirectory heirarchy. > I.e., a test that exercises > several categories of things (maybe preallocation, quota, and ENOSPC) > won't be hidden in any sort of "enospc" test directory. No, but it will be a generic of filesystem specific test, so woul dbe located in such a subdirectory. Basically, I'm thinking of something more fine grained like this: generic/group /001 /005 .... xfs/group /002 /003 .... So that we're not treading on each other's toes as much. Maintaining the group files still allows logical grouping of tests (like auto, quota, etc) so I don't see any change in functionality there. Maybe someone will have a good idea to improve the logical grouping implementation, but right now I'm not planning on killing that functionality at all... > > make it easier to add new tests without conflicts with others > > developing new tests. Some of the ideas Christoph and I have > > previously tossed around include: > > > > - break tests up into groups in their own subdirectories. > > e.g. generic tests, xfs/ext4/btrfs specific tests, stress > > tests, performance tests, large FS tests, etc > > - change the way we define groups of tests so we don't have > > a single registry of tests and their groups > > - allow different naming of tests, such as desciptive text > > names rather than just plain numbers > > - allow duplicate test names in different groups > > Despite what I said above, I don't disagree with any of this. > Perhaps the tests can be buried in one or more subdirectories, > but each FSTYP defines its own groups file to drive testing. I think allowing duplicate tests is a bad thing - keeping the logical grouping effectively allows us to only keep one copy of a test no matter where it ends up. > > I'm sure that other users of xfstests will have some ideas on how to > > improve it for the way they run it, so I'd like to gather and > > incorporate these ideas into any structural change we make to > > xfstests. > > Should be a good discussion. It might be useful to have a > proposal or two to work with as a starting point, or maybe > an outline of the types of changes (naming, directory > structure, etc.), to help keep things focused. Sure, though I don't want to done so much that it's all set in stone before I found out what other people consider pain points that need to be fixed... Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2012-01-04 20:36 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-01-03 23:44 [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? Dave Chinner 2012-01-04 17:18 ` Alex Elder 2012-01-04 20:35 ` Dave Chinner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).