* [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
@ 2013-03-21 10:59 Jan Schmidt
2013-03-21 19:50 ` Dave Chinner
2013-05-09 19:50 ` Rich Johnston
0 siblings, 2 replies; 10+ messages in thread
From: Jan Schmidt @ 2013-03-21 10:59 UTC (permalink / raw)
To: xfs
From: Jan Schmidt <list.btrfs@jan-o-sch.net>
This patch adds execution of a custom command in the middle of all fsstress
operations. Its intended use is the creation of snapshots in the middle of a
test run.
Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
---
ltp/fsstress.c | 26 ++++++++++++++++++++++++--
1 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/ltp/fsstress.c b/ltp/fsstress.c
index b4cfb25..5d5611f 100644
--- a/ltp/fsstress.c
+++ b/ltp/fsstress.c
@@ -247,6 +247,8 @@ unsigned long seed = 0;
ino_t top_ino;
int verbose = 0;
sig_atomic_t should_stop = 0;
+char *execute_cmd = NULL;
+int execute_freq = 1;
void add_to_flist(int, int, int);
void append_pathname(pathname_t *, char *);
@@ -313,13 +315,14 @@ int main(int argc, char **argv)
int nousage = 0;
xfs_error_injection_t err_inj;
struct sigaction action;
+ const char *allopts = "d:e:f:i:m:M:n:o:p:rs:S:vwx:X:zH";
errrange = errtag = 0;
umask(0);
nops = sizeof(ops) / sizeof(ops[0]);
ops_end = &ops[nops];
myprog = argv[0];
- while ((c = getopt(argc, argv, "d:e:f:i:m:M:n:o:p:rs:S:vwzH")) != -1) {
+ while ((c = getopt(argc, argv, allopts)) != -1) {
switch (c) {
case 'd':
dirname = optarg;
@@ -376,6 +379,9 @@ int main(int argc, char **argv)
case 'w':
write_freq();
break;
+ case 'x':
+ execute_cmd = optarg;
+ break;
case 'z':
zero_freq();
break;
@@ -390,6 +396,9 @@ int main(int argc, char **argv)
printf("\n");
nousage=1;
break;
+ case 'X':
+ execute_freq = strtoul(optarg, NULL, 0);
+ break;
case '?':
fprintf(stderr, "%s - invalid parameters\n",
myprog);
@@ -765,7 +774,9 @@ doproc(void)
int opno;
int rval;
opdesc_t *p;
+ int dividend;
+ dividend = (operations + execute_freq) / (execute_freq + 1);
sprintf(buf, "p%x", procid);
(void)mkdir(buf, 0777);
if (chdir(buf) < 0 || stat64(".", &statbuf) < 0) {
@@ -779,6 +790,15 @@ doproc(void)
if (namerand)
namerand = random();
for (opno = 0; opno < operations; opno++) {
+ if (execute_cmd && opno && opno % dividend == 0) {
+ if (verbose)
+ printf("%d: execute command %s\n", opno,
+ execute_cmd);
+ rval = system(execute_cmd);
+ if (rval)
+ fprintf(stderr, "execute command failed with "
+ "%d\n", rval);
+ }
p = &ops[freq_table[random() % freq_table_size]];
p->func(opno, random());
/*
@@ -1468,7 +1488,7 @@ usage(void)
printf("Usage: %s -H or\n", myprog);
printf(" %s [-d dir][-e errtg][-f op_name=freq][-n nops]\n",
myprog);
- printf(" [-p nproc][-r len][-s seed][-v][-w][-z][-S]\n");
+ printf(" [-p nproc][-r len][-s seed][-v][-w][-x cmd][-z][-S][-X ncmd]\n");
printf("where\n");
printf(" -d dir specifies the base directory for operations\n");
printf(" -e errtg specifies error injection stuff\n");
@@ -1483,8 +1503,10 @@ usage(void)
printf(" -s seed specifies the seed for the random generator (default random)\n");
printf(" -v specifies verbose mode\n");
printf(" -w zeros frequencies of non-write operations\n");
+ printf(" -x cmd execute command in the middle of operations\n");
printf(" -z zeros frequencies of all operations\n");
printf(" -S [c,t] prints the list of operations (omitting zero frequency) in command line or table style\n");
+ printf(" -X ncmd number of calls to the -x command (default 1)\n");
printf(" -H prints usage and exits\n");
}
--
1.7.2.2
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-21 10:59 [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options) Jan Schmidt
@ 2013-03-21 19:50 ` Dave Chinner
2013-03-21 20:51 ` Jan Schmidt
2013-05-09 19:50 ` Rich Johnston
1 sibling, 1 reply; 10+ messages in thread
From: Dave Chinner @ 2013-03-21 19:50 UTC (permalink / raw)
To: Jan Schmidt; +Cc: xfs
On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
>
> This patch adds execution of a custom command in the middle of all fsstress
> operations. Its intended use is the creation of snapshots in the middle of a
> test run.
Why do you need fsstress to do this? Why can't you just run fsstress
in the background and run a loop creating periodic snapshots in the
control script?
Also, did you intend that every process creates a snapshot? i.e. it
looks lik eif you run a 1000 processes, they'll all run a snapshot
operation at X operations? i.e. this will generate nproc * X
snapshots in a single run. This doesn't seem very wise to me....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-21 19:50 ` Dave Chinner
@ 2013-03-21 20:51 ` Jan Schmidt
2013-03-21 21:12 ` Dave Chinner
0 siblings, 1 reply; 10+ messages in thread
From: Jan Schmidt @ 2013-03-21 20:51 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On 21.03.2013 20:50, Dave Chinner wrote:
> On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
>> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
>>
>> This patch adds execution of a custom command in the middle of all fsstress
>> operations. Its intended use is the creation of snapshots in the middle of a
>> test run.
>
> Why do you need fsstress to do this? Why can't you just run fsstress
> in the background and run a loop creating periodic snapshots in the
> control script?
Because I want reproducible results. Same random seed should result in
the very same snapshots being created.
> Also, did you intend that every process creates a snapshot? i.e. it
> looks lik eif you run a 1000 processes, they'll all run a snapshot
> operation at X operations? i.e. this will generate nproc * X
> snapshots in a single run. This doesn't seem very wise to me....
Agreed, I haven't thought of running more than one process. For the sake
of reproducibility, I wouldn't want multiple processes for my test case
either.
I'm not sure if there are other applications than snapshot creation for
such a feature, so I cannot argue whether to have each process execute
such a command or not.
Thanks,
-Jan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-21 20:51 ` Jan Schmidt
@ 2013-03-21 21:12 ` Dave Chinner
2013-03-22 7:06 ` Jan Schmidt
0 siblings, 1 reply; 10+ messages in thread
From: Dave Chinner @ 2013-03-21 21:12 UTC (permalink / raw)
To: Jan Schmidt; +Cc: xfs
On Thu, Mar 21, 2013 at 09:51:05PM +0100, Jan Schmidt wrote:
>
>
> On 21.03.2013 20:50, Dave Chinner wrote:
> > On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
> >> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
> >>
> >> This patch adds execution of a custom command in the middle of all fsstress
> >> operations. Its intended use is the creation of snapshots in the middle of a
> >> test run.
> >
> > Why do you need fsstress to do this? Why can't you just run fsstress
> > in the background and run a loop creating periodic snapshots in the
> > control script?
>
> Because I want reproducible results. Same random seed should result in
> the very same snapshots being created.
Why can't you run fsstress for N operations, run a snapshot,
then run it again for M operations? That will give you exactly the
same results, wouldn't it?
> > Also, did you intend that every process creates a snapshot? i.e. it
> > looks lik eif you run a 1000 processes, they'll all run a snapshot
> > operation at X operations? i.e. this will generate nproc * X
> > snapshots in a single run. This doesn't seem very wise to me....
>
> Agreed, I haven't thought of running more than one process. For the sake
> of reproducibility, I wouldn't want multiple processes for my test case
> either.
>
> I'm not sure if there are other applications than snapshot creation for
> such a feature, so I cannot argue whether to have each process execute
> such a command or not.
If such a feature is necessary, I'd suggest that implementing the
snapshot ioctl as just another operation directly into fsstress is
probably a better way to implement this functionality. That way you
can control the frequency via the command line in exactly the same
way as every other operation....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-21 21:12 ` Dave Chinner
@ 2013-03-22 7:06 ` Jan Schmidt
2013-03-24 23:51 ` Dave Chinner
0 siblings, 1 reply; 10+ messages in thread
From: Jan Schmidt @ 2013-03-22 7:06 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Thu, March 21, 2013 at 22:12 (+0100), Dave Chinner wrote:> On Thu, Mar 21,
2013 at 09:51:05PM +0100, Jan Schmidt wrote:
>>
>>
>> On 21.03.2013 20:50, Dave Chinner wrote:
>>> On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
>>>> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
>>>>
>>>> This patch adds execution of a custom command in the middle of all fsstress
>>>> operations. Its intended use is the creation of snapshots in the middle of a
>>>> test run.
>>>
>>> Why do you need fsstress to do this? Why can't you just run fsstress
>>> in the background and run a loop creating periodic snapshots in the
>>> control script?
>>
>> Because I want reproducible results. Same random seed should result in
>> the very same snapshots being created.
>
> Why can't you run fsstress for N operations, run a snapshot,
> then run it again for M operations? That will give you exactly the
> same results, wouldn't it?
As far as I have understood what fsstress does, the second run would generate
different filenames, i.e. it would never rename / truncate / punch holes into /
... files created by the first run - it cannot even know that they exist.
>>> Also, did you intend that every process creates a snapshot? i.e. it
>>> looks lik eif you run a 1000 processes, they'll all run a snapshot
>>> operation at X operations? i.e. this will generate nproc * X
>>> snapshots in a single run. This doesn't seem very wise to me....
>>
>> Agreed, I haven't thought of running more than one process. For the sake
>> of reproducibility, I wouldn't want multiple processes for my test case
>> either.
>>
>> I'm not sure if there are other applications than snapshot creation for
>> such a feature, so I cannot argue whether to have each process execute
>> such a command or not.
>
> If such a feature is necessary, I'd suggest that implementing the
> snapshot ioctl as just another operation directly into fsstress is
> probably a better way to implement this functionality. That way you
> can control the frequency via the command line in exactly the same
> way as every other operation....
What I currently need is a function to make one reasonably weird snapshot. So my
plan goes like this: do n weird operations, make a snapshot (this is going to be
the base snapshot), do n weird operations (partly to the same files), make a
second snapshot (this is going to be the incremental snapshot, I create that one
myself after fsstress is done, currently). Having both snapshots with an equal
number of modification operations isn't required, however at least a fair number
of operations for each of them is desired.
Adding it as a normal fsstress operation would generate a whole lot of
snapshots. I could, for like 50k operations, scale all the factors for each
operation accordingly to get a single snapshot out of it. I still won't force it
anywhere near the middle that way, though. Also, going from 50k operation to 60k
operations gets cumbersome that way.
Plumbing that into fsstress the way I did is the only solution I could think of
to reach the mentioned goals. If nobody else needs it, I can of course keep it
local, here. However, I'd really like to make an xfstest out of it sooner or
later - currently, we've no test at all for (btrfs) send and receive.
-Jan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-22 7:06 ` Jan Schmidt
@ 2013-03-24 23:51 ` Dave Chinner
2013-04-05 12:07 ` Jan Schmidt
0 siblings, 1 reply; 10+ messages in thread
From: Dave Chinner @ 2013-03-24 23:51 UTC (permalink / raw)
To: Jan Schmidt; +Cc: xfs
On Fri, Mar 22, 2013 at 08:06:49AM +0100, Jan Schmidt wrote:
> On Thu, March 21, 2013 at 22:12 (+0100), Dave Chinner wrote:> On Thu, Mar 21,
> 2013 at 09:51:05PM +0100, Jan Schmidt wrote:
> >>
> >>
> >> On 21.03.2013 20:50, Dave Chinner wrote:
> >>> On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
> >>>> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
> >>>>
> >>>> This patch adds execution of a custom command in the middle of all fsstress
> >>>> operations. Its intended use is the creation of snapshots in the middle of a
> >>>> test run.
> >>>
> >>> Why do you need fsstress to do this? Why can't you just run fsstress
> >>> in the background and run a loop creating periodic snapshots in the
> >>> control script?
> >>
> >> Because I want reproducible results. Same random seed should result in
> >> the very same snapshots being created.
> >
> > Why can't you run fsstress for N operations, run a snapshot,
> > then run it again for M operations? That will give you exactly the
> > same results, wouldn't it?
>
> As far as I have understood what fsstress does, the second run would generate
> different filenames, i.e. it would never rename / truncate / punch holes into /
> ... files created by the first run - it cannot even know that they exist.
Yes, you are right.
> >>> Also, did you intend that every process creates a snapshot? i.e. it
> >>> looks lik eif you run a 1000 processes, they'll all run a snapshot
> >>> operation at X operations? i.e. this will generate nproc * X
> >>> snapshots in a single run. This doesn't seem very wise to me....
> >>
> >> Agreed, I haven't thought of running more than one process. For the sake
> >> of reproducibility, I wouldn't want multiple processes for my test case
> >> either.
> >>
> >> I'm not sure if there are other applications than snapshot creation for
> >> such a feature, so I cannot argue whether to have each process execute
> >> such a command or not.
> >
> > If such a feature is necessary, I'd suggest that implementing the
> > snapshot ioctl as just another operation directly into fsstress is
> > probably a better way to implement this functionality. That way you
> > can control the frequency via the command line in exactly the same
> > way as every other operation....
>
> What I currently need is a function to make one reasonably weird snapshot. So my
> plan goes like this: do n weird operations, make a snapshot (this is going to be
> the base snapshot), do n weird operations (partly to the same files), make a
> second snapshot (this is going to be the incremental snapshot, I create that one
> myself after fsstress is done, currently). Having both snapshots with an equal
> number of modification operations isn't required, however at least a fair number
> of operations for each of them is desired.
Ah, so you're wanting to test incremental backups based on
snapshots. Ok, that context puts it in a different light....
> Adding it as a normal fsstress operation would generate a whole lot of
> snapshots. I could, for like 50k operations, scale all the factors for each
> operation accordingly to get a single snapshot out of it. I still won't force it
> anywhere near the middle that way, though. Also, going from 50k operation to 60k
> operations gets cumbersome that way.
*nod*
> Plumbing that into fsstress the way I did is the only solution I could think of
> to reach the mentioned goals. If nobody else needs it, I can of course keep it
> local, here. However, I'd really like to make an xfstest out of it sooner or
> later - currently, we've no test at all for (btrfs) send and receive.
For send/receive, you should probably start with some basic tests
that are easy to verify first. e.g. the equivalent of the basic
incremental xfsdump/restore tests like 064/065 which do well
defined, easy to verify operations to determine correct behaviour.
I can see the value in adding a random variant in addition to these
basic tests, so I can see how having a predictable callout from
fsstress would be useful for incremental xfsdump/restore testing as
well.
FWIW, what does you current callout execute? A shell script that
runs a bunch of other commands that ends with a btrfs send?
The biggest question I have about this is how to make it valuable
for more types of fsstress execution, especially concurrent
execution. I can't see a use (yet) for a per-process callout, but
I'm wondering if we should have some kind of "wait for all processes
to do N ops, then run the callout" style of synchronisation.
I'm not sure what is best here as I don't know the full context of
what you are wanting to test (and how), but I think we can come up
with something better than "only works for single process
invocations". :)
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-24 23:51 ` Dave Chinner
@ 2013-04-05 12:07 ` Jan Schmidt
2013-05-03 14:43 ` Jan Schmidt
0 siblings, 1 reply; 10+ messages in thread
From: Jan Schmidt @ 2013-04-05 12:07 UTC (permalink / raw)
To: Dave Chinner; +Cc: xfs
On Mon, March 25, 2013 at 00:51 (+0100), Dave Chinner wrote:
> On Fri, Mar 22, 2013 at 08:06:49AM +0100, Jan Schmidt wrote:
>> On Thu, March 21, 2013 at 22:12 (+0100), Dave Chinner wrote:> On Thu, Mar 21,
>> 2013 at 09:51:05PM +0100, Jan Schmidt wrote:
>>>>
>>>>
>>>> On 21.03.2013 20:50, Dave Chinner wrote:
>>>>> On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
>>>>>> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
>>>>>>
>>>>>> This patch adds execution of a custom command in the middle of all fsstress
>>>>>> operations. Its intended use is the creation of snapshots in the middle of a
>>>>>> test run.
>>>>>
>>>>> Why do you need fsstress to do this? Why can't you just run fsstress
>>>>> in the background and run a loop creating periodic snapshots in the
>>>>> control script?
>>>>
>>>> Because I want reproducible results. Same random seed should result in
>>>> the very same snapshots being created.
>>>
>>> Why can't you run fsstress for N operations, run a snapshot,
>>> then run it again for M operations? That will give you exactly the
>>> same results, wouldn't it?
>>
>> As far as I have understood what fsstress does, the second run would generate
>> different filenames, i.e. it would never rename / truncate / punch holes into /
>> ... files created by the first run - it cannot even know that they exist.
>
> Yes, you are right.
>
>>>>> Also, did you intend that every process creates a snapshot? i.e. it
>>>>> looks lik eif you run a 1000 processes, they'll all run a snapshot
>>>>> operation at X operations? i.e. this will generate nproc * X
>>>>> snapshots in a single run. This doesn't seem very wise to me....
>>>>
>>>> Agreed, I haven't thought of running more than one process. For the sake
>>>> of reproducibility, I wouldn't want multiple processes for my test case
>>>> either.
>>>>
>>>> I'm not sure if there are other applications than snapshot creation for
>>>> such a feature, so I cannot argue whether to have each process execute
>>>> such a command or not.
>>>
>>> If such a feature is necessary, I'd suggest that implementing the
>>> snapshot ioctl as just another operation directly into fsstress is
>>> probably a better way to implement this functionality. That way you
>>> can control the frequency via the command line in exactly the same
>>> way as every other operation....
>>
>> What I currently need is a function to make one reasonably weird snapshot. So my
>> plan goes like this: do n weird operations, make a snapshot (this is going to be
>> the base snapshot), do n weird operations (partly to the same files), make a
>> second snapshot (this is going to be the incremental snapshot, I create that one
>> myself after fsstress is done, currently). Having both snapshots with an equal
>> number of modification operations isn't required, however at least a fair number
>> of operations for each of them is desired.
>
> Ah, so you're wanting to test incremental backups based on
> snapshots. Ok, that context puts it in a different light....
>
>> Adding it as a normal fsstress operation would generate a whole lot of
>> snapshots. I could, for like 50k operations, scale all the factors for each
>> operation accordingly to get a single snapshot out of it. I still won't force it
>> anywhere near the middle that way, though. Also, going from 50k operation to 60k
>> operations gets cumbersome that way.
>
> *nod*
>
>> Plumbing that into fsstress the way I did is the only solution I could think of
>> to reach the mentioned goals. If nobody else needs it, I can of course keep it
>> local, here. However, I'd really like to make an xfstest out of it sooner or
>> later - currently, we've no test at all for (btrfs) send and receive.
>
> For send/receive, you should probably start with some basic tests
> that are easy to verify first. e.g. the equivalent of the basic
> incremental xfsdump/restore tests like 064/065 which do well
> defined, easy to verify operations to determine correct behaviour.
That sounds like a good start.
> I can see the value in adding a random variant in addition to these
> basic tests, so I can see how having a predictable callout from
> fsstress would be useful for incremental xfsdump/restore testing as
> well.
>
> FWIW, what does you current callout execute? A shell script that
> runs a bunch of other commands that ends with a btrfs send?
It's basically just "btrfs subvol snapshot", but yeah, for more complex things
I'd put a shell script there.
> The biggest question I have about this is how to make it valuable
> for more types of fsstress execution, especially concurrent
> execution. I can't see a use (yet) for a per-process callout, but
> I'm wondering if we should have some kind of "wait for all processes
> to do N ops, then run the callout" style of synchronisation.
>
> I'm not sure what is best here as I don't know the full context of
> what you are wanting to test (and how), but I think we can come up
> with something better than "only works for single process
> invocations". :)
Well, in fact you do have the full context of what I'm wanting to test, as far
as I can see it.
I bet we could came up with a suggestion how to interpret something like the
proposed -x switch in multi process context. However, I don't like to code for
hypothetic situations I cannot really imagine a use case for. So, the best thing
I came up with is a switch that can do something meaningful in single process
applications of fsstress.
I'm happy to code the rest of it, if a good suggestion comes up how this could
be handled and how it could be useful to others as well.
Thanks!
-Jan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-04-05 12:07 ` Jan Schmidt
@ 2013-05-03 14:43 ` Jan Schmidt
2013-05-09 19:47 ` Rich Johnston
0 siblings, 1 reply; 10+ messages in thread
From: Jan Schmidt @ 2013-05-03 14:43 UTC (permalink / raw)
To: xfs
On Fri, April 05, 2013 at 14:07 (+0200), Jan Schmidt wrote:
>
> On Mon, March 25, 2013 at 00:51 (+0100), Dave Chinner wrote:
>> On Fri, Mar 22, 2013 at 08:06:49AM +0100, Jan Schmidt wrote:
>>> On Thu, March 21, 2013 at 22:12 (+0100), Dave Chinner wrote:> On Thu, Mar 21,
>>> 2013 at 09:51:05PM +0100, Jan Schmidt wrote:
>>>>>
>>>>>
>>>>> On 21.03.2013 20:50, Dave Chinner wrote:
>>>>>> On Thu, Mar 21, 2013 at 11:59:45AM +0100, Jan Schmidt wrote:
>>>>>>> From: Jan Schmidt <list.btrfs@jan-o-sch.net>
>>>>>>>
>>>>>>> This patch adds execution of a custom command in the middle of all fsstress
>>>>>>> operations. Its intended use is the creation of snapshots in the middle of a
>>>>>>> test run.
>>>>>>
>>>>>> Why do you need fsstress to do this? Why can't you just run fsstress
>>>>>> in the background and run a loop creating periodic snapshots in the
>>>>>> control script?
>>>>>
>>>>> Because I want reproducible results. Same random seed should result in
>>>>> the very same snapshots being created.
>>>>
>>>> Why can't you run fsstress for N operations, run a snapshot,
>>>> then run it again for M operations? That will give you exactly the
>>>> same results, wouldn't it?
>>>
>>> As far as I have understood what fsstress does, the second run would generate
>>> different filenames, i.e. it would never rename / truncate / punch holes into /
>>> ... files created by the first run - it cannot even know that they exist.
>>
>> Yes, you are right.
>>
>>>>>> Also, did you intend that every process creates a snapshot? i.e. it
>>>>>> looks lik eif you run a 1000 processes, they'll all run a snapshot
>>>>>> operation at X operations? i.e. this will generate nproc * X
>>>>>> snapshots in a single run. This doesn't seem very wise to me....
>>>>>
>>>>> Agreed, I haven't thought of running more than one process. For the sake
>>>>> of reproducibility, I wouldn't want multiple processes for my test case
>>>>> either.
>>>>>
>>>>> I'm not sure if there are other applications than snapshot creation for
>>>>> such a feature, so I cannot argue whether to have each process execute
>>>>> such a command or not.
>>>>
>>>> If such a feature is necessary, I'd suggest that implementing the
>>>> snapshot ioctl as just another operation directly into fsstress is
>>>> probably a better way to implement this functionality. That way you
>>>> can control the frequency via the command line in exactly the same
>>>> way as every other operation....
>>>
>>> What I currently need is a function to make one reasonably weird snapshot. So my
>>> plan goes like this: do n weird operations, make a snapshot (this is going to be
>>> the base snapshot), do n weird operations (partly to the same files), make a
>>> second snapshot (this is going to be the incremental snapshot, I create that one
>>> myself after fsstress is done, currently). Having both snapshots with an equal
>>> number of modification operations isn't required, however at least a fair number
>>> of operations for each of them is desired.
>>
>> Ah, so you're wanting to test incremental backups based on
>> snapshots. Ok, that context puts it in a different light....
>>
>>> Adding it as a normal fsstress operation would generate a whole lot of
>>> snapshots. I could, for like 50k operations, scale all the factors for each
>>> operation accordingly to get a single snapshot out of it. I still won't force it
>>> anywhere near the middle that way, though. Also, going from 50k operation to 60k
>>> operations gets cumbersome that way.
>>
>> *nod*
>>
>>> Plumbing that into fsstress the way I did is the only solution I could think of
>>> to reach the mentioned goals. If nobody else needs it, I can of course keep it
>>> local, here. However, I'd really like to make an xfstest out of it sooner or
>>> later - currently, we've no test at all for (btrfs) send and receive.
>>
>> For send/receive, you should probably start with some basic tests
>> that are easy to verify first. e.g. the equivalent of the basic
>> incremental xfsdump/restore tests like 064/065 which do well
>> defined, easy to verify operations to determine correct behaviour.
>
> That sounds like a good start.
>
>> I can see the value in adding a random variant in addition to these
>> basic tests, so I can see how having a predictable callout from
>> fsstress would be useful for incremental xfsdump/restore testing as
>> well.
>>
>> FWIW, what does you current callout execute? A shell script that
>> runs a bunch of other commands that ends with a btrfs send?
>
> It's basically just "btrfs subvol snapshot", but yeah, for more complex things
> I'd put a shell script there.
>
>> The biggest question I have about this is how to make it valuable
>> for more types of fsstress execution, especially concurrent
>> execution. I can't see a use (yet) for a per-process callout, but
>> I'm wondering if we should have some kind of "wait for all processes
>> to do N ops, then run the callout" style of synchronisation.
>>
>> I'm not sure what is best here as I don't know the full context of
>> what you are wanting to test (and how), but I think we can come up
>> with something better than "only works for single process
>> invocations". :)
>
> Well, in fact you do have the full context of what I'm wanting to test, as far
> as I can see it.
>
> I bet we could came up with a suggestion how to interpret something like the
> proposed -x switch in multi process context. However, I don't like to code for
> hypothetic situations I cannot really imagine a use case for. So, the best thing
> I came up with is a switch that can do something meaningful in single process
> applications of fsstress.
>
> I'm happy to code the rest of it, if a good suggestion comes up how this could
> be handled and how it could be useful to others as well.
Looks like there are no suggestions how to make -x useful for multiple workers.
Can we then have the single worker solution (original patch) merged for now?
-Jan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-05-03 14:43 ` Jan Schmidt
@ 2013-05-09 19:47 ` Rich Johnston
0 siblings, 0 replies; 10+ messages in thread
From: Rich Johnston @ 2013-05-09 19:47 UTC (permalink / raw)
To: Jan Schmidt; +Cc: xfs
On 05/03/2013 09:43 AM, Jan Schmidt wrote:
>
> Looks like there are no suggestions how to make -x useful for multiple workers.
> Can we then have the single worker solution (original patch) merged for now?
>
> -Jan
I don't see why not. Looks good.
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options)
2013-03-21 10:59 [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options) Jan Schmidt
2013-03-21 19:50 ` Dave Chinner
@ 2013-05-09 19:50 ` Rich Johnston
1 sibling, 0 replies; 10+ messages in thread
From: Rich Johnston @ 2013-05-09 19:50 UTC (permalink / raw)
To: Jan Schmidt; +Cc: xfs
Jan,
Thanks for the patch it has been committed.
--Rich
commit e47ebda011f69fab2c9f3090f50f1cfb9353f7e7
Author: Jan Schmidt <list.btrfs@jan-o-sch.net>
Date: Thu May 9 14:40:00 2013 -0500
xfstests: add execution of a custom command to fsstress (-x and -X
options)
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-05-09 19:50 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-03-21 10:59 [PATCH] xfstests: add execution of a custom command to fsstress (-x and -X options) Jan Schmidt
2013-03-21 19:50 ` Dave Chinner
2013-03-21 20:51 ` Jan Schmidt
2013-03-21 21:12 ` Dave Chinner
2013-03-22 7:06 ` Jan Schmidt
2013-03-24 23:51 ` Dave Chinner
2013-04-05 12:07 ` Jan Schmidt
2013-05-03 14:43 ` Jan Schmidt
2013-05-09 19:47 ` Rich Johnston
2013-05-09 19:50 ` Rich Johnston
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.