[Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Daniel Sangorrin]

When multiple boards try to connect to the same server
perform up to 6 retries with 60 seconds wait before giving up.
Note, that the default specs use a test duration of about 64
seconds (first 4 are discarded from the final results).

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Benchmark.iperf3/fuego_test.sh | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh b/engine/tests/Benchmark.iperf3/fuego_test.sh
index 4c27ce1..6876d90 100755
--- a/engine/tests/Benchmark.iperf3/fuego_test.sh
+++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
@@ -19,9 +19,24 @@ function test_run {
         echo "ERROR: set the server ip on the spec or board file"
         return 1
     fi
-    echo "Using server ip address: $IPERF3_SERVER_IP"
-    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
-    cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./iperf3 -V -c $IPERF3_SERVER_IP -f M -J --logfile $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output $BENCHMARK_IPERF3_CLIENT_PARAMS"
+
+    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR, SERVER IP: $IPERF3_SERVER_IP)"
+    json_file=$(mktemp)
+    for i in 1 2 3 4 5 6; do
+        cmd "$BOARD_TESTDIR/fuego.$TESTDIR/iperf3 -V -c $IPERF3_SERVER_IP -J --logfile $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output $BENCHMARK_IPERF3_CLIENT_PARAMS" && break || \
+            echo "The server seems busy running another iperf3 test. Trying again in 30 seconds"
+
+        # that was our last try so abort the job
+        if [ $i -eq 6 ]; then
+            abort_job "The server seems busy running another iperf3 test."
+        fi
+
+        # remove the json file before retrying, otherwise it gets appended
+        cmd "rm -f $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
+        sleep 60
+    done
+    rm -f $json_file
+
     report "cat $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
 }
 
-- 
2.7.4

Re: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Tim.Bird]

Daniel,

Just a quick note on this and your dbench patches.  I didn't have time
to do a full review or test these on Friday, but at a quick glance they look
OK.  I'll try to get to them on Monday.  Thanks for the patches!

I did some big changes to 'ftc' to support execution outside the docker
container, and I started to clean up some of the coding style issues in
ftc as well.  I'll announce those early next week as well.

I think we should have an e-mail discussion about priorities for the 1.3
release next week.  I've seen some breakages that we need to go back
and fix, before cutting a new release.  This release I experimented with

 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, March 23, 2018 12:02 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the
> same server
> 
> When multiple boards try to connect to the same server
> perform up to 6 retries with 60 seconds wait before giving up.
> Note, that the default specs use a test duration of about 64
> seconds (first 4 are discarded from the final results).
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Benchmark.iperf3/fuego_test.sh | 21 ++++++++++++++++++-
> --
>  1 file changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> b/engine/tests/Benchmark.iperf3/fuego_test.sh
> index 4c27ce1..6876d90 100755
> --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> @@ -19,9 +19,24 @@ function test_run {
>          echo "ERROR: set the server ip on the spec or board file"
>          return 1
>      fi
> -    echo "Using server ip address: $IPERF3_SERVER_IP"
> -    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> -    cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./iperf3 -V -c
> $IPERF3_SERVER_IP -f M -J --logfile
> $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> $BENCHMARK_IPERF3_CLIENT_PARAMS"
> +
> +    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR, SERVER IP:
> $IPERF3_SERVER_IP)"
> +    json_file=$(mktemp)
> +    for i in 1 2 3 4 5 6; do
> +        cmd "$BOARD_TESTDIR/fuego.$TESTDIR/iperf3 -V -c
> $IPERF3_SERVER_IP -J --logfile
> $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> $BENCHMARK_IPERF3_CLIENT_PARAMS" && break || \
> +            echo "The server seems busy running another iperf3 test. Trying again
> in 30 seconds"
> +
> +        # that was our last try so abort the job
> +        if [ $i -eq 6 ]; then
> +            abort_job "The server seems busy running another iperf3 test."
> +        fi
> +
> +        # remove the json file before retrying, otherwise it gets appended
> +        cmd "rm -f $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
> +        sleep 60
> +    done
> +    rm -f $json_file
> +
>      report "cat $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
>  }
> 
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Tim.Bird]

> -----Original Message-----
> From: Tim.Bird 
> Daniel,
> 
> Just a quick note on this and your dbench patches.  I didn't have time
> to do a full review or test these on Friday, but at a quick glance they look
> OK.  I'll try to get to them on Monday.  Thanks for the patches!
> 
> I did some big changes to 'ftc' to support execution outside the docker
> container, and I started to clean up some of the coding style issues in
> ftc as well.  I'll announce those early next week as well.
> 
> I think we should have an e-mail discussion about priorities for the 1.3
> release next week.  I've seen some breakages that we need to go back
> and fix, before cutting a new release.  This release I experimented with
(sorry - email sent prematurely)

new toolchains, and the new release testing code by profusion, as well
as some new report generation stuff, and running ftc both inside
and outside the docker container.

We'll need to see which of those we should try to fix for the 1.3 release,
and which to postpone for a future release.  Probably the profusion stuff
will need to wait, as it's diverged a bit and I think it will be a big effort to
integrate it.
 -- Tim

Re: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Tim.Bird]

This is OK for now.  In the future (unknown when), we should add support
for host-side resource reservation and release.  That might help with this situation
and others where a host-side service, feature or program should only be used
by one board at a time.

Maybe something like "ftc reserve iperf3_server" in run_test, before
executing the iperf3 client, followed by
"ftc release iperf3_server" in test_cleanup.

'ftc reserver <resource_name>' would reserve the resource, and if
already reserved wait until it is freed, and return immediately.  This could
be implemented with some kind of global reservation file pretty easily.

But since this doesn't exist yet, this is a good enough solution.
Applied.

 -- Tim

> -----Original Message-----
> From: Daniel Sangorrin on Friday, March 23, 2018 12:02 AM
> 
> When multiple boards try to connect to the same server
> perform up to 6 retries with 60 seconds wait before giving up.
> Note, that the default specs use a test duration of about 64
> seconds (first 4 are discarded from the final results).
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Benchmark.iperf3/fuego_test.sh | 21 ++++++++++++++++++-
> --
>  1 file changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> b/engine/tests/Benchmark.iperf3/fuego_test.sh
> index 4c27ce1..6876d90 100755
> --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> @@ -19,9 +19,24 @@ function test_run {
>          echo "ERROR: set the server ip on the spec or board file"
>          return 1
>      fi
> -    echo "Using server ip address: $IPERF3_SERVER_IP"
> -    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> -    cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./iperf3 -V -c
> $IPERF3_SERVER_IP -f M -J --logfile
> $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> $BENCHMARK_IPERF3_CLIENT_PARAMS"
> +
> +    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR, SERVER IP:
> $IPERF3_SERVER_IP)"
> +    json_file=$(mktemp)
> +    for i in 1 2 3 4 5 6; do
> +        cmd "$BOARD_TESTDIR/fuego.$TESTDIR/iperf3 -V -c
> $IPERF3_SERVER_IP -J --logfile
> $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> $BENCHMARK_IPERF3_CLIENT_PARAMS" && break || \
> +            echo "The server seems busy running another iperf3 test. Trying again
> in 30 seconds"
> +
> +        # that was our last try so abort the job
> +        if [ $i -eq 6 ]; then
> +            abort_job "The server seems busy running another iperf3 test."
> +        fi
> +
> +        # remove the json file before retrying, otherwise it gets appended
> +        cmd "rm -f $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
> +        sleep 60
> +    done
> +    rm -f $json_file
> +
>      report "cat $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
>  }
> 
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server

[Daniel Sangorrin]

> -----Original Message-----
> From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> Sent: Tuesday, March 27, 2018 8:11 AM
> To: daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH v2] iperf3: manage mutual exclusion when using the same server
> 
> This is OK for now.  In the future (unknown when), we should add support
> for host-side resource reservation and release.  That might help with this situation
> and others where a host-side service, feature or program should only be used
> by one board at a time.
> 
> Maybe something like "ftc reserve iperf3_server" in run_test, before
> executing the iperf3 client, followed by
> "ftc release iperf3_server" in test_cleanup.
> 
> 'ftc reserver <resource_name>' would reserve the resource, and if
> already reserved wait until it is freed, and return immediately.  This could
> be implemented with some kind of global reservation file pretty easily.
> 
> But since this doesn't exist yet, this is a good enough solution.
> Applied.

I like the idea. I will think about it.

Thanks,
Daniel



> 
>  -- Tim
> 
> > -----Original Message-----
> > From: Daniel Sangorrin on Friday, March 23, 2018 12:02 AM
> >
> > When multiple boards try to connect to the same server
> > perform up to 6 retries with 60 seconds wait before giving up.
> > Note, that the default specs use a test duration of about 64
> > seconds (first 4 are discarded from the final results).
> >
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > ---
> >  engine/tests/Benchmark.iperf3/fuego_test.sh | 21 ++++++++++++++++++-
> > --
> >  1 file changed, 18 insertions(+), 3 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > index 4c27ce1..6876d90 100755
> > --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > @@ -19,9 +19,24 @@ function test_run {
> >          echo "ERROR: set the server ip on the spec or board file"
> >          return 1
> >      fi
> > -    echo "Using server ip address: $IPERF3_SERVER_IP"
> > -    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> > -    cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./iperf3 -V -c
> > $IPERF3_SERVER_IP -f M -J --logfile
> > $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> > $BENCHMARK_IPERF3_CLIENT_PARAMS"
> > +
> > +    echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR, SERVER IP:
> > $IPERF3_SERVER_IP)"
> > +    json_file=$(mktemp)
> > +    for i in 1 2 3 4 5 6; do
> > +        cmd "$BOARD_TESTDIR/fuego.$TESTDIR/iperf3 -V -c
> > $IPERF3_SERVER_IP -J --logfile
> > $BOARD_TESTDIR/fuego.$TESTDIR/output.json --get-server-output
> > $BENCHMARK_IPERF3_CLIENT_PARAMS" && break || \
> > +            echo "The server seems busy running another iperf3 test. Trying again
> > in 30 seconds"
> > +
> > +        # that was our last try so abort the job
> > +        if [ $i -eq 6 ]; then
> > +            abort_job "The server seems busy running another iperf3 test."
> > +        fi
> > +
> > +        # remove the json file before retrying, otherwise it gets appended
> > +        cmd "rm -f $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
> > +        sleep 60
> > +    done
> > +    rm -f $json_file
> > +
> >      report "cat $BOARD_TESTDIR/fuego.$TESTDIR/output.json"
> >  }
> >
> > --
> > 2.7.4
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego