[Fuego] [PATCH 1/3] iperf3: remove parameter description from fuego_test

[Fuego] [PATCH 1/3] iperf3: remove parameter description from fuego_test

[Daniel Sangorrin]

The description is provided in test.yaml instead.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Benchmark.iperf3/fuego_test.sh | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh b/engine/tests/Benchmark.iperf3/fuego_test.sh
index 4bbfc22..e981967 100755
--- a/engine/tests/Benchmark.iperf3/fuego_test.sh
+++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
@@ -1,15 +1,6 @@
 # gitrepo=https://github.com/esnet/iperf.git
 tarball=iperf-3.1.3-source.tar.gz
 
-# Spec parameters
-#  - [Optional] server_ip (BENCHMARK_IPERF3_SERVER_IP): ip address of the server machine
-#        - if not provided, then SRV_IP _must_ be provided on the board file. Otherwise the test will fail.
-#        - if the server ip is assigned to the host, the test automatically starts the iperf3 server daemon
-#            - otherwise, the tester _must_ make sure that 'iperf3 -V -s -D' is already running on the server machine.
-#              FIXTHIS: add functionality to setup the server using a board file
-#  - [Optional] server_params (BENCHMARK_IPERF3_SERVER_PARAMS): extra parameters for the server
-#  - [Optional] client_params (BENCHMARK_IPERF3_SERVER_PARAMS): extra parameters for the client
-
 function test_build {
     ./configure --host=$HOST --prefix=$(pwd)/build
     make
@@ -22,7 +13,7 @@ function test_deploy {
 
 function test_run {
     # FIXTHIS: validate the server and client ips, and make sure they can communicate
-
+    # FIXTHIS: add functionality to setup the server using a board file
     # Get the server ip
     IPERF3_SERVER_IP=${BENCHMARK_IPERF3_SERVER_IP:-$SRV_IP}
     if [ -z "$IPERF3_SERVER_IP" ]; then
-- 
2.7.4

[Fuego] [PATCH 2/3] iperf3: compile the binary statically

[Daniel Sangorrin]

I didn't notice this because my targets had the library already
installed.
I tried using --enable-static but didn't work so I also opened
an issue upstream:
https://github.com/esnet/iperf/issues/708

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Benchmark.iperf3/fuego_test.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh b/engine/tests/Benchmark.iperf3/fuego_test.sh
index e981967..09eac86 100755
--- a/engine/tests/Benchmark.iperf3/fuego_test.sh
+++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
@@ -2,13 +2,12 @@
 tarball=iperf-3.1.3-source.tar.gz
 
 function test_build {
-    ./configure --host=$HOST --prefix=$(pwd)/build
+    LDFLAGS=-static ./configure --host=$HOST --prefix=$(pwd)/build
     make
-    make install
 }
 
 function test_deploy {
-    put build/bin/iperf3  $BOARD_TESTDIR/fuego.$TESTDIR/
+    put src/iperf3  $BOARD_TESTDIR/fuego.$TESTDIR/
 }
 
 function test_run {
-- 
2.7.4

[Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps

[Daniel Sangorrin]

The -f/--format is not being used on the JSON output report.
For that reason, I decided to use bps for now and open
an issue upstream:
https://github.com/esnet/iperf/issues/709

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Benchmark.iperf3/fuego_test.sh  | 2 +-
 engine/tests/Benchmark.iperf3/parser.py      | 2 +-
 engine/tests/Benchmark.iperf3/reference.json | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh b/engine/tests/Benchmark.iperf3/fuego_test.sh
index 09eac86..9f10743 100755
--- a/engine/tests/Benchmark.iperf3/fuego_test.sh
+++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
@@ -26,7 +26,7 @@ function test_run {
     if [ "$ROUTE" = "local" ]; then
         echo "Starting iperf3 server on localhost (SERVER IP: $IPERF3_SERVER_IP)"
         killall iperf3 2> /dev/null || true
-        iperf3 -V -s -D -f M $BENCHMARK_IPERF3_SERVER_PARAMS
+        iperf3 -V -s -D $BENCHMARK_IPERF3_SERVER_PARAMS
     fi
 
     echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
diff --git a/engine/tests/Benchmark.iperf3/parser.py b/engine/tests/Benchmark.iperf3/parser.py
index 1097803..a91b88a 100755
--- a/engine/tests/Benchmark.iperf3/parser.py
+++ b/engine/tests/Benchmark.iperf3/parser.py
@@ -51,7 +51,7 @@ with open(plib.TEST_LOG) as f:
     plt.plot(time, bits_per_second)
     plt.title('iperf3 client results')
     plt.xlabel('time (s)')
-    plt.ylabel('Mbps')
+    plt.ylabel('Bits per second (bps)')
     fig.savefig(os.environ['LOGDIR'] + '/iperf3.png')
 
 sys.exit(plib.process(measurements))
diff --git a/engine/tests/Benchmark.iperf3/reference.json b/engine/tests/Benchmark.iperf3/reference.json
index 930a94d..f10dc3a 100644
--- a/engine/tests/Benchmark.iperf3/reference.json
+++ b/engine/tests/Benchmark.iperf3/reference.json
@@ -8,7 +8,7 @@
                     "measurements":[
                         {
                             "name":"bits_per_second",
-                            "unit":"Mbps"
+                            "unit":"bps"
                         },
                         {
                             "name":"retransmits",
@@ -21,7 +21,7 @@
                     "measurements":[
                         {
                             "name":"bits_per_second",
-                            "unit":"Mbps"
+                            "unit":"bps"
                         }
                     ]
                 },
@@ -30,7 +30,7 @@
                     "measurements":[
                         {
                             "name":"bits_per_second",
-                            "unit":"Mbps"
+                            "unit":"bps"
                         },
                         {
                             "name":"jitter_ms",
-- 
2.7.4

Re: [Fuego] [PATCH 1/3] iperf3: remove parameter description from fuego_test

[Tim.Bird]

Applied. Thanks.
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, March 02, 2018 12:57 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 1/3] iperf3: remove parameter description from
> fuego_test
> 
> The description is provided in test.yaml instead.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Benchmark.iperf3/fuego_test.sh | 11 +----------
>  1 file changed, 1 insertion(+), 10 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> b/engine/tests/Benchmark.iperf3/fuego_test.sh
> index 4bbfc22..e981967 100755
> --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> @@ -1,15 +1,6 @@
>  # gitrepo=https://github.com/esnet/iperf.git
>  tarball=iperf-3.1.3-source.tar.gz
> 
> -# Spec parameters
> -#  - [Optional] server_ip (BENCHMARK_IPERF3_SERVER_IP): ip address of
> the server machine
> -#        - if not provided, then SRV_IP _must_ be provided on the board file.
> Otherwise the test will fail.
> -#        - if the server ip is assigned to the host, the test automatically starts
> the iperf3 server daemon
> -#            - otherwise, the tester _must_ make sure that 'iperf3 -V -s -D' is
> already running on the server machine.
> -#              FIXTHIS: add functionality to setup the server using a board file
> -#  - [Optional] server_params (BENCHMARK_IPERF3_SERVER_PARAMS):
> extra parameters for the server
> -#  - [Optional] client_params (BENCHMARK_IPERF3_SERVER_PARAMS):
> extra parameters for the client
> -
>  function test_build {
>      ./configure --host=$HOST --prefix=$(pwd)/build
>      make
> @@ -22,7 +13,7 @@ function test_deploy {
> 
>  function test_run {
>      # FIXTHIS: validate the server and client ips, and make sure they can
> communicate
> -
> +    # FIXTHIS: add functionality to setup the server using a board file
>      # Get the server ip
>      IPERF3_SERVER_IP=${BENCHMARK_IPERF3_SERVER_IP:-$SRV_IP}
>      if [ -z "$IPERF3_SERVER_IP" ]; then
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps

[Tim.Bird]

A couple of issues I saw while testing this:

Does the iperf3 server only support one connection at a time?
I tried running multiple tests at the same time, and one of them
got (from the test console log): 

#### doing fuego phase: run ########
Using server ip address: 10.0.1.61
Starting iperf3 server on localhost (SERVER IP: 10.0.1.61)
Starting iperf3 client on the target (CLIENT IP: 10.0.1.91)
{
	"start":	{
		"connected":	[],
		"version":	"iperf 3.1.3",
		"system_info":	"Linux m3ulcb 4.9.0-yocto-standard #1 SMP PREEMPT Wed Jul 5 15:05:42 PDT 2017 aarch64"
	},
	"intervals":	[],
	"end":	{
	},
	"error":	"error - unable to connect to server: Connection refused"
}

The second run, without any contention for the server, ran OK.

Also, most of my board fail the test due to very high CPU utilization.

What is the expected CPU utilization?

It would be nice to see the png even in the failure case, as there might
be useful data in it.

Overall, patch looks good, but test could use some refinement.

Thanks,
 -- Tim

> -----Original Message-----
> From: Daniel Sangorrin
> The -f/--format is not being used on the JSON output report.
> For that reason, I decided to use bps for now and open
> an issue upstream:
> https://github.com/esnet/iperf/issues/709
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/tests/Benchmark.iperf3/fuego_test.sh  | 2 +-
>  engine/tests/Benchmark.iperf3/parser.py      | 2 +-
>  engine/tests/Benchmark.iperf3/reference.json | 6 +++---
>  3 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> b/engine/tests/Benchmark.iperf3/fuego_test.sh
> index 09eac86..9f10743 100755
> --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> @@ -26,7 +26,7 @@ function test_run {
>      if [ "$ROUTE" = "local" ]; then
>          echo "Starting iperf3 server on localhost (SERVER IP:
> $IPERF3_SERVER_IP)"
>          killall iperf3 2> /dev/null || true
> -        iperf3 -V -s -D -f M $BENCHMARK_IPERF3_SERVER_PARAMS
> +        iperf3 -V -s -D $BENCHMARK_IPERF3_SERVER_PARAMS
>      fi
> 
>      echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> diff --git a/engine/tests/Benchmark.iperf3/parser.py
> b/engine/tests/Benchmark.iperf3/parser.py
> index 1097803..a91b88a 100755
> --- a/engine/tests/Benchmark.iperf3/parser.py
> +++ b/engine/tests/Benchmark.iperf3/parser.py
> @@ -51,7 +51,7 @@ with open(plib.TEST_LOG) as f:
>      plt.plot(time, bits_per_second)
>      plt.title('iperf3 client results')
>      plt.xlabel('time (s)')
> -    plt.ylabel('Mbps')
> +    plt.ylabel('Bits per second (bps)')
>      fig.savefig(os.environ['LOGDIR'] + '/iperf3.png')
> 
>  sys.exit(plib.process(measurements))
> diff --git a/engine/tests/Benchmark.iperf3/reference.json
> b/engine/tests/Benchmark.iperf3/reference.json
> index 930a94d..f10dc3a 100644
> --- a/engine/tests/Benchmark.iperf3/reference.json
> +++ b/engine/tests/Benchmark.iperf3/reference.json
> @@ -8,7 +8,7 @@
>                      "measurements":[
>                          {
>                              "name":"bits_per_second",
> -                            "unit":"Mbps"
> +                            "unit":"bps"
>                          },
>                          {
>                              "name":"retransmits",
> @@ -21,7 +21,7 @@
>                      "measurements":[
>                          {
>                              "name":"bits_per_second",
> -                            "unit":"Mbps"
> +                            "unit":"bps"
>                          }
>                      ]
>                  },
> @@ -30,7 +30,7 @@
>                      "measurements":[
>                          {
>                              "name":"bits_per_second",
> -                            "unit":"Mbps"
> +                            "unit":"bps"
>                          },
>                          {
>                              "name":"jitter_ms",
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps

[Daniel Sangorrin]

> -----Original Message-----
> From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> Sent: Thursday, March 08, 2018 8:37 AM
> To: daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps
> 
> A couple of issues I saw while testing this:
> 
> Does the iperf3 server only support one connection at a time?
> I tried running multiple tests at the same time, and one of them
> got (from the test console log):

iperf3 has the -P option for the client to run a test with multiple parallel connections.
However, it does not support multiple instances of iperf3's client running at the same time.
# You would need to run it twice on separate ports (-p port)
There is a wrapper script called bwctl that ensures that only one test is running at a time.
I will investigate how to use it and see if that solves the problem.
 
> #### doing fuego phase: run ########
> Using server ip address: 10.0.1.61
> Starting iperf3 server on localhost (SERVER IP: 10.0.1.61)
> Starting iperf3 client on the target (CLIENT IP: 10.0.1.91)
> {
> 	"start":	{
> 		"connected":	[],
> 		"version":	"iperf 3.1.3",
> 		"system_info":	"Linux m3ulcb 4.9.0-yocto-standard #1 SMP PREEMPT Wed Jul 5 15:05:42 PDT 2017 aarch64"
> 	},
> 	"intervals":	[],
> 	"end":	{
> 	},
> 	"error":	"error - unable to connect to server: Connection refused"
> }
> 
> The second run, without any contention for the server, ran OK.
> 
> Also, most of my board fail the test due to very high CPU utilization.
> 
> What is the expected CPU utilization?

It depends on the board I guess.

Beaglebone black: 68.3%
An intel i7 PC: 10.7%
Zynq702: 54.2%
Tegrak1 (jetson): 36.8%

You can control the amount of bandwidth with the -b option (e.g. -b 100M).

> It would be nice to see the png even in the failure case, as there might
> be useful data in it.

OK.

> Overall, patch looks good, but test could use some refinement.

Thanks. I just found another problem myself.
I was using the stdout JSON file directly, but sometimes other messages get
intermingled and the parser gives a "can't decode JSON" error. I will fix this
and send you another patch.

Thanks,
Daniel

> 
> Thanks,
>  -- Tim
> 
> > -----Original Message-----
> > From: Daniel Sangorrin
> > The -f/--format is not being used on the JSON output report.
> > For that reason, I decided to use bps for now and open
> > an issue upstream:
> > https://github.com/esnet/iperf/issues/709
> >
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > ---
> >  engine/tests/Benchmark.iperf3/fuego_test.sh  | 2 +-
> >  engine/tests/Benchmark.iperf3/parser.py      | 2 +-
> >  engine/tests/Benchmark.iperf3/reference.json | 6 +++---
> >  3 files changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > index 09eac86..9f10743 100755
> > --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > @@ -26,7 +26,7 @@ function test_run {
> >      if [ "$ROUTE" = "local" ]; then
> >          echo "Starting iperf3 server on localhost (SERVER IP:
> > $IPERF3_SERVER_IP)"
> >          killall iperf3 2> /dev/null || true
> > -        iperf3 -V -s -D -f M $BENCHMARK_IPERF3_SERVER_PARAMS
> > +        iperf3 -V -s -D $BENCHMARK_IPERF3_SERVER_PARAMS
> >      fi
> >
> >      echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> > diff --git a/engine/tests/Benchmark.iperf3/parser.py
> > b/engine/tests/Benchmark.iperf3/parser.py
> > index 1097803..a91b88a 100755
> > --- a/engine/tests/Benchmark.iperf3/parser.py
> > +++ b/engine/tests/Benchmark.iperf3/parser.py
> > @@ -51,7 +51,7 @@ with open(plib.TEST_LOG) as f:
> >      plt.plot(time, bits_per_second)
> >      plt.title('iperf3 client results')
> >      plt.xlabel('time (s)')
> > -    plt.ylabel('Mbps')
> > +    plt.ylabel('Bits per second (bps)')
> >      fig.savefig(os.environ['LOGDIR'] + '/iperf3.png')
> >
> >  sys.exit(plib.process(measurements))
> > diff --git a/engine/tests/Benchmark.iperf3/reference.json
> > b/engine/tests/Benchmark.iperf3/reference.json
> > index 930a94d..f10dc3a 100644
> > --- a/engine/tests/Benchmark.iperf3/reference.json
> > +++ b/engine/tests/Benchmark.iperf3/reference.json
> > @@ -8,7 +8,7 @@
> >                      "measurements":[
> >                          {
> >                              "name":"bits_per_second",
> > -                            "unit":"Mbps"
> > +                            "unit":"bps"
> >                          },
> >                          {
> >                              "name":"retransmits",
> > @@ -21,7 +21,7 @@
> >                      "measurements":[
> >                          {
> >                              "name":"bits_per_second",
> > -                            "unit":"Mbps"
> > +                            "unit":"bps"
> >                          }
> >                      ]
> >                  },
> > @@ -30,7 +30,7 @@
> >                      "measurements":[
> >                          {
> >                              "name":"bits_per_second",
> > -                            "unit":"Mbps"
> > +                            "unit":"bps"
> >                          },
> >                          {
> >                              "name":"jitter_ms",
> > --
> > 2.7.4
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps

[Daniel Sangorrin]

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Thursday, March 08, 2018 1:46 PM
> To: Tim.Bird@sony.com; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps
> 
> > -----Original Message-----
> > From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> > Sent: Thursday, March 08, 2018 8:37 AM
> > To: daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org
> > Subject: RE: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps
> >
> > A couple of issues I saw while testing this:
> >
> > Does the iperf3 server only support one connection at a time?
> > I tried running multiple tests at the same time, and one of them
> > got (from the test console log):
> 
> iperf3 has the -P option for the client to run a test with multiple parallel connections.
> However, it does not support multiple instances of iperf3's client running at the same time.
> # You would need to run it twice on separate ports (-p port)
> There is a wrapper script called bwctl that ensures that only one test is running at a time.
> I will investigate how to use it and see if that solves the problem.

I checked bwctl, but it was too much for our purposes. Instread I have added retry
logic to fuego_test.sh for running iperf3 on multiple boards at the same time.

> 
> > #### doing fuego phase: run ########
> > Using server ip address: 10.0.1.61
> > Starting iperf3 server on localhost (SERVER IP: 10.0.1.61)
> > Starting iperf3 client on the target (CLIENT IP: 10.0.1.91)
> > {
> > 	"start":	{
> > 		"connected":	[],
> > 		"version":	"iperf 3.1.3",
> > 		"system_info":	"Linux m3ulcb 4.9.0-yocto-standard #1 SMP PREEMPT Wed Jul 5 15:05:42 PDT 2017 aarch64"
> > 	},
> > 	"intervals":	[],
> > 	"end":	{
> > 	},
> > 	"error":	"error - unable to connect to server: Connection refused"
> > }
> >
> > The second run, without any contention for the server, ran OK.
> >
> > Also, most of my board fail the test due to very high CPU utilization.
> >
> > What is the expected CPU utilization?
> 
> It depends on the board I guess.
> 
> Beaglebone black: 68.3%
> An intel i7 PC: 10.7%
> Zynq702: 54.2%
> Tegrak1 (jetson): 36.8%
> 
> You can control the amount of bandwidth with the -b option (e.g. -b 100M).

I have changed the default specs to be a bit more lax about the CPU utilization
and allow 100%.

> 
> > It would be nice to see the png even in the failure case, as there might
> > be useful data in it.
> 
> OK.

I added the links to the fail case as well.

> 
> > Overall, patch looks good, but test could use some refinement.
> 
> Thanks. I just found another problem myself.
> I was using the stdout JSON file directly, but sometimes other messages get
> intermingled and the parser gives a "can't decode JSON" error. I will fix this
> and send you another patch.

This should be fixed in my latest patches. They are on my git repository
as well for your convenience.

Daniel
 
> >
> > Thanks,
> >  -- Tim
> >
> > > -----Original Message-----
> > > From: Daniel Sangorrin
> > > The -f/--format is not being used on the JSON output report.
> > > For that reason, I decided to use bps for now and open
> > > an issue upstream:
> > > https://github.com/esnet/iperf/issues/709
> > >
> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > > ---
> > >  engine/tests/Benchmark.iperf3/fuego_test.sh  | 2 +-
> > >  engine/tests/Benchmark.iperf3/parser.py      | 2 +-
> > >  engine/tests/Benchmark.iperf3/reference.json | 6 +++---
> > >  3 files changed, 5 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > > b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > > index 09eac86..9f10743 100755
> > > --- a/engine/tests/Benchmark.iperf3/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.iperf3/fuego_test.sh
> > > @@ -26,7 +26,7 @@ function test_run {
> > >      if [ "$ROUTE" = "local" ]; then
> > >          echo "Starting iperf3 server on localhost (SERVER IP:
> > > $IPERF3_SERVER_IP)"
> > >          killall iperf3 2> /dev/null || true
> > > -        iperf3 -V -s -D -f M $BENCHMARK_IPERF3_SERVER_PARAMS
> > > +        iperf3 -V -s -D $BENCHMARK_IPERF3_SERVER_PARAMS
> > >      fi
> > >
> > >      echo "Starting iperf3 client on the target (CLIENT IP: $IPADDR)"
> > > diff --git a/engine/tests/Benchmark.iperf3/parser.py
> > > b/engine/tests/Benchmark.iperf3/parser.py
> > > index 1097803..a91b88a 100755
> > > --- a/engine/tests/Benchmark.iperf3/parser.py
> > > +++ b/engine/tests/Benchmark.iperf3/parser.py
> > > @@ -51,7 +51,7 @@ with open(plib.TEST_LOG) as f:
> > >      plt.plot(time, bits_per_second)
> > >      plt.title('iperf3 client results')
> > >      plt.xlabel('time (s)')
> > > -    plt.ylabel('Mbps')
> > > +    plt.ylabel('Bits per second (bps)')
> > >      fig.savefig(os.environ['LOGDIR'] + '/iperf3.png')
> > >
> > >  sys.exit(plib.process(measurements))
> > > diff --git a/engine/tests/Benchmark.iperf3/reference.json
> > > b/engine/tests/Benchmark.iperf3/reference.json
> > > index 930a94d..f10dc3a 100644
> > > --- a/engine/tests/Benchmark.iperf3/reference.json
> > > +++ b/engine/tests/Benchmark.iperf3/reference.json
> > > @@ -8,7 +8,7 @@
> > >                      "measurements":[
> > >                          {
> > >                              "name":"bits_per_second",
> > > -                            "unit":"Mbps"
> > > +                            "unit":"bps"
> > >                          },
> > >                          {
> > >                              "name":"retransmits",
> > > @@ -21,7 +21,7 @@
> > >                      "measurements":[
> > >                          {
> > >                              "name":"bits_per_second",
> > > -                            "unit":"Mbps"
> > > +                            "unit":"bps"
> > >                          }
> > >                      ]
> > >                  },
> > > @@ -30,7 +30,7 @@
> > >                      "measurements":[
> > >                          {
> > >                              "name":"bits_per_second",
> > > -                            "unit":"Mbps"
> > > +                            "unit":"bps"
> > >                          },
> > >                          {
> > >                              "name":"jitter_ms",
> > > --
> > > 2.7.4
> > >
> > >
> > > _______________________________________________
> > > Fuego mailing list
> > > Fuego@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps

[Tim.Bird]

> -----Original Message-----
> From: Daniel Sangorrin
> 
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Thursday, March 08, 2018 1:46 PM
> > To: Tim.Bird@sony.com; fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps
> >
> > > -----Original Message-----
> > > From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> > > Sent: Thursday, March 08, 2018 8:37 AM
> > > To: daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org
> > > Subject: RE: [Fuego] [PATCH 3/3] iperf3: use bps instead of Mbps
> > >
> > > A couple of issues I saw while testing this:
> > >
> > > Does the iperf3 server only support one connection at a time?
> > > I tried running multiple tests at the same time, and one of them
> > > got (from the test console log):
> >
> > iperf3 has the -P option for the client to run a test with multiple parallel
> connections.
> > However, it does not support multiple instances of iperf3's client running at
> the same time.
> > # You would need to run it twice on separate ports (-p port)
> > There is a wrapper script called bwctl that ensures that only one test is
> running at a time.
> > I will investigate how to use it and see if that solves the problem.
> 
> I checked bwctl, but it was too much for our purposes. Instread I have added
> retry
> logic to fuego_test.sh for running iperf3 on multiple boards at the same time.
> 
> >
> > > #### doing fuego phase: run ########
> > > Using server ip address: 10.0.1.61
> > > Starting iperf3 server on localhost (SERVER IP: 10.0.1.61)
> > > Starting iperf3 client on the target (CLIENT IP: 10.0.1.91)
> > > {
> > > 	"start":	{
> > > 		"connected":	[],
> > > 		"version":	"iperf 3.1.3",
> > > 		"system_info":	"Linux m3ulcb 4.9.0-yocto-standard #1 SMP
> PREEMPT Wed Jul 5 15:05:42 PDT 2017 aarch64"
> > > 	},
> > > 	"intervals":	[],
> > > 	"end":	{
> > > 	},
> > > 	"error":	"error - unable to connect to server: Connection
> refused"
> > > }
> > >
> > > The second run, without any contention for the server, ran OK.
> > >
> > > Also, most of my board fail the test due to very high CPU utilization.
> > >
> > > What is the expected CPU utilization?
> >
> > It depends on the board I guess.
> >
> > Beaglebone black: 68.3%
> > An intel i7 PC: 10.7%
> > Zynq702: 54.2%
> > Tegrak1 (jetson): 36.8%
> >
> > You can control the amount of bandwidth with the -b option (e.g. -b 100M).
> 
> I have changed the default specs to be a bit more lax about the CPU
> utilization
> and allow 100%.
> 
> >
> > > It would be nice to see the png even in the failure case, as there might
> > > be useful data in it.
> >
> > OK.
> 
> I added the links to the fail case as well.
> 
> >
> > > Overall, patch looks good, but test could use some refinement.
> >
> > Thanks. I just found another problem myself.
> > I was using the stdout JSON file directly, but sometimes other messages
> get
> > intermingled and the parser gives a "can't decode JSON" error. I will fix this
> > and send you another patch.
> 
> This should be fixed in my latest patches. They are on my git repository
> as well for your convenience.
> 
> Daniel

Thanks.  Most of these were applied.
 -- Tim