[Fuego] [PATCH 01/15] migratetest: add RT group warning

[Fuego] [PATCH 01/15] migratetest: add RT group warning

[Qiu Tingting]

Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
By following the warning, right settings of boards could be setted before test.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.migratetest/fuego_test.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/engine/tests/Benchmark.migratetest/fuego_test.sh b/engine/tests/Benchmark.migratetest/fuego_test.sh
index bc3627a..43e3bea 100755
--- a/engine/tests/Benchmark.migratetest/fuego_test.sh
+++ b/engine/tests/Benchmark.migratetest/fuego_test.sh
@@ -5,6 +5,12 @@ TEST_COMMAND="rt-migrate-test"
 
 function test_pre_check {
     assert_define BENCHMARK_RT_MIGRATE_TEST_PARAMS
+    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
+        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel. Please check the RT"
+        echo "settings as following if this test failed with 'Unable to change scheduling policy'."
+        echo "- If the user does not need the RT groups functionality, disable CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w kernel.sched_rt_runtime_us=-1'"
+        echo "- If the user does want to use RT groups functionality, you may want to assign the ssh daemon service a realtime budget by specifyihg the cpu.rt_runtime_us attribute"
+    fi
 }
 
 function test_build {
-- 
2.7.4

[Fuego] [PATCH 02/15] migratetest: change testName and branch of spec file

[Qiu Tingting]

In file 'spec.json',
1. the value of testName is not same as the job name, which makes the PER_JOB_BUILD doesn't work.
Change the testName with job name.
2. The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which makes test_build fail.
Use exist upstream branch.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.migratetest/fuego_test.sh | 4 ++--
 engine/tests/Benchmark.migratetest/spec.json     | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/engine/tests/Benchmark.migratetest/fuego_test.sh b/engine/tests/Benchmark.migratetest/fuego_test.sh
index 43e3bea..eeaa4f6 100755
--- a/engine/tests/Benchmark.migratetest/fuego_test.sh
+++ b/engine/tests/Benchmark.migratetest/fuego_test.sh
@@ -4,7 +4,7 @@ NEED_ROOT=1
 TEST_COMMAND="rt-migrate-test"
 
 function test_pre_check {
-    assert_define BENCHMARK_RT_MIGRATE_TEST_PARAMS
+    assert_define BENCHMARK_MIGRATETEST_PARAMS
     if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
         echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel. Please check the RT"
         echo "settings as following if this test failed with 'Unable to change scheduling policy'."
@@ -23,5 +23,5 @@ function test_deploy {
 }
 
 function test_run {
-    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./${TEST_COMMAND} $BENCHMARK_RT_MIGRATE_TEST_PARAMS"
+    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./${TEST_COMMAND} $BENCHMARK_MIGRATETEST_PARAMS"
 }
diff --git a/engine/tests/Benchmark.migratetest/spec.json b/engine/tests/Benchmark.migratetest/spec.json
index 6d42f36..8270c95 100644
--- a/engine/tests/Benchmark.migratetest/spec.json
+++ b/engine/tests/Benchmark.migratetest/spec.json
@@ -1,5 +1,5 @@
 {
-    "testName": "Benchmark.rt_migrate_test",
+    "testName": "Benchmark.migratetest",
     "specs": {
         "default": {
             "PARAMS": "-l 10"
@@ -7,7 +7,7 @@
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
-            "gitref": "unstable/devel/v1.1.1",
+            "gitref": "unstable/devel/latest",
             "PARAMS": " "
         }
     }
-- 
2.7.4

[Fuego] [PATCH 03/15] migratetest: add min_interval graph

[Qiu Tingting]

Test case 'min_interval' is defined and value is checked, but there is no graph.
Add graph of 'min_interval'.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.migratetest/chart_config.json | 1 +
 1 file changed, 1 insertion(+)

diff --git a/engine/tests/Benchmark.migratetest/chart_config.json b/engine/tests/Benchmark.migratetest/chart_config.json
index 3cbaef2..0b7c587 100644
--- a/engine/tests/Benchmark.migratetest/chart_config.json
+++ b/engine/tests/Benchmark.migratetest/chart_config.json
@@ -1,5 +1,6 @@
 {
 	"chart_type": "measure_plot",
 	"measures": ["default.intervals.max_interval",
+        "default.intervals.min_interval",
         "default.intervals.avg_interval"]
 }
-- 
2.7.4

[Fuego] [PATCH 04/15] ptsematest: add min_latency graph

[Qiu Tingting]

Test case 'min_latency' is defined and value is checked, but there is no graph.
Add graph of 'min_latency'.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.ptsematest/chart_config.json | 1 +
 1 file changed, 1 insertion(+)

diff --git a/engine/tests/Benchmark.ptsematest/chart_config.json b/engine/tests/Benchmark.ptsematest/chart_config.json
index cdaf6a2..aa73167 100644
--- a/engine/tests/Benchmark.ptsematest/chart_config.json
+++ b/engine/tests/Benchmark.ptsematest/chart_config.json
@@ -1,5 +1,6 @@
 {
 	"chart_type": "measure_plot",
 	"measures": ["default.latencies.max_latency",
+        "default.latencies.min_latency",
         "default.latencies.avg_latency"]
 }
-- 
2.7.4

[Fuego] [PATCH 05/15] ptsematest: fix no nproc cmd on board

[Qiu Tingting]

If there is no nproc cmd on board, can not get target_cpu_number.
Get target_cpu_number from /proc/cpuinfo file.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.ptsematest/fuego_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh b/engine/tests/Benchmark.ptsematest/fuego_test.sh
index fc1a385..5f3fb44 100755
--- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
@@ -19,7 +19,7 @@ function test_run {
     # ptsematest does not support a option for printing a summary only on exit.
     # So, We get some lines at the end of the command's output.
     # The number for getting the lines depends on the cpu number of target machine.
-    target_cpu_number=$(cmd "nproc")
+    target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
     getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
     report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest $BENCHMARK_PTSEMATEST_PARAMS | tail -$getting_line_number"
 }
-- 
2.7.4

[Fuego] [PATCH 06/15] ptsematest: change original upstream branch

[Qiu Tingting]

The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which makes test_build fail.
Use exist upstream branch.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.ptsematest/spec.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Benchmark.ptsematest/spec.json b/engine/tests/Benchmark.ptsematest/spec.json
index 8fd2db9..aa5d6ce 100644
--- a/engine/tests/Benchmark.ptsematest/spec.json
+++ b/engine/tests/Benchmark.ptsematest/spec.json
@@ -7,7 +7,7 @@
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
-            "gitref": "unstable/devel/v1.1.1",
+            "gitref": "unstable/devel/latest",
             "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
         }
     }
-- 
2.7.4

[Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of PARAMS

[Qiu Tingting]

Cmd ptsematest has -t num parameter.
When cpu num is not setted, default value is max num of cpu.
Maybe some of cpu is not working, so we should set the available cpu num.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.ptsematest/fuego_test.sh | 2 +-
 engine/tests/Benchmark.ptsematest/spec.json     | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh b/engine/tests/Benchmark.ptsematest/fuego_test.sh
index 5f3fb44..5f0633e 100755
--- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
@@ -21,5 +21,5 @@ function test_run {
     # The number for getting the lines depends on the cpu number of target machine.
     target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
     getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
-    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest $BENCHMARK_PTSEMATEST_PARAMS | tail -$getting_line_number"
+    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest $BENCHMARK_PTSEMATEST_PARAMS$target_cpu_number | tail -$getting_line_number"
 }
diff --git a/engine/tests/Benchmark.ptsematest/spec.json b/engine/tests/Benchmark.ptsematest/spec.json
index aa5d6ce..16bc9ee 100644
--- a/engine/tests/Benchmark.ptsematest/spec.json
+++ b/engine/tests/Benchmark.ptsematest/spec.json
@@ -2,13 +2,13 @@
     "testName": "Benchmark.ptsematest",
     "specs": {
         "default": {
-            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
+            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
         },
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
             "gitref": "unstable/devel/latest",
-            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
+            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
         }
     }
 }
-- 
2.7.4

[Fuego] [PATCH 08/15] ptsematest: add RT group warning

[Qiu Tingting]

Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
By following the warning, right settings of boards could be setted before test.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.ptsematest/fuego_test.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh b/engine/tests/Benchmark.ptsematest/fuego_test.sh
index 5f0633e..77e041a 100755
--- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
@@ -4,6 +4,12 @@ NEED_ROOT=1
 
 function test_pre_check {
     assert_define BENCHMARK_PTSEMATEST_PARAMS
+    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
+        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel. Please check the RT"
+        echo "settings as following if this test failed with 'Unable to change scheduling policy'."
+        echo "- If the user does not need the RT groups functionality, disable CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w kernel.sched_rt_runtime_us=-1'"
+        echo "- If the user does want to use RT groups functionality, you may want to assign the ssh daemon service a realtime budget by specifyihg the cpu.rt_runtime_us attribute"
+    fi
 }
 
 function test_build {
-- 
2.7.4

[Fuego] [PATCH 09/15] svsematest: add min_latency graph

[Qiu Tingting]

Test case 'min_latency' is defined and value is checked, but there is no graph.
Add graph of 'min_latency'.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.svsematest/chart_config.json | 1 +
 1 file changed, 1 insertion(+)

diff --git a/engine/tests/Benchmark.svsematest/chart_config.json b/engine/tests/Benchmark.svsematest/chart_config.json
index cdaf6a2..aa73167 100644
--- a/engine/tests/Benchmark.svsematest/chart_config.json
+++ b/engine/tests/Benchmark.svsematest/chart_config.json
@@ -1,5 +1,6 @@
 {
 	"chart_type": "measure_plot",
 	"measures": ["default.latencies.max_latency",
+        "default.latencies.min_latency",
         "default.latencies.avg_latency"]
 }
-- 
2.7.4

[Fuego] [PATCH 10/15] svsematest: change original upstream branch

[Qiu Tingting]

The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which makes test_build fail.
Use exist upstream branch.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.svsematest/spec.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Benchmark.svsematest/spec.json b/engine/tests/Benchmark.svsematest/spec.json
index 1a9a767..00e77c1 100644
--- a/engine/tests/Benchmark.svsematest/spec.json
+++ b/engine/tests/Benchmark.svsematest/spec.json
@@ -7,7 +7,7 @@
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
-            "gitref": "unstable/devel/v1.1.1",
+            "gitref": "unstable/devel/latest",
             "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
         }
     }
-- 
2.7.4

[Fuego] [PATCH 11/15] svsematest: fix no nproc cmd on board

[Qiu Tingting]

If there is no nproc cmd on board, can not get target_cpu_number.
Get target_cpu_number from /proc/cpuinfo file.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh b/engine/tests/Benchmark.svsematest/fuego_test.sh
index 815cec2..f2ebd18 100755
--- a/engine/tests/Benchmark.svsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
@@ -19,7 +19,7 @@ function test_run {
     # svsematest does not support a option for printing a summary only on exit.
     # So, We get some lines at the end of the command's output.
     # The number for getting the lines depends on the cpu number of target machine.
-    target_cpu_number=$(cmd "nproc")
+    target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
     getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
     report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
 }
-- 
2.7.4

[Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Qiu Tingting]

Cmd svsematest has -t num parameter.
When cpu num is not setted, default value is max num of cpu.
Maybe some of cpu is not working, so we should set the available cpu num.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
 engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh b/engine/tests/Benchmark.svsematest/fuego_test.sh
index f2ebd18..2988cd4 100755
--- a/engine/tests/Benchmark.svsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
@@ -21,5 +21,5 @@ function test_run {
     # The number for getting the lines depends on the cpu number of target machine.
     target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
     getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
-    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
+    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail -$getting_line_number"
 }
diff --git a/engine/tests/Benchmark.svsematest/spec.json b/engine/tests/Benchmark.svsematest/spec.json
index 00e77c1..e814600 100644
--- a/engine/tests/Benchmark.svsematest/spec.json
+++ b/engine/tests/Benchmark.svsematest/spec.json
@@ -2,13 +2,13 @@
     "testName": "Benchmark.svsematest",
     "specs": {
         "default": {
-            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
+            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
         },
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
             "gitref": "unstable/devel/latest",
-            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
+            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
         }
     }
 }
-- 
2.7.4

[Fuego] [PATCH 13/15] svsematest: add RT group warning

[Qiu Tingting]

Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
By following the warning, right settings of boards could be setted before test.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.svsematest/fuego_test.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh b/engine/tests/Benchmark.svsematest/fuego_test.sh
index 2988cd4..6559ea6 100755
--- a/engine/tests/Benchmark.svsematest/fuego_test.sh
+++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
@@ -4,6 +4,12 @@ NEED_ROOT=1
 
 function test_pre_check {
     assert_define BENCHMARK_SVSEMATEST_PARAMS
+    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
+        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel. Please check the RT"
+        echo "settings as following if this test failed with 'Unable to change scheduling policy'."
+        echo "- If the user does not need the RT groups functionality, disable CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w kernel.sched_rt_runtime_us=-1'"
+        echo "- If the user does want to use RT groups functionality, you may want to assign the ssh daemon service a realtime budget by specifyihg the cpu.rt_runtime_us attribute"
+    fi
 }
 
 function test_build {
-- 
2.7.4

[Fuego] [PATCH 14/15] hackbench: change original upstream branch

[Qiu Tingting]

The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which makes test_build fail.
Use exist upstream branch.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Benchmark.hackbench/spec.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Benchmark.hackbench/spec.json b/engine/tests/Benchmark.hackbench/spec.json
index ea90c43..78ddb07 100644
--- a/engine/tests/Benchmark.hackbench/spec.json
+++ b/engine/tests/Benchmark.hackbench/spec.json
@@ -16,7 +16,7 @@
         "latest": {
             "PER_JOB_BUILD": "true",
             "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
-            "gitref": "unstable/devel/v1.1.1",
+            "gitref": "unstable/devel/latest",
             "PARAMS": " "
         }
     }
-- 
2.7.4

[Fuego] [PATCH 15/15] pi_tests: add RT group warning

[Qiu Tingting]

Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
By following the warning, right settings of boards could be setted before test.

Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
---
 engine/tests/Functional.pi_tests/fuego_test.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh b/engine/tests/Functional.pi_tests/fuego_test.sh
index 8df76d3..bf94a63 100755
--- a/engine/tests/Functional.pi_tests/fuego_test.sh
+++ b/engine/tests/Functional.pi_tests/fuego_test.sh
@@ -4,6 +4,12 @@ NEED_ROOT=1
 
 function test_pre_check {
     assert_define FUNCTIONAL_PI_TESTS_PARAMS
+    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
+        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel. Please check the RT"
+        echo "settings as following if this test failed with 'Unable to change scheduling policy'."
+        echo "- If the user does not need the RT groups functionality, disable CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w kernel.sched_rt_runtime_us=-1'"
+        echo "- If the user does want to use RT groups functionality, you may want to assign the ssh daemon service a realtime budget by specifyihg the cpu.rt_runtime_us attribute"
+    fi
 }
 
 function test_build {
-- 
2.7.4

Re: [Fuego] [PATCH 01/15] migratetest: add RT group warning

[Tim.Bird]

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 01/15] migratetest: add RT group warning
> 
> Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
> By following the warning, right settings of boards could be setted before
> test.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.migratetest/fuego_test.sh | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/engine/tests/Benchmark.migratetest/fuego_test.sh
> b/engine/tests/Benchmark.migratetest/fuego_test.sh
> index bc3627a..43e3bea 100755
> --- a/engine/tests/Benchmark.migratetest/fuego_test.sh
> +++ b/engine/tests/Benchmark.migratetest/fuego_test.sh
> @@ -5,6 +5,12 @@ TEST_COMMAND="rt-migrate-test"
> 
>  function test_pre_check {
>      assert_define BENCHMARK_RT_MIGRATE_TEST_PARAMS
> +    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
> +        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel.
> Please check the RT"
> +        echo "settings as following if this test failed with 'Unable to change
> scheduling policy'."
> +        echo "- If the user does not need the RT groups functionality, disable
> CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w
> kernel.sched_rt_runtime_us=-1'"
> +        echo "- If the user does want to use RT groups functionality, you may
> want to assign the ssh daemon service a realtime budget by specifyihg the
> cpu.rt_runtime_us attribute"
> +    fi
>  }
> 
>  function test_build {
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 02/15] migratetest: change testName and branch of spec file

[Tim.Bird]

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 02/15] migratetest: change testName and branch of
> spec file
> 
> In file 'spec.json',
> 1. the value of testName is not same as the job name, which makes the
> PER_JOB_BUILD doesn't work.
> Change the testName with job name.
> 2. The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which
> makes test_build fail.
> Use exist upstream branch.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.migratetest/fuego_test.sh | 4 ++--
>  engine/tests/Benchmark.migratetest/spec.json     | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.migratetest/fuego_test.sh
> b/engine/tests/Benchmark.migratetest/fuego_test.sh
> index 43e3bea..eeaa4f6 100755
> --- a/engine/tests/Benchmark.migratetest/fuego_test.sh
> +++ b/engine/tests/Benchmark.migratetest/fuego_test.sh
> @@ -4,7 +4,7 @@ NEED_ROOT=1
>  TEST_COMMAND="rt-migrate-test"
> 
>  function test_pre_check {
> -    assert_define BENCHMARK_RT_MIGRATE_TEST_PARAMS
> +    assert_define BENCHMARK_MIGRATETEST_PARAMS
>      if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
>          echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel.
> Please check the RT"
>          echo "settings as following if this test failed with 'Unable to change
> scheduling policy'."
> @@ -23,5 +23,5 @@ function test_deploy {
>  }
> 
>  function test_run {
> -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./${TEST_COMMAND}
> $BENCHMARK_RT_MIGRATE_TEST_PARAMS"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./${TEST_COMMAND}
> $BENCHMARK_MIGRATETEST_PARAMS"
>  }
> diff --git a/engine/tests/Benchmark.migratetest/spec.json
> b/engine/tests/Benchmark.migratetest/spec.json
> index 6d42f36..8270c95 100644
> --- a/engine/tests/Benchmark.migratetest/spec.json
> +++ b/engine/tests/Benchmark.migratetest/spec.json
> @@ -1,5 +1,5 @@
>  {
> -    "testName": "Benchmark.rt_migrate_test",
> +    "testName": "Benchmark.migratetest",
>      "specs": {
>          "default": {
>              "PARAMS": "-l 10"
> @@ -7,7 +7,7 @@
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> -            "gitref": "unstable/devel/v1.1.1",
> +            "gitref": "unstable/devel/latest",
>              "PARAMS": " "
>          }
>      }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 03/15] migratetest: add min_interval graph

[Tim.Bird]

Applied.  Thanks.
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 03/15] migratetest: add min_interval graph
> 
> Test case 'min_interval' is defined and value is checked, but there is no
> graph.
> Add graph of 'min_interval'.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.migratetest/chart_config.json | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/engine/tests/Benchmark.migratetest/chart_config.json
> b/engine/tests/Benchmark.migratetest/chart_config.json
> index 3cbaef2..0b7c587 100644
> --- a/engine/tests/Benchmark.migratetest/chart_config.json
> +++ b/engine/tests/Benchmark.migratetest/chart_config.json
> @@ -1,5 +1,6 @@
>  {
>  	"chart_type": "measure_plot",
>  	"measures": ["default.intervals.max_interval",
> +        "default.intervals.min_interval",
>          "default.intervals.avg_interval"]
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 04/15] ptsematest: add min_latency graph

[Tim.Bird]

Applied.  Thanks.
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 04/15] ptsematest: add min_latency graph
> 
> Test case 'min_latency' is defined and value is checked, but there is no graph.
> Add graph of 'min_latency'.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.ptsematest/chart_config.json | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/engine/tests/Benchmark.ptsematest/chart_config.json
> b/engine/tests/Benchmark.ptsematest/chart_config.json
> index cdaf6a2..aa73167 100644
> --- a/engine/tests/Benchmark.ptsematest/chart_config.json
> +++ b/engine/tests/Benchmark.ptsematest/chart_config.json
> @@ -1,5 +1,6 @@
>  {
>  	"chart_type": "measure_plot",
>  	"measures": ["default.latencies.max_latency",
> +        "default.latencies.min_latency",
>          "default.latencies.avg_latency"]
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 05/15] ptsematest: fix no nproc cmd on board

[Tim.Bird]

It is nice to see a reduction in the dependencies of the test on
target features.  This improvement in the generality of the patch
is much appreciated.

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 05/15] ptsematest: fix no nproc cmd on board
> 
> If there is no nproc cmd on board, can not get target_cpu_number.
> Get target_cpu_number from /proc/cpuinfo file.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.ptsematest/fuego_test.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> index fc1a385..5f3fb44 100755
> --- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> @@ -19,7 +19,7 @@ function test_run {
>      # ptsematest does not support a option for printing a summary only on
> exit.
>      # So, We get some lines at the end of the command's output.
>      # The number for getting the lines depends on the cpu number of target
> machine.
> -    target_cpu_number=$(cmd "nproc")
> +    target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
>      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
>      report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest
> $BENCHMARK_PTSEMATEST_PARAMS | tail -$getting_line_number"
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 06/15] ptsematest: change original upstream branch

[Tim.Bird]

Sounds good.  Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 06/15] ptsematest: change original upstream branch
> 
> The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which
> makes test_build fail.
> Use exist upstream branch.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.ptsematest/spec.json | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/engine/tests/Benchmark.ptsematest/spec.json
> b/engine/tests/Benchmark.ptsematest/spec.json
> index 8fd2db9..aa5d6ce 100644
> --- a/engine/tests/Benchmark.ptsematest/spec.json
> +++ b/engine/tests/Benchmark.ptsematest/spec.json
> @@ -7,7 +7,7 @@
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> -            "gitref": "unstable/devel/v1.1.1",
> +            "gitref": "unstable/devel/latest",
>              "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
>          }
>      }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of PARAMS

[Tim.Bird]

> -----Original Message-----
> From: Qiu Tingting on Wednesday, November 14, 2018 8:26 PM
> 
> Cmd ptsematest has -t num parameter.
> When cpu num is not setted, default value is max num of cpu.
> Maybe some of cpu is not working, so we should set the available cpu num.

Can you explain this more?  This change adjusts
the code to have ptsematest use the number of cpus reported
by /proc/cpuinfo.   This seems like it would be the maximum number of
cpus on the system. 

How would ptsematest use a different number of cpus?

> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.ptsematest/fuego_test.sh | 2 +-
>  engine/tests/Benchmark.ptsematest/spec.json     | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> index 5f3fb44..5f0633e 100755
> --- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> @@ -21,5 +21,5 @@ function test_run {
>      # The number for getting the lines depends on the cpu number of target
> machine.
>      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
>      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest
> $BENCHMARK_PTSEMATEST_PARAMS | tail -$getting_line_number"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest
> $BENCHMARK_PTSEMATEST_PARAMS$target_cpu_number | tail -
> $getting_line_number"
>  }
> diff --git a/engine/tests/Benchmark.ptsematest/spec.json
> b/engine/tests/Benchmark.ptsematest/spec.json
> index aa5d6ce..16bc9ee 100644
> --- a/engine/tests/Benchmark.ptsematest/spec.json
> +++ b/engine/tests/Benchmark.ptsematest/spec.json
> @@ -2,13 +2,13 @@
>      "testName": "Benchmark.ptsematest",
>      "specs": {
>          "default": {
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
>          },
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
>              "gitref": "unstable/devel/latest",
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
>          }
>      }
>  }
> --
> 2.7.4

Not applied yet.  Please explain more what's going on.
 -- Tim

Re: [Fuego] [PATCH 08/15] ptsematest: add RT group warning

[Tim.Bird]

Applied.  Thanks,
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 08/15] ptsematest: add RT group warning
> 
> Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
> By following the warning, right settings of boards could be setted before
> test.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.ptsematest/fuego_test.sh | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> index 5f0633e..77e041a 100755
> --- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> @@ -4,6 +4,12 @@ NEED_ROOT=1
> 
>  function test_pre_check {
>      assert_define BENCHMARK_PTSEMATEST_PARAMS
> +    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
> +        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel.
> Please check the RT"
> +        echo "settings as following if this test failed with 'Unable to change
> scheduling policy'."
> +        echo "- If the user does not need the RT groups functionality, disable
> CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w
> kernel.sched_rt_runtime_us=-1'"
> +        echo "- If the user does want to use RT groups functionality, you may
> want to assign the ssh daemon service a realtime budget by specifyihg the
> cpu.rt_runtime_us attribute"
> +    fi
>  }
> 
>  function test_build {
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 09/15] svsematest: add min_latency graph

[Tim.Bird]

Applied.  Thanks.
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 09/15] svsematest: add min_latency graph
> 
> Test case 'min_latency' is defined and value is checked, but there is no graph.
> Add graph of 'min_latency'.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/chart_config.json | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/engine/tests/Benchmark.svsematest/chart_config.json
> b/engine/tests/Benchmark.svsematest/chart_config.json
> index cdaf6a2..aa73167 100644
> --- a/engine/tests/Benchmark.svsematest/chart_config.json
> +++ b/engine/tests/Benchmark.svsematest/chart_config.json
> @@ -1,5 +1,6 @@
>  {
>  	"chart_type": "measure_plot",
>  	"measures": ["default.latencies.max_latency",
> +        "default.latencies.min_latency",
>          "default.latencies.avg_latency"]
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 10/15] svsematest: change original upstream branch

[Tim.Bird]

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 10/15] svsematest: change original upstream branch
> 
> The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which
> makes test_build fail.
> Use exist upstream branch.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/spec.json | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/engine/tests/Benchmark.svsematest/spec.json
> b/engine/tests/Benchmark.svsematest/spec.json
> index 1a9a767..00e77c1 100644
> --- a/engine/tests/Benchmark.svsematest/spec.json
> +++ b/engine/tests/Benchmark.svsematest/spec.json
> @@ -7,7 +7,7 @@
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> -            "gitref": "unstable/devel/v1.1.1",
> +            "gitref": "unstable/devel/latest",
>              "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
>          }
>      }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 11/15] svsematest: fix no nproc cmd on board

[Tim.Bird]

Applied.  Thanks,
 -- Tim

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 11/15] svsematest: fix no nproc cmd on board
> 
> If there is no nproc cmd on board, can not get target_cpu_number.
> Get target_cpu_number from /proc/cpuinfo file.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> b/engine/tests/Benchmark.svsematest/fuego_test.sh
> index 815cec2..f2ebd18 100755
> --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> @@ -19,7 +19,7 @@ function test_run {
>      # svsematest does not support a option for printing a summary only on
> exit.
>      # So, We get some lines at the end of the command's output.
>      # The number for getting the lines depends on the cpu number of target
> machine.
> -    target_cpu_number=$(cmd "nproc")
> +    target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
>      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
>      report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Tim.Bird]

Same question about this one as ptsematest.

Please explain better.  I don't understand how taking the number of
processors from /proc/cpuinfo is going to omit processors that
are not working.

I guess I don't understand the different algorithms that are being
used by the RT test programs (ptsematest and svsematest) and
the /proc/cpuinfo scan, that would yield a different numbers.

What is the behavior that is faulty, if you don't use the -t parameter
to these tests?
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of
> PARAMS
> 
> Cmd svsematest has -t num parameter.
> When cpu num is not setted, default value is max num of cpu.
> Maybe some of cpu is not working, so we should set the available cpu num.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
>  engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> b/engine/tests/Benchmark.svsematest/fuego_test.sh
> index f2ebd18..2988cd4 100755
> --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> @@ -21,5 +21,5 @@ function test_run {
>      # The number for getting the lines depends on the cpu number of target
> machine.
>      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
>      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail -
> $getting_line_number"
>  }
> diff --git a/engine/tests/Benchmark.svsematest/spec.json
> b/engine/tests/Benchmark.svsematest/spec.json
> index 00e77c1..e814600 100644
> --- a/engine/tests/Benchmark.svsematest/spec.json
> +++ b/engine/tests/Benchmark.svsematest/spec.json
> @@ -2,13 +2,13 @@
>      "testName": "Benchmark.svsematest",
>      "specs": {
>          "default": {
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
>          },
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
>              "gitref": "unstable/devel/latest",
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
>          }
>      }
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 13/15] svsematest: add RT group warning

[Tim.Bird]

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 13/15] svsematest: add RT group warning
> 
> Add a warning if CONFIG_RT_GROUP_SCHED enabled in kernel.
> By following the warning, right settings of boards could be setted before
> test.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/fuego_test.sh | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> b/engine/tests/Benchmark.svsematest/fuego_test.sh
> index 2988cd4..6559ea6 100755
> --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> @@ -4,6 +4,12 @@ NEED_ROOT=1
> 
>  function test_pre_check {
>      assert_define BENCHMARK_SVSEMATEST_PARAMS
> +    if check_kconfig "CONFIG_RT_GROUP_SCHED=y"; then
> +        echo "WARNING: CONFIG_RT_GROUP_SCHED enabled in your kernel.
> Please check the RT"
> +        echo "settings as following if this test failed with 'Unable to change
> scheduling policy'."
> +        echo "- If the user does not need the RT groups functionality, disable
> CONFIG_RT_GROUP_SCHED and compile the kernel again, or 'sysctl -w
> kernel.sched_rt_runtime_us=-1'"
> +        echo "- If the user does want to use RT groups functionality, you may
> want to assign the ssh daemon service a realtime budget by specifyihg the
> cpu.rt_runtime_us attribute"
> +    fi
>  }
> 
>  function test_build {
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 14/15] hackbench: change original upstream branch

[Tim.Bird]

One comment below, but...

Applied.  Thanks.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> Sent: Wednesday, November 14, 2018 8:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 14/15] hackbench: change original upstream branch
> 
> The original upstream branch 'unstable/devel/v1.1.1' doesn't exist, which
> makes test_build fail.
> Use exist upstream branch.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.hackbench/spec.json | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/engine/tests/Benchmark.hackbench/spec.json
> b/engine/tests/Benchmark.hackbench/spec.json
> index ea90c43..78ddb07 100644
> --- a/engine/tests/Benchmark.hackbench/spec.json
> +++ b/engine/tests/Benchmark.hackbench/spec.json
> @@ -16,7 +16,7 @@
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> -            "gitref": "unstable/devel/v1.1.1",
> +            "gitref": "unstable/devel/latest",

In general, if it's possible to specify a fixed version of the test, that's my preference.
Some people want to always use the latest version of the test, but personally
I prefer to only change the version manually (and with human thought).

However, if v1.1.1 is not even available, this is an obvious improvement.

>              "PARAMS": " "
>          }
>      }
> --
> 2.7.4

Re: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of PARAMS

[Qiu, Tingting]

Hi,Tim

> -----Original Message-----
> From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> Sent: Friday, November 16, 2018 7:40 AM
> To: Qiu, Tingting/仇 婷婷; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of
> PARAMS
> 
> 
> 
> > -----Original Message-----
> > From: Qiu Tingting on Wednesday, November 14, 2018 8:26 PM
> >
> > Cmd ptsematest has -t num parameter.
> > When cpu num is not setted, default value is max num of cpu.
> > Maybe some of cpu is not working, so we should set the available cpu num.
> 
> Can you explain this more?  This change adjusts
> the code to have ptsematest use the number of cpus reported
> by /proc/cpuinfo.   This seems like it would be the maximum number of
> cpus on the system.

Referring to help information, we know that '-t' parameter means one thread per available processor.
So, it would be the maximum available number of cpus on the system.
ptsematest usage
===========================
-t       --threads         one thread per available processor
-t [NUM] --threads=NUM     number of threads:
                           without NUM, threads = max_cpus
                           without -t default = 1
===========================

By default, threads num is setted by sysconf(_SC_NPROCESSORS_CONF).
But, value of _SC_NPROCESSORS_CONF is maximum number of cpus on system, 
which include available processor and unavailable processor.
Then, there are some warning message in ptsematest's output, like:
===========================
WARNING: Could not set CPU affinity to CPU #2
WARNING: Could not set CPU affinity to CPU #2
WARNING: Could not set CPU affinity to CPU #3
WARNING: Could not set CPU affinity to CPU #3
...
===========================
And available processor number is got by /proc/cpuinfo before ptsematest cmd.
So，I think it's better to set available processor number by '-t [NUM]' parameter.

Regards,
Qiu Tingting
> 
> How would ptsematest use a different number of cpus?
> 
> >
> > Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> > ---
> >  engine/tests/Benchmark.ptsematest/fuego_test.sh | 2 +-
> >  engine/tests/Benchmark.ptsematest/spec.json     | 4 ++--
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> > b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> > index 5f3fb44..5f0633e 100755
> > --- a/engine/tests/Benchmark.ptsematest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.ptsematest/fuego_test.sh
> > @@ -21,5 +21,5 @@ function test_run {
> >      # The number for getting the lines depends on the cpu number of target
> > machine.
> >      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
> >      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> > -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest
> > $BENCHMARK_PTSEMATEST_PARAMS | tail -$getting_line_number"
> > +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ptsematest
> > $BENCHMARK_PTSEMATEST_PARAMS$target_cpu_number | tail -
> > $getting_line_number"
> >  }
> > diff --git a/engine/tests/Benchmark.ptsematest/spec.json
> > b/engine/tests/Benchmark.ptsematest/spec.json
> > index aa5d6ce..16bc9ee 100644
> > --- a/engine/tests/Benchmark.ptsematest/spec.json
> > +++ b/engine/tests/Benchmark.ptsematest/spec.json
> > @@ -2,13 +2,13 @@
> >      "testName": "Benchmark.ptsematest",
> >      "specs": {
> >          "default": {
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
> >          },
> >          "latest": {
> >              "PER_JOB_BUILD": "true",
> >              "gitrepo":
> "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> >              "gitref": "unstable/devel/latest",
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l100000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l100000 -t"
> >          }
> >      }
> >  }
> > --
> > 2.7.4
> 
> Not applied yet.  Please explain more what's going on.
>  -- Tim
> 

Re: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of PARAMS

[Tim.Bird]

> -----Original Message-----
> From: Qiu, Tingting 
> 
> Hi,Tim
> 
> > -----Original Message-----
> > From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> > > -----Original Message-----
> > > From: Qiu Tingting on Wednesday, November 14, 2018 8:26 PM
> > >
> > > Cmd ptsematest has -t num parameter.
> > > When cpu num is not setted, default value is max num of cpu.
> > > Maybe some of cpu is not working, so we should set the available cpu
> num.
> >
> > Can you explain this more?  This change adjusts
> > the code to have ptsematest use the number of cpus reported
> > by /proc/cpuinfo.   This seems like it would be the maximum number of
> > cpus on the system.
> 
> Referring to help information, we know that '-t' parameter means one
> thread per available processor.
> So, it would be the maximum available number of cpus on the system.
> ptsematest usage
> ===========================
> -t       --threads         one thread per available processor
> -t [NUM] --threads=NUM     number of threads:
>                            without NUM, threads = max_cpus
>                            without -t default = 1
> ===========================
> 
> By default, threads num is setted by sysconf(_SC_NPROCESSORS_CONF).
> But, value of _SC_NPROCESSORS_CONF is maximum number of cpus on
> system,
> which include available processor and unavailable processor.

So, does /proc/cpuinfo not report the unavailable processors?

 -- Tim

Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Qiu, Tingting]

Hi,Tim

> -----Original Message-----
> From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> Sent: Friday, November 16, 2018 7:46 AM
> To: Qiu, Tingting/仇 婷婷; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of
> PARAMS
> 
> Same question about this one as ptsematest.
> 
> Please explain better.  I don't understand how taking the number of
> processors from /proc/cpuinfo is going to omit processors that
> are not working.
> 
> I guess I don't understand the different algorithms that are being
> used by the RT test programs (ptsematest and svsematest) and
> the /proc/cpuinfo scan, that would yield a different numbers.
> 
> What is the behavior that is faulty, if you don't use the -t parameter
> to these tests?
'-t' parameter means one thread per available processor.
By default, threads number is set by sysconf(_SC_NPROCESSORS_CONF).
As ptsmematest, I think it's better to set available processor number by '-t [NUM]' parameter.
NUM can got by /proc/cpuinfo.
Other rt test(pmqtest, sigwaittest, deadlinetest) has the similar problem. 

Regards,
Qiu Tingting

>  -- Tim
> 
> 
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Qiu Tingting
> > Sent: Wednesday, November 14, 2018 8:26 PM
> > To: fuego@lists.linuxfoundation.org
> > Subject: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of
> > PARAMS
> >
> > Cmd svsematest has -t num parameter.
> > When cpu num is not setted, default value is max num of cpu.
> > Maybe some of cpu is not working, so we should set the available cpu num.
> >
> > Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> > ---
> >  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
> >  engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > index f2ebd18..2988cd4 100755
> > --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > @@ -21,5 +21,5 @@ function test_run {
> >      # The number for getting the lines depends on the cpu number of target
> > machine.
> >      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
> >      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> > -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
> > +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail -
> > $getting_line_number"
> >  }
> > diff --git a/engine/tests/Benchmark.svsematest/spec.json
> > b/engine/tests/Benchmark.svsematest/spec.json
> > index 00e77c1..e814600 100644
> > --- a/engine/tests/Benchmark.svsematest/spec.json
> > +++ b/engine/tests/Benchmark.svsematest/spec.json
> > @@ -2,13 +2,13 @@
> >      "testName": "Benchmark.svsematest",
> >      "specs": {
> >          "default": {
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> >          },
> >          "latest": {
> >              "PER_JOB_BUILD": "true",
> >              "gitrepo":
> "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> >              "gitref": "unstable/devel/latest",
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> >          }
> >      }
> >  }
> > --
> > 2.7.4
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 

Re: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of PARAMS

[Qiu, Tingting]

Hi, Tim

> So, does /proc/cpuinfo not report the unavailable processors?
No, /proc/cpuinfo does not report the unavailable processors.

Regards,
Qiu Tingting

> -----Original Message-----
> From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> Sent: Friday, November 16, 2018 12:31 PM
> To: Qiu, Tingting/仇 婷婷
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 07/15] ptsematest: fix unavailable cpu num of
> PARAMS
> 
> > -----Original Message-----
> > From: Qiu, Tingting
> >
> > Hi,Tim
> >
> > > -----Original Message-----
> > > From: Tim.Bird@sony.com [mailto:Tim.Bird@sony.com]
> > > > -----Original Message-----
> > > > From: Qiu Tingting on Wednesday, November 14, 2018 8:26 PM
> > > >
> > > > Cmd ptsematest has -t num parameter.
> > > > When cpu num is not setted, default value is max num of cpu.
> > > > Maybe some of cpu is not working, so we should set the available cpu
> > num.
> > >
> > > Can you explain this more?  This change adjusts
> > > the code to have ptsematest use the number of cpus reported
> > > by /proc/cpuinfo.   This seems like it would be the maximum number of
> > > cpus on the system.
> >
> > Referring to help information, we know that '-t' parameter means one
> > thread per available processor.
> > So, it would be the maximum available number of cpus on the system.
> > ptsematest usage
> > ===========================
> > -t       --threads         one thread per available processor
> > -t [NUM] --threads=NUM     number of threads:
> >                            without NUM, threads = max_cpus
> >                            without -t default = 1
> > ===========================
> >
> > By default, threads num is setted by sysconf(_SC_NPROCESSORS_CONF).
> > But, value of _SC_NPROCESSORS_CONF is maximum number of cpus on
> > system,
> > which include available processor and unavailable processor.
> 
> So, does /proc/cpuinfo not report the unavailable processors?
> 
>  -- Tim
> 
> 

Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Daniel Sangorrin]

Hi Qiu,

Please see my comments below.

> -----Original Message-----
> <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Qiu Tingting
> Sent: Thursday, November 15, 2018 1:26 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS
> 
> Cmd svsematest has -t num parameter.
> When cpu num is not setted, default value is max num of cpu.
> Maybe some of cpu is not working, so we should set the available cpu num.
> 
> Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> ---
>  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
>  engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> b/engine/tests/Benchmark.svsematest/fuego_test.sh
> index f2ebd18..2988cd4 100755
> --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> @@ -21,5 +21,5 @@ function test_run {
>      # The number for getting the lines depends on the cpu number of target
> machine.
>      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")

This will not work if you have a "hole" in your cpus (E.g: if you have cpu 0,1,3 online, but cpu 2 offline)

>      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail
> -$getting_line_number"
>  }

The number of threads should be specified in the spec, not limited to the number of online cores. For example, I may want to run 200 threads on two cores. Instead, what you want to use is the -a (affinity) argument. Unfortunately, unlike cyclictest, it seems that in svsematest you can only specify a single cpu number instead of a cpu set.

> diff --git a/engine/tests/Benchmark.svsematest/spec.json
> b/engine/tests/Benchmark.svsematest/spec.json
> index 00e77c1..e814600 100644
> --- a/engine/tests/Benchmark.svsematest/spec.json
> +++ b/engine/tests/Benchmark.svsematest/spec.json
> @@ -2,13 +2,13 @@
>      "testName": "Benchmark.svsematest",
>      "specs": {
>          "default": {
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"

Putting the -t at the end is not a good idea.
If we really wanted to handle -t automatically, as you suggest, then you should put add -t directly inside fuego_test.sh.
# as I said though, what we want to modify is the affinity

By the way, you should think of values in spec.json files as common examples. You can override them with values specific to your board in two ways:
1) Specifying the value on your board definition file
BENCHMARK_SVSEMATEST_PARAMS="-a 1 -t -p99 -i100 -d25 -l1000"
Tim: I just tested it and prolog.sh is not overriding the variable, might be a regression!
Tim: it seems that ftc:get_specs still doesn't support spec files in fuego-ro/boards or absolute paths. Shall I fix that?
2) Using the option --dynamic-vars (ftc run-test --help)

Summarizing, I think there are mainly two things we can do:
 - Modify the spec for your board by setting the affinity (-a N) to a processor you know it's going to be online (you can do that in two ways as I said)
 - Patch the program to specify affinity as a set of cpus (as in cyclictest) and then calculate that affinity automatically by using /proc/cpuinfo (unless the user already had specified it on the spec).

Thanks
Daniel


>          },
>          "latest": {
>              "PER_JOB_BUILD": "true",
>              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
>              "gitref": "unstable/devel/latest",
> -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
>          }
>      }
>  }
> --
> 2.7.4
> 
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Tim.Bird]

> -----Original Message-----
> From: Daniel Sangorrin
> Sent: Thursday, November 15, 2018 9:31 PM
> To: 'Qiu Tingting' <qiutt@cn.fujitsu.com>; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of
> PARAMS
> 
> Hi Qiu,
> 
> Please see my comments below.
> 
> > -----Original Message-----
> > From:  Qiu Tingting
> >
> > Cmd svsematest has -t num parameter.
> > When cpu num is not setted, default value is max num of cpu.
> > Maybe some of cpu is not working, so we should set the available cpu num.
> >
> > Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> > ---
> >  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
> >  engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > index f2ebd18..2988cd4 100755
> > --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > @@ -21,5 +21,5 @@ function test_run {
> >      # The number for getting the lines depends on the cpu number of target
> > machine.
> >      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
> 
> This will not work if you have a "hole" in your cpus (E.g: if you have cpu 0,1,3
> online, but cpu 2 offline)
> 
> >      getting_line_number=$(( $target_cpu_number + $target_cpu_number
> ))
> > -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
> > +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail
> > -$getting_line_number"
> >  }
> 
> The number of threads should be specified in the spec, not limited to the
> number of online cores. For example, I may want to run 200 threads on two
> cores. Instead, what you want to use is the -a (affinity) argument.
> Unfortunately, unlike cyclictest, it seems that in svsematest you can only
> specify a single cpu number instead of a cpu set.
> 
> > diff --git a/engine/tests/Benchmark.svsematest/spec.json
> > b/engine/tests/Benchmark.svsematest/spec.json
> > index 00e77c1..e814600 100644
> > --- a/engine/tests/Benchmark.svsematest/spec.json
> > +++ b/engine/tests/Benchmark.svsematest/spec.json
> > @@ -2,13 +2,13 @@
> >      "testName": "Benchmark.svsematest",
> >      "specs": {
> >          "default": {
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> 
> Putting the -t at the end is not a good idea.
> If we really wanted to handle -t automatically, as you suggest, then you
> should put add -t directly inside fuego_test.sh.
> # as I said though, what we want to modify is the affinity
> 
> By the way, you should think of values in spec.json files as common
> examples. You can override them with values specific to your board in two
> ways:
> 1) Specifying the value on your board definition file
> BENCHMARK_SVSEMATEST_PARAMS="-a 1 -t -p99 -i100 -d25 -l1000"
> Tim: I just tested it and prolog.sh is not overriding the variable, might be a
> regression!
I'll try to take a look at this.  That's an important mechanism for overriding
test variables from a spec.

> Tim: it seems that ftc:get_specs still doesn't support spec files in fuego-
> ro/boards or absolute paths. Shall I fix that?

Yes please. :-)

> 2) Using the option --dynamic-vars (ftc run-test --help)
> 
> Summarizing, I think there are mainly two things we can do:
>  - Modify the spec for your board by setting the affinity (-a N) to a processor
> you know it's going to be online (you can do that in two ways as I said)
>  - Patch the program to specify affinity as a set of cpus (as in cyclictest) and
> then calculate that affinity automatically by using /proc/cpuinfo (unless the
> user already had specified it on the spec).
> 
> Thanks
> Daniel
> 
> 
> >          },
> >          "latest": {
> >              "PER_JOB_BUILD": "true",
> >              "gitrepo": "https://git.kernel.org/pub/scm/utils/rt-tests/rt-
> tests.git",
> >              "gitref": "unstable/devel/latest",
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> >          }
> >      }
> >  }
> > --
> > 2.7.4
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS

[Qiu, Tingting]

Hi, Daniel

Thanks for your replay.
I will try to Specify the value on my board definition file.

Regards,
Qiu Tingting

> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Friday, November 16, 2018 2:31 PM
> To: Qiu, Tingting/仇 婷婷; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of
> PARAMS
> 
> Hi Qiu,
> 
> Please see my comments below.
> 
> > -----Original Message-----
> > <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Qiu Tingting
> > Sent: Thursday, November 15, 2018 1:26 PM
> > To: fuego@lists.linuxfoundation.org
> > Subject: [Fuego] [PATCH 12/15] svsematest: fix unavailable cpu num of PARAMS
> >
> > Cmd svsematest has -t num parameter.
> > When cpu num is not setted, default value is max num of cpu.
> > Maybe some of cpu is not working, so we should set the available cpu num.
> >
> > Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> > ---
> >  engine/tests/Benchmark.svsematest/fuego_test.sh | 2 +-
> >  engine/tests/Benchmark.svsematest/spec.json     | 4 ++--
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > index f2ebd18..2988cd4 100755
> > --- a/engine/tests/Benchmark.svsematest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.svsematest/fuego_test.sh
> > @@ -21,5 +21,5 @@ function test_run {
> >      # The number for getting the lines depends on the cpu number of target
> > machine.
> >      target_cpu_number=$(cmd "cat /proc/cpuinfo | grep processor | wc -l")
> 
> This will not work if you have a "hole" in your cpus (E.g: if you have cpu
> 0,1,3 online, but cpu 2 offline)
> 
> >      getting_line_number=$(( $target_cpu_number + $target_cpu_number ))
> > -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS | tail -$getting_line_number"
> > +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./svsematest
> > $BENCHMARK_SVSEMATEST_PARAMS$target_cpu_number | tail
> > -$getting_line_number"
> >  }
> 
> The number of threads should be specified in the spec, not limited to the number
> of online cores. For example, I may want to run 200 threads on two cores. Instead,
> what you want to use is the -a (affinity) argument. Unfortunately, unlike
> cyclictest, it seems that in svsematest you can only specify a single cpu number
> instead of a cpu set.
> 
> > diff --git a/engine/tests/Benchmark.svsematest/spec.json
> > b/engine/tests/Benchmark.svsematest/spec.json
> > index 00e77c1..e814600 100644
> > --- a/engine/tests/Benchmark.svsematest/spec.json
> > +++ b/engine/tests/Benchmark.svsematest/spec.json
> > @@ -2,13 +2,13 @@
> >      "testName": "Benchmark.svsematest",
> >      "specs": {
> >          "default": {
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> 
> Putting the -t at the end is not a good idea.
> If we really wanted to handle -t automatically, as you suggest, then you should
> put add -t directly inside fuego_test.sh.
> # as I said though, what we want to modify is the affinity
> 
> By the way, you should think of values in spec.json files as common examples.
> You can override them with values specific to your board in two ways:
> 1) Specifying the value on your board definition file
> BENCHMARK_SVSEMATEST_PARAMS="-a 1 -t -p99 -i100 -d25 -l1000"
> Tim: I just tested it and prolog.sh is not overriding the variable, might be
> a regression!
> Tim: it seems that ftc:get_specs still doesn't support spec files in
> fuego-ro/boards or absolute paths. Shall I fix that?
> 2) Using the option --dynamic-vars (ftc run-test --help)
> 
> Summarizing, I think there are mainly two things we can do:
>  - Modify the spec for your board by setting the affinity (-a N) to a processor
> you know it's going to be online (you can do that in two ways as I said)
>  - Patch the program to specify affinity as a set of cpus (as in cyclictest)
> and then calculate that affinity automatically by using /proc/cpuinfo (unless
> the user already had specified it on the spec).
> 
> Thanks
> Daniel
> 
> 
> >          },
> >          "latest": {
> >              "PER_JOB_BUILD": "true",
> >              "gitrepo":
> "https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git",
> >              "gitref": "unstable/devel/latest",
> > -            "PARAMS": "-a -t -p99 -i100 -d25 -l10000"
> > +            "PARAMS": "-a -p99 -i100 -d25 -l10000 -t"
> >          }
> >      }
> >  }
> > --
> > 2.7.4
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 
>