[Buildroot] [RFC PATCH 1/4] .gitlab-ci.yml: save rootfs as artifact for runtime tests

[Buildroot] [RFC PATCH 1/4] .gitlab-ci.yml: save rootfs as artifact for runtime tests

[Ricardo Martincoski]

Sometimes when a test fails in a gitlab pipeline the reason of the
failure cannot be determined using only the logfile.

Add the modified rootfs as an artifact of the job to improve
troubleshooting. To accomplish this, always use -k option from the test
infra, unconditionally add the resulting images to the artifacts, and
let the runner do the cleanup for us.

These artifacts can also be useful when a test fails locally but pass at
gitlab runners.

When the test does not generate a image, this message is displayed in
the runner log:
WARNING: test-output/*/images/*: no matching files

Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
See also
http://lists.busybox.net/pipermail/buildroot/2017-July/199332.html
---
 .gitlab-ci.yml    | 5 ++++-
 .gitlab-ci.yml.in | 5 ++++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 42707ad1a6..f9e5b1fa6b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -47,12 +47,15 @@ check-DEVELOPERS:
             - output/build/packages-file-list.txt
 
 .runtime_test: &runtime_test
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ ${CI_BUILD_NAME}
+    # Keep build directories so the rootfs can be an artifact of the job. The
+    # runner will clean up those files for us.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
         paths:
             - test-output/*.log
+            - test-output/*/images/*
 acmesystems_aria_g25_128mb_defconfig: *defconfig
 acmesystems_aria_g25_256mb_defconfig: *defconfig
 acmesystems_arietta_g25_128mb_defconfig: *defconfig
diff --git a/.gitlab-ci.yml.in b/.gitlab-ci.yml.in
index 3088677e67..b8fce9ef96 100644
--- a/.gitlab-ci.yml.in
+++ b/.gitlab-ci.yml.in
@@ -47,9 +47,12 @@ check-DEVELOPERS:
             - output/build/packages-file-list.txt
 
 .runtime_test: &runtime_test
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ ${CI_BUILD_NAME}
+    # Keep build directories so the rootfs can be an artifact of the job. The
+    # runner will clean up those files for us.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
         paths:
             - test-output/*.log
+            - test-output/*/images/*
-- 
2.13.0

[Buildroot] [RFC PATCH 2/4] support/testing: allow to use extra large timeouts

[Ricardo Martincoski]

Add a parameter to run-tests to act as a multiplier for all timeouts of
emulator.
It can be used to avoid sporadic failures on slow host machines as well
in elastic runners on the cloud.

Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
See also
http://lists.busybox.net/pipermail/buildroot/2017-July/199329.html
---
 support/testing/infra/basetest.py |  4 +++-
 support/testing/infra/emulator.py | 13 ++++++++++---
 support/testing/run-tests         |  9 +++++++++
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 431605b23f..1ef76a988b 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -35,6 +35,7 @@ class BRTest(unittest.TestCase):
     logtofile = True
     keepbuilds = False
     jlevel = 0
+    elastic_timeout = 1
 
     def __init__(self, names):
         super(BRTest, self).__init__(names)
@@ -60,7 +61,8 @@ class BRTest(unittest.TestCase):
             self.b.build()
             self.show_msg("Building done")
 
-        self.emulator = Emulator(self.builddir, self.downloaddir, self.logtofile)
+        self.emulator = Emulator(self.builddir, self.downloaddir,
+                                 self.logtofile, self.elastic_timeout)
 
     def tearDown(self):
         self.show_msg("Cleaning up")
diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 9b079cbf23..7f02a01553 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -5,10 +5,14 @@ import infra
 
 class Emulator(object):
 
-    def __init__(self, builddir, downloaddir, logtofile):
+    def __init__(self, builddir, downloaddir, logtofile, multiplier):
         self.qemu = None
         self.downloaddir = downloaddir
         self.logfile = infra.open_log_file(builddir, "run", logtofile)
+        # We use elastic runners on the cloud to runs our tests. Those runners
+        # can take a long time to run the emulator. Use a timeout multiplier
+        # when running the tests to avoid sporadic failures.
+        self.multiplier = multiplier
 
     # Start Qemu to boot the system
     #
@@ -65,7 +69,8 @@ class Emulator(object):
             qemu_cmd += ["-append", " ".join(kernel_cmdline)]
 
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
-        self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:], timeout=5)
+        self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
+                                  timeout=5 * self.multiplier)
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
 
@@ -75,7 +80,7 @@ class Emulator(object):
         # The login prompt can take some time to appear when running multiple
         # instances in parallel, so set the timeout to a large value
         index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
-                                 timeout=60)
+                                 timeout=60 * self.multiplier)
         if index != 0:
             self.logfile.write("==> System does not boot")
             raise SystemError("System does not boot")
@@ -93,6 +98,8 @@ class Emulator(object):
     # return a tuple (output, exit_code)
     def run(self, cmd, timeout=-1):
         self.qemu.sendline(cmd)
+        if timeout != -1:
+            timeout *= self.multiplier
         self.qemu.expect("# ", timeout=timeout)
         # Remove double carriage return from qemu stdout so str.splitlines()
         # works as expected.
diff --git a/support/testing/run-tests b/support/testing/run-tests
index 0cb673c61f..33ac60d6b2 100755
--- a/support/testing/run-tests
+++ b/support/testing/run-tests
@@ -28,6 +28,8 @@ def main():
                         help='number of testcases to run simultaneously')
     parser.add_argument('-j', '--jlevel', type=int,
                         help='BR2_JLEVEL to use for each testcase')
+    parser.add_argument('-e', '--elastic-timeout', type=int, default=1,
+                        help='increase timeouts (useful for slow machines)')
 
     args = parser.parse_args()
 
@@ -97,6 +99,13 @@ def main():
         # the user can override the auto calculated value
         BRTest.jlevel = args.jlevel
 
+    if args.elastic_timeout < 1:
+        print "Invalid multiplier for timeout values"
+        print ""
+        parser.print_help()
+        return 1
+    BRTest.elastic_timeout = args.elastic_timeout
+
     nose2_args = ["-v",
                   "-N", str(args.testcases),
                   "-s", "support/testing",
-- 
2.13.0

[Buildroot] [RFC PATCH 3/4] .gitlab-ci.yml: use large timeouts for runtime tests

[Ricardo Martincoski]

Multiply the timeouts for emulator in the gitlab runners by 10 to avoid
sporadic failures in elastic runners.

Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
See also
http://lists.busybox.net/pipermail/buildroot/2017-July/199329.html
---
 .gitlab-ci.yml    | 4 +++-
 .gitlab-ci.yml.in | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f9e5b1fa6b..b1a1cd31a4 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -49,7 +49,9 @@ check-DEVELOPERS:
 .runtime_test: &runtime_test
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
+    # Multiply every emulator timeout by 10 to avoid sporadic failures in
+    # elastic runners.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k -e 10 ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
diff --git a/.gitlab-ci.yml.in b/.gitlab-ci.yml.in
index b8fce9ef96..9d3ddfec3c 100644
--- a/.gitlab-ci.yml.in
+++ b/.gitlab-ci.yml.in
@@ -49,7 +49,9 @@ check-DEVELOPERS:
 .runtime_test: &runtime_test
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
+    # Multiply every emulator timeout by 10 to avoid sporadic failures in
+    # elastic runners.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k -e 10 ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
-- 
2.13.0

[Buildroot] [RFC PATCH 4/4] testing/infra/emulator: remove qemu warnings about audio

[Ricardo Martincoski]

The default audio backend for qemu is configured at compile time. It
generates annoying warning messages to qemu's stderr when running our
tests, like these:
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument

Explicitly set the audio backend to "none" at runtime to remove those
messages from our logs. There is no command line argument for this, so
use an environment variable when starting qemu.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 support/testing/infra/emulator.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 7f02a01553..fe3da869e7 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -70,7 +70,9 @@ class Emulator(object):
 
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
         self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
-                                  timeout=5 * self.multiplier)
+                                  timeout=5 * self.multiplier,
+                                  env={"QEMU_AUDIO_DRV": "none"})
+
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
 
-- 
2.13.0

[Buildroot] [RFC PATCH 1/4] .gitlab-ci.yml: save rootfs as artifact for runtime tests

[Thomas Petazzoni]

Hello,

On Sun, 30 Jul 2017 01:49:43 -0300, Ricardo Martincoski wrote:
> Sometimes when a test fails in a gitlab pipeline the reason of the
> failure cannot be determined using only the logfile.
> 
> Add the modified rootfs as an artifact of the job to improve
> troubleshooting. To accomplish this, always use -k option from the test
> infra, unconditionally add the resulting images to the artifacts, and
> let the runner do the cleanup for us.
> 
> These artifacts can also be useful when a test fails locally but pass at
> gitlab runners.
> 
> When the test does not generate a image, this message is displayed in
> the runner log:
> WARNING: test-output/*/images/*: no matching files
> 
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> ---
> See also
> http://lists.busybox.net/pipermail/buildroot/2017-July/199332.html
> ---
>  .gitlab-ci.yml    | 5 ++++-
>  .gitlab-ci.yml.in | 5 ++++-
>  2 files changed, 8 insertions(+), 2 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [RFC PATCH 2/4] support/testing: allow to use extra large timeouts

[Thomas Petazzoni]

Hello,

On Sun, 30 Jul 2017 01:49:44 -0300, Ricardo Martincoski wrote:

> diff --git a/support/testing/run-tests b/support/testing/run-tests
> index 0cb673c61f..33ac60d6b2 100755
> --- a/support/testing/run-tests
> +++ b/support/testing/run-tests
> @@ -28,6 +28,8 @@ def main():
>                          help='number of testcases to run simultaneously')
>      parser.add_argument('-j', '--jlevel', type=int,
>                          help='BR2_JLEVEL to use for each testcase')
> +    parser.add_argument('-e', '--elastic-timeout', type=int, default=1,
> +                        help='increase timeouts (useful for slow machines)')

I find the choice of the "elastic timeout" a bit weird. To me, it feels
like a timeout that will reduce and extend like an elastic, not a
timeout multiplier.

Perhaps we should call this --timeout-multiplier or something like
that ?

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [RFC PATCH 4/4] testing/infra/emulator: remove qemu warnings about audio

[Thomas Petazzoni]

Hello,

On Sun, 30 Jul 2017 01:49:46 -0300, Ricardo Martincoski wrote:
> The default audio backend for qemu is configured at compile time. It
> generates annoying warning messages to qemu's stderr when running our
> tests, like these:
> pulseaudio: set_sink_input_volume() failed
> pulseaudio: Reason: Invalid argument
> pulseaudio: set_sink_input_mute() failed
> pulseaudio: Reason: Invalid argument
> 
> Explicitly set the audio backend to "none" at runtime to remove those
> messages from our logs. There is no command line argument for this, so
> use an environment variable when starting qemu.
> 
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> ---
>  support/testing/infra/emulator.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [RFC PATCH 2/4] support/testing: allow to use extra large timeouts

[Ricardo Martincoski]

Hello,

On Mon, Jul 31, 2017 at 04:25 PM, Thomas Petazzoni wrote:

> On Sun, 30 Jul 2017 01:49:44 -0300, Ricardo Martincoski wrote:
[snip]
>> +    parser.add_argument('-e', '--elastic-timeout', type=int, default=1,
>> +                        help='increase timeouts (useful for slow machines)')
> 
> I find the choice of the "elastic timeout" a bit weird. To me, it feels
> like a timeout that will reduce and extend like an elastic, not a
> timeout multiplier.
> 
> Perhaps we should call this --timeout-multiplier or something like
> that ?

Sure. I will fix and resend.
I will use -m as short name because -t is already in use.

Regards,
Ricardo

[Buildroot] [RFC PATCH 2/4] support/testing: allow to use extra large timeouts

[Thomas Petazzoni]

Hello,

On Mon, 31 Jul 2017 20:27:59 -0300, Ricardo Martincoski wrote:

> > Perhaps we should call this --timeout-multiplier or something like
> > that ?  
> 
> Sure. I will fix and resend.
> I will use -m as short name because -t is already in use.

Or perhaps this option is "specialized" enough that it doesn't need a
short name, so that we avoid "polluting" the namespace of short names,
which as you've seen, already has some collisions :)

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v2 1/2] support/testing: allow to use a multiplier for timeouts

[Ricardo Martincoski]

Add a parameter to run-tests to act as a multiplier for all timeouts of
emulator.
It can be used to avoid sporadic failures on slow host machines as well
in elastic runners on the cloud.

Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
Changes v1 -> v2:
  - use clearer parameter name (elastic-timeout -> timeout-multiplier)
    (Thomas);
  - adapt commit message;
---
 support/testing/infra/basetest.py |  4 +++-
 support/testing/infra/emulator.py | 13 ++++++++++---
 support/testing/run-tests         |  9 +++++++++
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 431605b23f..493dea5125 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -35,6 +35,7 @@ class BRTest(unittest.TestCase):
     logtofile = True
     keepbuilds = False
     jlevel = 0
+    timeout_multiplier = 1
 
     def __init__(self, names):
         super(BRTest, self).__init__(names)
@@ -60,7 +61,8 @@ class BRTest(unittest.TestCase):
             self.b.build()
             self.show_msg("Building done")
 
-        self.emulator = Emulator(self.builddir, self.downloaddir, self.logtofile)
+        self.emulator = Emulator(self.builddir, self.downloaddir,
+                                 self.logtofile, self.timeout_multiplier)
 
     def tearDown(self):
         self.show_msg("Cleaning up")
diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 588a92f6a2..214a1067e9 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -5,10 +5,14 @@ import infra
 
 class Emulator(object):
 
-    def __init__(self, builddir, downloaddir, logtofile):
+    def __init__(self, builddir, downloaddir, logtofile, multiplier):
         self.qemu = None
         self.downloaddir = downloaddir
         self.logfile = infra.open_log_file(builddir, "run", logtofile)
+        # We use elastic runners on the cloud to runs our tests. Those runners
+        # can take a long time to run the emulator. Use a timeout multiplier
+        # when running the tests to avoid sporadic failures.
+        self.multiplier = multiplier
 
     # Start Qemu to boot the system
     #
@@ -65,7 +69,8 @@ class Emulator(object):
             qemu_cmd += ["-append", " ".join(kernel_cmdline)]
 
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
-        self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:], timeout=5,
+        self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
+                                  timeout=5 * self.multiplier,
                                   env={"QEMU_AUDIO_DRV": "none"})
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
@@ -76,7 +81,7 @@ class Emulator(object):
         # The login prompt can take some time to appear when running multiple
         # instances in parallel, so set the timeout to a large value
         index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
-                                 timeout=60)
+                                 timeout=60 * self.multiplier)
         if index != 0:
             self.logfile.write("==> System does not boot")
             raise SystemError("System does not boot")
@@ -94,6 +99,8 @@ class Emulator(object):
     # return a tuple (output, exit_code)
     def run(self, cmd, timeout=-1):
         self.qemu.sendline(cmd)
+        if timeout != -1:
+            timeout *= self.multiplier
         self.qemu.expect("# ", timeout=timeout)
         # Remove double carriage return from qemu stdout so str.splitlines()
         # works as expected.
diff --git a/support/testing/run-tests b/support/testing/run-tests
index 0cb673c61f..95c1565f72 100755
--- a/support/testing/run-tests
+++ b/support/testing/run-tests
@@ -28,6 +28,8 @@ def main():
                         help='number of testcases to run simultaneously')
     parser.add_argument('-j', '--jlevel', type=int,
                         help='BR2_JLEVEL to use for each testcase')
+    parser.add_argument('--timeout-multiplier', type=int, default=1,
+                        help='increase timeouts (useful for slow machines)')
 
     args = parser.parse_args()
 
@@ -97,6 +99,13 @@ def main():
         # the user can override the auto calculated value
         BRTest.jlevel = args.jlevel
 
+    if args.timeout_multiplier < 1:
+        print "Invalid multiplier for timeout values"
+        print ""
+        parser.print_help()
+        return 1
+    BRTest.timeout_multiplier = args.timeout_multiplier
+
     nose2_args = ["-v",
                   "-N", str(args.testcases),
                   "-s", "support/testing",
-- 
2.13.0

[Buildroot] [PATCH v2 2/2] .gitlab-ci.yml: use large timeouts for runtime tests

[Ricardo Martincoski]

Multiply the timeouts for emulator in the gitlab runners by 10 to avoid
sporadic failures in elastic runners.

Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
Changes v1 -> v2:
  - use --timeout-multiplier instead of -e, as it changed in v2 of the
    previous patch;

Sample pipeline using only gitlab runners:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/10605884

PS: for this other sporadic error:
ERROR: Job failed: execution took longer than 1h0m0s seconds
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/28241402
see
https://docs.gitlab.com/ee/user/project/pipelines/settings.html#timeout
---
 .gitlab-ci.yml    | 4 +++-
 .gitlab-ci.yml.in | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 90efd81e50..19335d7bfd 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -49,7 +49,9 @@ check-DEVELOPERS:
 .runtime_test: &runtime_test
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
+    # Multiply every emulator timeout by 10 to avoid sporadic failures in
+    # elastic runners.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k --timeout-multiplier 10 ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
diff --git a/.gitlab-ci.yml.in b/.gitlab-ci.yml.in
index b8fce9ef96..3abf7d5313 100644
--- a/.gitlab-ci.yml.in
+++ b/.gitlab-ci.yml.in
@@ -49,7 +49,9 @@ check-DEVELOPERS:
 .runtime_test: &runtime_test
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
-    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k ${CI_BUILD_NAME}
+    # Multiply every emulator timeout by 10 to avoid sporadic failures in
+    # elastic runners.
+    script: ./support/testing/run-tests -o test-output/ -d test-dl/ -k --timeout-multiplier 10 ${CI_BUILD_NAME}
     artifacts:
         when: always
         expire_in: 2 weeks
-- 
2.13.0

[Buildroot] [PATCH v2 1/2] support/testing: allow to use a multiplier for timeouts

[Arnout Vandecappelle]

On 05-08-17 04:05, Ricardo Martincoski wrote:
> Add a parameter to run-tests to act as a multiplier for all timeouts of
> emulator.
> It can be used to avoid sporadic failures on slow host machines as well
> in elastic runners on the cloud.
> 
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>

[snip]
> -    def __init__(self, builddir, downloaddir, logtofile):
> +    def __init__(self, builddir, downloaddir, logtofile, multiplier):

 For consistency, it would be better to use timeout_multiplier here as well.
I've fixed that up and applied both to master, thanks.

 Regards,
 Arnout

[snip]

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF