DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: David Marchand <david.marchand@redhat.com>
To: dev@dpdk.org
Cc: thomas@monjalon.net, aconole@redhat.com, msantana@redhat.com
Subject: [dpdk-dev] [PATCH 13/14] test: do not start tests in parallel
Date: Tue,  4 Jun 2019 10:59:51 +0200
Message-ID: <1559638792-8608-14-git-send-email-david.marchand@redhat.com> (raw)
In-Reply-To: <1559638792-8608-1-git-send-email-david.marchand@redhat.com>

Running the tests in parallel has two drawbacks:
- the tests are racing on the hugepages allocations,
- the tests are sharing the cores to run their checks which results in
  undeterministic execution time,

This results in random failures.
For better reproducibility in CI, start them all in a serialised way.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 app/test/meson.build | 25 +++++--------------------
 1 file changed, 5 insertions(+), 20 deletions(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index 212cd1b..f0797c9 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -144,8 +144,7 @@ test_deps = ['acl',
 	'timer'
 ]
 
-# All test cases in fast_parallel_test_names list are parallel
-fast_parallel_test_names = [
+fast_test_names = [
         'acl_autotest',
         'alarm_autotest',
         'atomic_autotest',
@@ -192,10 +191,6 @@ fast_parallel_test_names = [
         'timer_autotest',
         'user_delay_us',
         'version_autotest',
-]
-
-# All test cases in fast_non_parallel_test_names list are non-parallel
-fast_non_parallel_test_names = [
         'bitratestats_autotest',
         'crc_autotest',
         'delay_us_sleep_autotest',
@@ -219,7 +214,6 @@ fast_non_parallel_test_names = [
         'thash_autotest',
 ]
 
-# All test cases in perf_test_names list are non-parallel
 perf_test_names = [
         'ring_perf_autotest',
         'mempool_perf_autotest',
@@ -247,7 +241,6 @@ perf_test_names = [
         'stack_lf_perf_autotest',
 ]
 
-# All test cases in driver_test_names list are non-parallel
 driver_test_names = [
         'cryptodev_aesni_mb_autotest',
         'cryptodev_aesni_gcm_autotest',
@@ -269,7 +262,6 @@ driver_test_names = [
         'link_bonding_rssconf_autotest',
 ]
 
-# All test cases in dump_test_names list are non-parallel
 dump_test_names = [
         'dump_struct_sizes',
         'dump_mempool',
@@ -318,7 +310,7 @@ if dpdk_conf.has('RTE_LIBRTE_COMPRESSDEV')
 		test_dep_objs += compress_test_dep
 		test_sources += 'test_compressdev.c'
 		test_deps += 'compressdev'
-		fast_non_parallel_test_names += 'compressdev_autotest'
+		fast_test_names += 'compressdev_autotest'
 	endif
 endif
 
@@ -367,30 +359,23 @@ if get_option('tests')
 	num_cores_arg = '-l ' + num_cores
 
 	test_args = [num_cores_arg]
-	foreach arg : fast_parallel_test_names
+	foreach arg : fast_test_names
 		if host_machine.system() == 'linux'
 			test(arg, dpdk_test,
 				  env : ['DPDK_TEST=' + arg],
 				  args : test_args +
 					 ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds_fast,
+			is_parallel : false,
 			suite : 'fast-tests')
 		else
 			test(arg, dpdk_test,
 				env : ['DPDK_TEST=' + arg],
 				args : test_args,
 			timeout : timeout_seconds_fast,
-			suite : 'fast-tests')
-		endif
-	endforeach
-
-	foreach arg : fast_non_parallel_test_names
-		test(arg, dpdk_test,
-			env : ['DPDK_TEST=' + arg],
-			args : test_args,
-			timeout : timeout_seconds_fast,
 			is_parallel : false,
 			suite : 'fast-tests')
+		endif
 	endforeach
 
 	foreach arg : perf_test_names
-- 
1.8.3.1


  parent reply index

Thread overview: 71+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-04  8:59 [dpdk-dev] [PATCH 00/14] Unit tests fixes for CI David Marchand
2019-06-04  8:59 ` [dpdk-dev] [PATCH 01/14] test/bonding: add missing sources for link bonding RSS David Marchand
2019-06-04 12:59   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 02/14] test/crypto: move tests to the driver specific list David Marchand
2019-06-04 13:00   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 03/14] test/eventdev: " David Marchand
2019-06-04 13:04   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 04/14] test/hash: fix off-by-one check on core count David Marchand
2019-06-04 13:05   ` Aaron Conole
2019-06-05 20:02   ` Wang, Yipeng1
2019-06-04  8:59 ` [dpdk-dev] [PATCH 05/14] test/hash: rectify slaveid to point to valid cores David Marchand
2019-06-05 20:02   ` Wang, Yipeng1
2019-06-04  8:59 ` [dpdk-dev] [PATCH 06/14] test/hash: clean remaining trace of scaling autotest David Marchand
2019-06-04 13:31   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 07/14] test/latencystats: fix stack smashing David Marchand
2019-06-04 13:38   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 08/14] test/stack: fix lock-free test name David Marchand
2019-06-04 13:06   ` Aaron Conole
2019-06-04  8:59 ` [dpdk-dev] [PATCH 09/14] test/eal: set memory channel config only in dedicated test David Marchand
2019-06-04 13:11   ` Aaron Conole
2019-06-26  9:44   ` Burakov, Anatoly
2019-06-04  8:59 ` [dpdk-dev] [PATCH 10/14] test/eal: set core mask/list " David Marchand
2019-06-04 13:12   ` Aaron Conole
2019-06-26  9:45   ` Burakov, Anatoly
2019-06-04  8:59 ` [dpdk-dev] [PATCH 11/14] test/eal: check number of cores before running subtests David Marchand
2019-06-04 13:26   ` Aaron Conole
2019-06-26  9:47   ` Burakov, Anatoly
2019-06-04  8:59 ` [dpdk-dev] [PATCH 12/14] test/eal: make the test pass again David Marchand
2019-06-04 13:29   ` Aaron Conole
2019-06-04 13:50     ` David Marchand
2019-06-26  9:49   ` Burakov, Anatoly
2019-06-26 10:03     ` David Marchand
2019-06-04  8:59 ` David Marchand [this message]
2019-06-04  8:59 ` [dpdk-dev] [PATCH 14/14] test: skip tests when missing requirements David Marchand
2019-06-07 20:54   ` Honnappa Nagarahalli
2019-06-08  8:01     ` David Marchand
2019-06-11  4:08       ` Honnappa Nagarahalli
2019-06-04 15:49 ` [dpdk-dev] [PATCH 00/14] Unit tests fixes for CI Michael Santana Francisco
2019-06-27 16:34   ` Thomas Monjalon
2019-07-01 12:17     ` Aaron Conole
2019-06-15  6:42 ` [dpdk-dev] [PATCH v2 00/15] " David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 01/15] test/bonding: add missing sources for link bonding RSS David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 02/15] test/crypto: move tests to the driver specific list David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 03/15] test/eventdev: " David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 04/15] test/hash: fix off-by-one check on core count David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 05/15] test/hash: rectify slaveid to point to valid cores David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 06/15] test/hash: clean remaining trace of scaling autotest David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 07/15] test/latencystats: fix stack smashing David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 08/15] test/rcu: remove arbitrary limit on max core count David Marchand
2019-06-28 12:56     ` [dpdk-dev] [dpdk-stable] " Ferruh Yigit
2019-06-28 13:32       ` David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 09/15] test/stack: fix lock-free test name David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 10/15] test/eal: set memory channel config only in dedicated test David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 11/15] test/eal: set core mask/list " David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 12/15] test/eal: check number of cores before running subtests David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 13/15] test: split into shorter subtests for CI David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 14/15] test: do not start tests in parallel David Marchand
2019-06-15  6:42   ` [dpdk-dev] [PATCH v2 15/15] test: skip tests when missing requirements David Marchand
2019-06-17 10:00   ` [dpdk-dev] [PATCH v2 00/15] Unit tests fixes for CI Bruce Richardson
2019-06-17 10:46     ` David Marchand
2019-06-17 11:17       ` Bruce Richardson
2019-06-17 11:41         ` David Marchand
2019-06-17 11:56           ` Bruce Richardson
2019-06-17 13:44             ` David Marchand
2019-06-27 20:36   ` Thomas Monjalon
2019-07-01 16:04     ` Aaron Conole
2019-07-01 16:22       ` Thomas Monjalon
2019-07-01 16:45       ` David Marchand
2019-07-01 18:07         ` Michael Santana Francisco
2019-07-09 15:50           ` Michael Santana Francisco
2019-07-10  8:18             ` David Marchand

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1559638792-8608-14-git-send-email-david.marchand@redhat.com \
    --to=david.marchand@redhat.com \
    --cc=aconole@redhat.com \
    --cc=dev@dpdk.org \
    --cc=msantana@redhat.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org dpdk-dev@archiver.kernel.org
	public-inbox-index dpdk-dev


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox