DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
* [dpdk-dev] [PATCH] app/test: fix autotest_runner crash
@ 2019-06-12 13:33 Herakliusz Lipiec
  2019-06-12 14:18 ` Burakov, Anatoly
  2019-06-12 14:46 ` [dpdk-dev] [PATCH v2] " Herakliusz Lipiec
  0 siblings, 2 replies; 5+ messages in thread
From: Herakliusz Lipiec @ 2019-06-12 13:33 UTC (permalink / raw)
  Cc: dev, Herakliusz Lipiec, anatoly.burakov, stable

On some systems when dpdk test is executed with make test command
autotest_runner crashes in first_cpu_on_node. This happens when list
of available cpus contains something that is not a cpu as first element.
Fixed by removing all non-cpu values from list of available cpus.

Bugzilla ID: 253
Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
Cc: anatoly.burakov@intel.com
Cc: stable@dpdk.org
Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
---
 app/test/autotest_runner.py | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index b72716e1e..7aece8905 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -43,11 +43,10 @@ def get_numa_nodes():
 # find first (or any, really) CPU on a particular node, will be used to spread
 # processes around NUMA nodes to avoid exhausting memory on particular node
 def first_cpu_on_node(node_nr):
-    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
-    cpu_name = os.path.basename(cpu_path)
-    m = re.match(r"cpu(\d+)", cpu_name)
-    return int(m.group(1))
-
+    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
+    r = re.compile(r"cpu(\d+)")
+    cpu_name = filter(None ,map(r.match,  map(os.path.basename, cpu_path)))
+    return int(next(iter(cpu_name)).group(1))
 
 pool_child = None  # per-process child
 
-- 
2.17.2


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] [PATCH] app/test: fix autotest_runner crash
  2019-06-12 13:33 [dpdk-dev] [PATCH] app/test: fix autotest_runner crash Herakliusz Lipiec
@ 2019-06-12 14:18 ` Burakov, Anatoly
  2019-06-12 14:46 ` [dpdk-dev] [PATCH v2] " Herakliusz Lipiec
  1 sibling, 0 replies; 5+ messages in thread
From: Burakov, Anatoly @ 2019-06-12 14:18 UTC (permalink / raw)
  To: Herakliusz Lipiec; +Cc: dev, stable

On 12-Jun-19 2:33 PM, Herakliusz Lipiec wrote:
> On some systems when dpdk test is executed with make test command
> autotest_runner crashes in first_cpu_on_node. This happens when list
> of available cpus contains something that is not a cpu as first element.
> Fixed by removing all non-cpu values from list of available cpus.
> 
> Bugzilla ID: 253
> Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
> Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> ---
>   app/test/autotest_runner.py | 9 ++++-----
>   1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
> index b72716e1e..7aece8905 100644
> --- a/app/test/autotest_runner.py
> +++ b/app/test/autotest_runner.py
> @@ -43,11 +43,10 @@ def get_numa_nodes():
>   # find first (or any, really) CPU on a particular node, will be used to spread
>   # processes around NUMA nodes to avoid exhausting memory on particular node
>   def first_cpu_on_node(node_nr):
> -    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
> -    cpu_name = os.path.basename(cpu_path)
> -    m = re.match(r"cpu(\d+)", cpu_name)
> -    return int(m.group(1))
> -

This is an unnecessary whitespace change.

> +    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
> +    r = re.compile(r"cpu(\d+)")
> +    cpu_name = filter(None ,map(r.match,  map(os.path.basename, cpu_path)))

Typo, should be "None, map" rather than "None ,map". Also, perhaps 
splitting this on multiple lines would benefit readability, like this:

cpu_name = filter(None,
         map(r.match,
             map(os.path.basename, cpu_path)
         )
)

No preference though, can be left as is.

> +    return int(next(iter(cpu_name)).group(1))

IMO too much going on on one line, maybe leave the return as it was and 
just fix the match? As in,

m = next(iter(cpu_name))
return int(m.group(1))

Also probably needs a comment explaining why iter() is there - namely, 
that return value from a filter is a list in Python 2, but a generator 
in Python 3, and next() only works with generators.

Otherwise,

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

>   
>   pool_child = None  # per-process child
>   
> 


-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash
  2019-06-12 13:33 [dpdk-dev] [PATCH] app/test: fix autotest_runner crash Herakliusz Lipiec
  2019-06-12 14:18 ` Burakov, Anatoly
@ 2019-06-12 14:46 ` " Herakliusz Lipiec
  2019-06-12 14:48   ` Burakov, Anatoly
  1 sibling, 1 reply; 5+ messages in thread
From: Herakliusz Lipiec @ 2019-06-12 14:46 UTC (permalink / raw)
  Cc: dev, Herakliusz Lipiec, anatoly.burakov, stable

On some systems when dpdk test is executed with make test command
autotest_runner crashes in first_cpu_on_node. This happens when list
of available cpus contains something that is not a cpu as first element.
Fixed by removing all non-cpu values from list of available cpus.

Bugzilla ID: 253
Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
Cc: anatoly.burakov@intel.com
Cc: stable@dpdk.org
Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
---
 app/test/autotest_runner.py | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index b72716e1e..95e74c760 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -43,9 +43,16 @@ def get_numa_nodes():
 # find first (or any, really) CPU on a particular node, will be used to spread
 # processes around NUMA nodes to avoid exhausting memory on particular node
 def first_cpu_on_node(node_nr):
-    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
-    cpu_name = os.path.basename(cpu_path)
-    m = re.match(r"cpu(\d+)", cpu_name)
+    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
+    r = re.compile(r"cpu(\d+)")
+    cpu_name = filter(None,
+            map(r.match,
+                map(os.path.basename, cpu_path)
+            )
+    )
+    # for compatibility between python 3 and 2 we need to make interable out
+    # of filter return as it returns list in python 2 and a generator in 3
+    m = next(iter(cpu_name))
     return int(m.group(1))
 
 
-- 
2.17.2


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash
  2019-06-12 14:46 ` [dpdk-dev] [PATCH v2] " Herakliusz Lipiec
@ 2019-06-12 14:48   ` Burakov, Anatoly
  2019-07-04 21:19     ` Thomas Monjalon
  0 siblings, 1 reply; 5+ messages in thread
From: Burakov, Anatoly @ 2019-06-12 14:48 UTC (permalink / raw)
  To: Herakliusz Lipiec; +Cc: dev, stable

On 12-Jun-19 3:46 PM, Herakliusz Lipiec wrote:
> On some systems when dpdk test is executed with make test command
> autotest_runner crashes in first_cpu_on_node. This happens when list
> of available cpus contains something that is not a cpu as first element.
> Fixed by removing all non-cpu values from list of available cpus.
> 
> Bugzilla ID: 253
> Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
> Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> ---

Missed reviewed tag from previous revision.

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash
  2019-06-12 14:48   ` Burakov, Anatoly
@ 2019-07-04 21:19     ` Thomas Monjalon
  0 siblings, 0 replies; 5+ messages in thread
From: Thomas Monjalon @ 2019-07-04 21:19 UTC (permalink / raw)
  To: Herakliusz Lipiec; +Cc: dev, Burakov, Anatoly, stable

12/06/2019 16:48, Burakov, Anatoly:
> On 12-Jun-19 3:46 PM, Herakliusz Lipiec wrote:
> > On some systems when dpdk test is executed with make test command
> > autotest_runner crashes in first_cpu_on_node. This happens when list
> > of available cpus contains something that is not a cpu as first element.
> > Fixed by removing all non-cpu values from list of available cpus.
> > 
> > Bugzilla ID: 253
> > Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> > Cc: anatoly.burakov@intel.com
> > Cc: stable@dpdk.org
> > Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> 
> Missed reviewed tag from previous revision.
> 
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

Applied, thanks



^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-12 13:33 [dpdk-dev] [PATCH] app/test: fix autotest_runner crash Herakliusz Lipiec
2019-06-12 14:18 ` Burakov, Anatoly
2019-06-12 14:46 ` [dpdk-dev] [PATCH v2] " Herakliusz Lipiec
2019-06-12 14:48   ` Burakov, Anatoly
2019-07-04 21:19     ` Thomas Monjalon

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org dpdk-dev@archiver.kernel.org
	public-inbox-index dpdk-dev


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox