linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 1/2] torture: Select line in sed and replace grep
@ 2022-02-22 12:07 Paul Menzel
  2022-02-22 12:07 ` [PATCH 2/2] torture: Make thread detection more robust by using lspcu Paul Menzel
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Menzel @ 2022-02-22 12:07 UTC (permalink / raw)
  To: Paul E. McKenney, Josh Triplett, Steven Rostedt,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan
  Cc: Paul Menzel, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

sed’s switch `-n` (`--silent`) suppresses the automatic printing of
the pattern space, therefore, allowing to replace grep by only
printing the current pattern space using the command `p`.

Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
---
 tools/testing/selftests/rcutorture/bin/functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index c35ba24f994c..5cff520955e6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -301,7 +301,7 @@ specify_qemu_cpus () {
 			echo $2 -smp $3
 			;;
 		qemu-system-ppc64)
-			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
+			nt="`lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'`"
 			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
 			;;
 		esac
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-22 12:07 [PATCH 1/2] torture: Select line in sed and replace grep Paul Menzel
@ 2022-02-22 12:07 ` Paul Menzel
  2022-02-22 17:43   ` Paul E. McKenney
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Menzel @ 2022-02-22 12:07 UTC (permalink / raw)
  To: Paul E. McKenney, Josh Triplett, Steven Rostedt,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan
  Cc: Paul Menzel, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

For consecutive numbers *lscpu* collapses the output and just shows the
range with start and end. The processors are numbered that way on POWER8.

    $ sudo ppc64_cpu --smt=8
    $ lscpu | grep '^NUMA node'
    NUMA node(s):                    2
    NUMA node0 CPU(s):               0-79
    NUMA node8 CPU(s):               80-159

This causes the heuristic to detect the number threads per core, looking
for the number after the first comma, to fail, and QEMU aborts because of
invalid arguments.

    $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
    $

(Before the last patch, the whole line was returned.)

    $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
    NUMA node0 CPU(s):               0-79

*lscpu* shows the number of threads per core, so use that value directly.

    $ sudo ppc64_cpu --smt=8
    $ lscpu | grep 'Thread(s) per core'
    Thread(s) per core:              8
    $ sudo ppc64_cpu --smt=off
    $ lscpu | grep 'Thread(s) per core'
    Thread(s) per core:              1

Note, the replaced heuristic is also incorrect for that case, where the
threads per core are disabled.

    $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
    8

Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
---
 tools/testing/selftests/rcutorture/bin/functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 5cff520955e6..66d0414d8e4b 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -301,7 +301,7 @@ specify_qemu_cpus () {
 			echo $2 -smp $3
 			;;
 		qemu-system-ppc64)
-			nt="`lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'`"
+			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
 			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
 			;;
 		esac
-- 
2.35.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-22 12:07 ` [PATCH 2/2] torture: Make thread detection more robust by using lspcu Paul Menzel
@ 2022-02-22 17:43   ` Paul E. McKenney
  2022-02-24  8:24     ` Paul Menzel
  0 siblings, 1 reply; 7+ messages in thread
From: Paul E. McKenney @ 2022-02-22 17:43 UTC (permalink / raw)
  To: Paul Menzel
  Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
	Shuah Khan, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
> For consecutive numbers *lscpu* collapses the output and just shows the
> range with start and end. The processors are numbered that way on POWER8.
> 
>     $ sudo ppc64_cpu --smt=8
>     $ lscpu | grep '^NUMA node'
>     NUMA node(s):                    2
>     NUMA node0 CPU(s):               0-79
>     NUMA node8 CPU(s):               80-159
> 
> This causes the heuristic to detect the number threads per core, looking
> for the number after the first comma, to fail, and QEMU aborts because of
> invalid arguments.
> 
>     $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>     $
> 
> (Before the last patch, the whole line was returned.)
> 
>     $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>     NUMA node0 CPU(s):               0-79
> 
> *lscpu* shows the number of threads per core, so use that value directly.
> 
>     $ sudo ppc64_cpu --smt=8
>     $ lscpu | grep 'Thread(s) per core'
>     Thread(s) per core:              8
>     $ sudo ppc64_cpu --smt=off
>     $ lscpu | grep 'Thread(s) per core'
>     Thread(s) per core:              1
> 
> Note, the replaced heuristic is also incorrect for that case, where the
> threads per core are disabled.
> 
>     $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>     8
> 
> Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>

Makes sense, and thank you for chasing this down and for the fix!

But should this patch and 1/2 be merged?  Or am I confused and they
are somehow affecting two different lines of scripting?

							Thanx, Paul

> ---
>  tools/testing/selftests/rcutorture/bin/functions.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> index 5cff520955e6..66d0414d8e4b 100644
> --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> @@ -301,7 +301,7 @@ specify_qemu_cpus () {
>  			echo $2 -smp $3
>  			;;
>  		qemu-system-ppc64)
> -			nt="`lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'`"
> +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
>  			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
>  			;;
>  		esac
> -- 
> 2.35.1
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-22 17:43   ` Paul E. McKenney
@ 2022-02-24  8:24     ` Paul Menzel
  2022-02-24 20:56       ` Paul E. McKenney
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Menzel @ 2022-02-24  8:24 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
	Shuah Khan, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

Dear Paul,


Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
> On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
>> For consecutive numbers *lscpu* collapses the output and just shows the
>> range with start and end. The processors are numbered that way on POWER8.
>>
>>      $ sudo ppc64_cpu --smt=8
>>      $ lscpu | grep '^NUMA node'
>>      NUMA node(s):                    2
>>      NUMA node0 CPU(s):               0-79
>>      NUMA node8 CPU(s):               80-159
>>
>> This causes the heuristic to detect the number threads per core, looking
>> for the number after the first comma, to fail, and QEMU aborts because of
>> invalid arguments.
>>
>>      $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>      $
>>
>> (Before the last patch, the whole line was returned.)
>>
>>      $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>>      NUMA node0 CPU(s):               0-79
>>
>> *lscpu* shows the number of threads per core, so use that value directly.
>>
>>      $ sudo ppc64_cpu --smt=8
>>      $ lscpu | grep 'Thread(s) per core'
>>      Thread(s) per core:              8
>>      $ sudo ppc64_cpu --smt=off
>>      $ lscpu | grep 'Thread(s) per core'
>>      Thread(s) per core:              1
>>
>> Note, the replaced heuristic is also incorrect for that case, where the
>> threads per core are disabled.
>>
>>      $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>      8
>>
>> Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> 
> Makes sense, and thank you for chasing this down and for the fix!
> 
> But should this patch and 1/2 be merged?  Or am I confused and they
> are somehow affecting two different lines of scripting?

You are right. I guess with 1/2 I just wanted to document clearly, what 
I learned in #sed@irc.libera.chat, that means, how to avoid using grep, 
when sed is used.


Kind regards,

Paul


>> ---
>>   tools/testing/selftests/rcutorture/bin/functions.sh | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
>> index 5cff520955e6..66d0414d8e4b 100644
>> --- a/tools/testing/selftests/rcutorture/bin/functions.sh
>> +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
>> @@ -301,7 +301,7 @@ specify_qemu_cpus () {
>>   			echo $2 -smp $3
>>   			;;
>>   		qemu-system-ppc64)
>> -			nt="`lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'`"
>> +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
>>   			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
>>   			;;
>>   		esac
>> -- 
>> 2.35.1
>>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-24  8:24     ` Paul Menzel
@ 2022-02-24 20:56       ` Paul E. McKenney
  2022-02-25  5:49         ` Paul Menzel
  0 siblings, 1 reply; 7+ messages in thread
From: Paul E. McKenney @ 2022-02-24 20:56 UTC (permalink / raw)
  To: Paul Menzel
  Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
	Shuah Khan, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:
> Dear Paul,
> 
> 
> Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
> > On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
> > > For consecutive numbers *lscpu* collapses the output and just shows the
> > > range with start and end. The processors are numbered that way on POWER8.
> > > 
> > >      $ sudo ppc64_cpu --smt=8
> > >      $ lscpu | grep '^NUMA node'
> > >      NUMA node(s):                    2
> > >      NUMA node0 CPU(s):               0-79
> > >      NUMA node8 CPU(s):               80-159
> > > 
> > > This causes the heuristic to detect the number threads per core, looking
> > > for the number after the first comma, to fail, and QEMU aborts because of
> > > invalid arguments.
> > > 
> > >      $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > >      $
> > > 
> > > (Before the last patch, the whole line was returned.)
> > > 
> > >      $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> > >      NUMA node0 CPU(s):               0-79
> > > 
> > > *lscpu* shows the number of threads per core, so use that value directly.
> > > 
> > >      $ sudo ppc64_cpu --smt=8
> > >      $ lscpu | grep 'Thread(s) per core'
> > >      Thread(s) per core:              8
> > >      $ sudo ppc64_cpu --smt=off
> > >      $ lscpu | grep 'Thread(s) per core'
> > >      Thread(s) per core:              1
> > > 
> > > Note, the replaced heuristic is also incorrect for that case, where the
> > > threads per core are disabled.
> > > 
> > >      $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > >      8
> > > 
> > > Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> > 
> > Makes sense, and thank you for chasing this down and for the fix!
> > 
> > But should this patch and 1/2 be merged?  Or am I confused and they
> > are somehow affecting two different lines of scripting?
> 
> You are right. I guess with 1/2 I just wanted to document clearly, what I
> learned in #sed@irc.libera.chat, that means, how to avoid using grep, when
> sed is used.

Nothing wrong with that!

I have merged the two patches as shown below.  Does this work for you?

							Thanx, Paul

------------------------------------------------------------------------

commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
Author: Paul Menzel <pmenzel@molgen.mpg.de>
Date:   Tue Feb 22 13:07:16 2022 +0100

    torture: Make thread detection more robust by using lspcu
    
    For consecutive numbers the lscpu command collapses the output and just
    shows the range with start and end. The processors are numbered that
    way on POWER8.
    
        $ sudo ppc64_cpu --smt=8
        $ lscpu | grep '^NUMA node'
        NUMA node(s):                    2
        NUMA node0 CPU(s):               0-79
        NUMA node8 CPU(s):               80-159
    
    This causes the heuristic to detect the number threads per core, looking
    for the number after the first comma, to fail, and QEMU aborts because of
    invalid arguments.
    
        $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
        NUMA node0 CPU(s):               0-79
    
    But the lscpu command shows the number of threads per core:
    
        $ sudo ppc64_cpu --smt=8
        $ lscpu | grep 'Thread(s) per core'
        Thread(s) per core:              8
        $ sudo ppc64_cpu --smt=off
        $ lscpu | grep 'Thread(s) per core'
        Thread(s) per core:              1
    
    This commit therefore directly uses that value.
    
    Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index c35ba24f994c3..66d0414d8e4bc 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -301,7 +301,7 @@ specify_qemu_cpus () {
 			echo $2 -smp $3
 			;;
 		qemu-system-ppc64)
-			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
+			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
 			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
 			;;
 		esac

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-24 20:56       ` Paul E. McKenney
@ 2022-02-25  5:49         ` Paul Menzel
  2022-02-25 20:45           ` Paul E. McKenney
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Menzel @ 2022-02-25  5:49 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
	Shuah Khan, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

Dear Paul,


Am 24.02.22 um 21:56 schrieb Paul E. McKenney:
> On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:

>> Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
>>> On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
>>>> For consecutive numbers *lscpu* collapses the output and just shows the
>>>> range with start and end. The processors are numbered that way on POWER8.
>>>>
>>>>       $ sudo ppc64_cpu --smt=8
>>>>       $ lscpu | grep '^NUMA node'
>>>>       NUMA node(s):                    2
>>>>       NUMA node0 CPU(s):               0-79
>>>>       NUMA node8 CPU(s):               80-159
>>>>
>>>> This causes the heuristic to detect the number threads per core, looking
>>>> for the number after the first comma, to fail, and QEMU aborts because of
>>>> invalid arguments.
>>>>
>>>>       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>>>       $
>>>>
>>>> (Before the last patch, the whole line was returned.)
>>>>
>>>>       $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>>>>       NUMA node0 CPU(s):               0-79
>>>>
>>>> *lscpu* shows the number of threads per core, so use that value directly.
>>>>
>>>>       $ sudo ppc64_cpu --smt=8
>>>>       $ lscpu | grep 'Thread(s) per core'
>>>>       Thread(s) per core:              8
>>>>       $ sudo ppc64_cpu --smt=off
>>>>       $ lscpu | grep 'Thread(s) per core'
>>>>       Thread(s) per core:              1
>>>>
>>>> Note, the replaced heuristic is also incorrect for that case, where the
>>>> threads per core are disabled.
>>>>
>>>>       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>>>       8
>>>>
>>>> Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
>>>
>>> Makes sense, and thank you for chasing this down and for the fix!
>>>
>>> But should this patch and 1/2 be merged?  Or am I confused and they
>>> are somehow affecting two different lines of scripting?
>>
>> You are right. I guess with 1/2 I just wanted to document clearly, what I
>> learned in #sed@irc.libera.chat, that means, how to avoid using grep, when
>> sed is used.
> 
> Nothing wrong with that!
> 
> I have merged the two patches as shown below.  Does this work for you?
> 
> 							Thanx, Paul
> 
> ------------------------------------------------------------------------
> 
> commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
> Author: Paul Menzel <pmenzel@molgen.mpg.de>
> Date:   Tue Feb 22 13:07:16 2022 +0100
> 
>      torture: Make thread detection more robust by using lspcu
>      
>      For consecutive numbers the lscpu command collapses the output and just
>      shows the range with start and end. The processors are numbered that
>      way on POWER8.
>      
>          $ sudo ppc64_cpu --smt=8
>          $ lscpu | grep '^NUMA node'
>          NUMA node(s):                    2
>          NUMA node0 CPU(s):               0-79
>          NUMA node8 CPU(s):               80-159
>      
>      This causes the heuristic to detect the number threads per core, looking
>      for the number after the first comma, to fail, and QEMU aborts because of
>      invalid arguments.
>      
>          $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>          NUMA node0 CPU(s):               0-79
>      
>      But the lscpu command shows the number of threads per core:
>      
>          $ sudo ppc64_cpu --smt=8
>          $ lscpu | grep 'Thread(s) per core'
>          Thread(s) per core:              8
>          $ sudo ppc64_cpu --smt=off
>          $ lscpu | grep 'Thread(s) per core'
>          Thread(s) per core:              1
>      
>      This commit therefore directly uses that value.

Maybe extend: …, and replaces `grep` by using using sed’s switch `-n` 
and the command p.

>      
>      Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
>      Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> 
> diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> index c35ba24f994c3..66d0414d8e4bc 100644
> --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> @@ -301,7 +301,7 @@ specify_qemu_cpus () {
>   			echo $2 -smp $3
>   			;;
>   		qemu-system-ppc64)
> -			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
> +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
>   			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
>   			;;
>   		esac

Thank you for doing that, and sorry for the extra work.


Kind regards,

Paul

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu
  2022-02-25  5:49         ` Paul Menzel
@ 2022-02-25 20:45           ` Paul E. McKenney
  0 siblings, 0 replies; 7+ messages in thread
From: Paul E. McKenney @ 2022-02-25 20:45 UTC (permalink / raw)
  To: Paul Menzel
  Cc: Josh Triplett, Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan,
	Shuah Khan, Zhouyi Zhou, rcu, linux-kselftest, linux-kernel

On Fri, Feb 25, 2022 at 06:49:32AM +0100, Paul Menzel wrote:
> Dear Paul,
> 
> 
> Am 24.02.22 um 21:56 schrieb Paul E. McKenney:
> > On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:
> 
> > > Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
> > > > On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
> > > > > For consecutive numbers *lscpu* collapses the output and just shows the
> > > > > range with start and end. The processors are numbered that way on POWER8.
> > > > > 
> > > > >       $ sudo ppc64_cpu --smt=8
> > > > >       $ lscpu | grep '^NUMA node'
> > > > >       NUMA node(s):                    2
> > > > >       NUMA node0 CPU(s):               0-79
> > > > >       NUMA node8 CPU(s):               80-159
> > > > > 
> > > > > This causes the heuristic to detect the number threads per core, looking
> > > > > for the number after the first comma, to fail, and QEMU aborts because of
> > > > > invalid arguments.
> > > > > 
> > > > >       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > >       $
> > > > > 
> > > > > (Before the last patch, the whole line was returned.)
> > > > > 
> > > > >       $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> > > > >       NUMA node0 CPU(s):               0-79
> > > > > 
> > > > > *lscpu* shows the number of threads per core, so use that value directly.
> > > > > 
> > > > >       $ sudo ppc64_cpu --smt=8
> > > > >       $ lscpu | grep 'Thread(s) per core'
> > > > >       Thread(s) per core:              8
> > > > >       $ sudo ppc64_cpu --smt=off
> > > > >       $ lscpu | grep 'Thread(s) per core'
> > > > >       Thread(s) per core:              1
> > > > > 
> > > > > Note, the replaced heuristic is also incorrect for that case, where the
> > > > > threads per core are disabled.
> > > > > 
> > > > >       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > >       8
> > > > > 
> > > > > Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> > > > 
> > > > Makes sense, and thank you for chasing this down and for the fix!
> > > > 
> > > > But should this patch and 1/2 be merged?  Or am I confused and they
> > > > are somehow affecting two different lines of scripting?
> > > 
> > > You are right. I guess with 1/2 I just wanted to document clearly, what I
> > > learned in #sed@irc.libera.chat, that means, how to avoid using grep, when
> > > sed is used.
> > 
> > Nothing wrong with that!
> > 
> > I have merged the two patches as shown below.  Does this work for you?
> > 
> > 							Thanx, Paul
> > 
> > ------------------------------------------------------------------------
> > 
> > commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
> > Author: Paul Menzel <pmenzel@molgen.mpg.de>
> > Date:   Tue Feb 22 13:07:16 2022 +0100
> > 
> >      torture: Make thread detection more robust by using lspcu
> >      For consecutive numbers the lscpu command collapses the output and just
> >      shows the range with start and end. The processors are numbered that
> >      way on POWER8.
> >          $ sudo ppc64_cpu --smt=8
> >          $ lscpu | grep '^NUMA node'
> >          NUMA node(s):                    2
> >          NUMA node0 CPU(s):               0-79
> >          NUMA node8 CPU(s):               80-159
> >      This causes the heuristic to detect the number threads per core, looking
> >      for the number after the first comma, to fail, and QEMU aborts because of
> >      invalid arguments.
> >          $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> >          NUMA node0 CPU(s):               0-79
> >      But the lscpu command shows the number of threads per core:
> >          $ sudo ppc64_cpu --smt=8
> >          $ lscpu | grep 'Thread(s) per core'
> >          Thread(s) per core:              8
> >          $ sudo ppc64_cpu --smt=off
> >          $ lscpu | grep 'Thread(s) per core'
> >          Thread(s) per core:              1
> >      This commit therefore directly uses that value.
> 
> Maybe extend: …, and replaces `grep` by using using sed’s switch `-n` and
> the command p.

Ah, good point -- I will update this on the next rebase.

> >      Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> >      Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> > 
> > diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> > index c35ba24f994c3..66d0414d8e4bc 100644
> > --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> > @@ -301,7 +301,7 @@ specify_qemu_cpus () {
> >   			echo $2 -smp $3
> >   			;;
> >   		qemu-system-ppc64)
> > -			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
> > +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
> >   			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
> >   			;;
> >   		esac
> 
> Thank you for doing that, and sorry for the extra work.

No complaints!  You are after all testing on powerpc, and getting that
working better is a good thing.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2022-02-25 20:45 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-22 12:07 [PATCH 1/2] torture: Select line in sed and replace grep Paul Menzel
2022-02-22 12:07 ` [PATCH 2/2] torture: Make thread detection more robust by using lspcu Paul Menzel
2022-02-22 17:43   ` Paul E. McKenney
2022-02-24  8:24     ` Paul Menzel
2022-02-24 20:56       ` Paul E. McKenney
2022-02-25  5:49         ` Paul Menzel
2022-02-25 20:45           ` Paul E. McKenney

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).