[PATCH v4 0/3] Improve Coccinelle Parallelisation

[PATCH v4 0/3] Improve Coccinelle Parallelisation

[Sumera Priyadarsini]

Presently, Coccinelle uses at most one thread per core to improve
performance in machines with more than 2 hyperthreads. Modify
coccicheck to use all available threads in machines upto 4 hyperthreads.
Further, modify the coccicheck script to improve portability.

Modify documentation to reflect the same. 

Sumera Priyadarsini (3):
  scripts: coccicheck: Add quotes to improve portability
  scripts: coccicheck: Change default condition for parallelism
  Documentation: Coccinelle: Modify Parallelisation information in docs

 Documentation/dev-tools/coccinelle.rst | 6 ++++--
 scripts/coccicheck                     | 4 ++--
 2 files changed, 6 insertions(+), 4 deletions(-)

-- 
2.25.1

[PATCH v4 1/3] scripts: coccicheck: Add quotes to improve portability

[Sumera Priyadarsini]

While fetching the number of threads per core with lscpu,
the [:digit:] set is used for translation of digits from 0-9.
However, using [:digit:] instead of "[:digit:]" does not seem
to work uniformly for some shell types and configurations
(such as zsh).

Therefore, modify coccicheck to use double quotes around the
[:digit:] set for uniformity and better portability.

Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
---
 scripts/coccicheck | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/coccicheck b/scripts/coccicheck
index 6789751607f5..d67907b8a38b 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -76,7 +76,7 @@ else
     fi
 
     # Use only one thread per core by default if hyperthreading is enabled
-    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
+    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
 	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
-- 
2.25.1

[PATCH v4 2/3] scripts: coccicheck: Change default condition for parallelism

[Sumera Priyadarsini]

Currently, Coccinelle uses at most one thread per core by default in
machines with more than 2 hyperthreads. However, for systems with only 4
hyperthreads, this does not improve performance.

Modify coccicheck to use all available threads in machines with
upto 4 hyperthreads.

Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
---
 scripts/coccicheck | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/coccicheck b/scripts/coccicheck
index d67907b8a38b..209bb0427b43 100755
--- a/scripts/coccicheck
+++ b/scripts/coccicheck
@@ -79,7 +79,7 @@ else
     THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
-	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 4 ] ; then
 		NPROC=$((NPROC/2))
 	fi
     else
-- 
2.25.1

[PATCH v4 3/3] Documentation: Coccinelle: Modify Parallelisation information in docs

[Sumera Priyadarsini]

This patchset modifies coccicheck to use at most one thread per core by
default in machines with more than 4 hyperthreads for optimal performance.
Modify documentation in coccinelle.rst to reflect the same.

Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>
---
 Documentation/dev-tools/coccinelle.rst | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/Documentation/dev-tools/coccinelle.rst b/Documentation/dev-tools/coccinelle.rst
index 74c5e6aeeff5..530d8d313601 100644
--- a/Documentation/dev-tools/coccinelle.rst
+++ b/Documentation/dev-tools/coccinelle.rst
@@ -130,8 +130,10 @@ To enable verbose messages set the V= variable, for example::
 Coccinelle parallelization
 --------------------------
 
-By default, coccicheck tries to run as parallel as possible. To change
-the parallelism, set the J= variable. For example, to run across 4 CPUs::
+By default, coccicheck uses at most 1 thread per core in a machine
+with more than 4 hyperthreads. In a machine with upto 4 threads,
+all threads are used. To change the parallelism, set the J= variable.
+For example, to run across 4 CPUs::
 
    make coccicheck MODE=report J=4
 
-- 
2.25.1

Re: [Cocci] [PATCH v4 2/3] scripts: coccicheck: Change default condition for parallelism

[Julia Lawall]

On Sun, 11 Oct 2020, Sumera Priyadarsini wrote:

> Currently, Coccinelle uses at most one thread per core by default in
> machines with more than 2 hyperthreads. However, for systems with only 4
> hyperthreads, this does not improve performance.
>
> Modify coccicheck to use all available threads in machines with
> upto 4 hyperthreads.
>
> Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>

Applied, thanks.

> ---
>  scripts/coccicheck | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/coccicheck b/scripts/coccicheck
> index d67907b8a38b..209bb0427b43 100755
> --- a/scripts/coccicheck
> +++ b/scripts/coccicheck
> @@ -79,7 +79,7 @@ else
>      THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")
>      if [ -z "$J" ]; then
>          NPROC=$(getconf _NPROCESSORS_ONLN)
> -	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
> +	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 4 ] ; then
>  		NPROC=$((NPROC/2))
>  	fi
>      else
> --
> 2.25.1
>
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

Re: [Cocci] [PATCH v4 1/3] scripts: coccicheck: Add quotes to improve portability

[Julia Lawall]

On Sun, 11 Oct 2020, Sumera Priyadarsini wrote:

> While fetching the number of threads per core with lscpu,
> the [:digit:] set is used for translation of digits from 0-9.
> However, using [:digit:] instead of "[:digit:]" does not seem
> to work uniformly for some shell types and configurations
> (such as zsh).
>
> Therefore, modify coccicheck to use double quotes around the
> [:digit:] set for uniformity and better portability.
>
> Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>

Applied, thanks.

> ---
>  scripts/coccicheck | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/coccicheck b/scripts/coccicheck
> index 6789751607f5..d67907b8a38b 100755
> --- a/scripts/coccicheck
> +++ b/scripts/coccicheck
> @@ -76,7 +76,7 @@ else
>      fi
>
>      # Use only one thread per core by default if hyperthreading is enabled
> -    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
> +    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")
>      if [ -z "$J" ]; then
>          NPROC=$(getconf _NPROCESSORS_ONLN)
>  	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
> --
> 2.25.1
>
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

Re: [Cocci] [PATCH v4 3/3] Documentation: Coccinelle: Modify Parallelisation information in docs

[Julia Lawall]

On Sun, 11 Oct 2020, Sumera Priyadarsini wrote:

> This patchset modifies coccicheck to use at most one thread per core by
> default in machines with more than 4 hyperthreads for optimal performance.
> Modify documentation in coccinelle.rst to reflect the same.
>
> Signed-off-by: Sumera Priyadarsini <sylphrenadin@gmail.com>

Acked-by: Julia Lawall <julia.lawall@inria.fr>

Jonathan, will you take this patch?

thanks,
julia

> ---
>  Documentation/dev-tools/coccinelle.rst | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/dev-tools/coccinelle.rst b/Documentation/dev-tools/coccinelle.rst
> index 74c5e6aeeff5..530d8d313601 100644
> --- a/Documentation/dev-tools/coccinelle.rst
> +++ b/Documentation/dev-tools/coccinelle.rst
> @@ -130,8 +130,10 @@ To enable verbose messages set the V= variable, for example::
>  Coccinelle parallelization
>  --------------------------
>
> -By default, coccicheck tries to run as parallel as possible. To change
> -the parallelism, set the J= variable. For example, to run across 4 CPUs::
> +By default, coccicheck uses at most 1 thread per core in a machine
> +with more than 4 hyperthreads. In a machine with upto 4 threads,
> +all threads are used. To change the parallelism, set the J= variable.
> +For example, to run across 4 CPUs::
>
>     make coccicheck MODE=report J=4
>
> --
> 2.25.1
>
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>