git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"René Scharfe" <l.s.r@web.de>, "Jeff King" <peff@peff.net>,
	"SZEDER Gábor" <szeder.dev@gmail.com>, "Eric Wong" <e@80x24.org>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v4 3/4] Makefile/coccicheck: allow for setting xargs concurrency
Date: Mon, 22 Mar 2021 13:11:49 +0100	[thread overview]
Message-ID: <9d5814dacdc281389c4cb163ddbe4b749e6c0852.1616414951.git.avarab@gmail.com> (raw)
In-Reply-To: <cover.1616414951.git.avarab@gmail.com>

Extend the SPATCH_BATCH_SIZE facility added in
960154b9c17 (coccicheck: optionally batch spatch invocations,
2019-05-06) and bcb4edf7af7 (coccicheck: make batch size of 0 mean
"unlimited", 2019-05-08) to allow for both setting
SPATCH_BATCH_SIZE=N, and also to have a more advanced SPATCH_XARGS
argument.

The reason to replace the "$$limit" is that now you actually see under
V=1 what argument your program will get invoked with.

The reason for the "9999" limit is that if you e.g. try to define an
"$(XARGS)" which is conditionally an empty string or not depending on
this setting then e.g.:

    echo $(FILES) | $(XARGS) $(XARGS_FLAGS) $(SPATCH)

Over multiple lines with "\" will error out. I think just setting it
to "xargs -n 9999" as a trivial workaround is the best solution here.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index e43a9618df5..eaac14275bb 100644
--- a/Makefile
+++ b/Makefile
@@ -1206,6 +1206,25 @@ SPATCH_FLAGS = --all-includes --include-headers-for-types --patch .
 # Setting it to 0 will feed all files in a single spatch invocation.
 SPATCH_BATCH_SIZE = 1
 
+# For the 'coccicheck' target; SPATCH_XARGS can be used to manually
+# tweak the xargs invocation. By default we invoke "xargs -n 1", and
+# "xargs -n 9999" under SPATCH_BATCH_SIZE=0.
+#
+# Setting SPATCH_XARGS overrides SPATCH_BATCH_SIZE. To get concurrency
+# when targeting a single contrib/coccinelle/%.patch use e.g. "-P" if
+# your xargs(1) supports it:
+#
+#    make contrib/coccinelle/strbuf.cocci.patch SPATCH_XARGS="xargs -P 8 -n 8"
+#
+# Or a combination -jN and "xargs -P":
+#
+#    make -j4 coccicheck SPATCH_XARGS="xargs -P 2 -n 8"
+ifeq (0,$(SPATCH_BATCH_SIZE))
+SPATCH_XARGS = xargs -n 9999
+else
+SPATCH_XARGS = xargs -n $(SPATCH_BATCH_SIZE)
+endif
+
 include config.mak.uname
 -include config.mak.autogen
 -include config.mak
@@ -2866,12 +2885,7 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))
 
 %.cocci.patch: %.cocci $(COCCI_SOURCES)
 	$(QUIET_SPATCH) \
-	if test $(SPATCH_BATCH_SIZE) = 0; then \
-		limit=; \
-	else \
-		limit='-n $(SPATCH_BATCH_SIZE)'; \
-	fi; \
-	if ! echo $(COCCI_SOURCES) | xargs $$limit \
+	if ! echo $(COCCI_SOURCES) | $(SPATCH_XARGS) \
 		$(SPATCH) --sp-file $< $(SPATCH_FLAGS) \
 		>$@+ 2>$@.log; \
 	then \
-- 
2.31.0.285.gb40d23e604f


  parent reply	other threads:[~2021-03-22 12:13 UTC|newest]

Thread overview: 103+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-12 23:48 [RFC PATCH] *.h: remove extern from function declarations Denton Liu
2019-04-13  1:24 ` Jeff King
2019-04-13  5:45   ` Junio C Hamano
2019-04-15 18:24 ` [PATCH v2 0/3] " Denton Liu
2019-04-15 18:24   ` [PATCH v2 1/3] *.[ch]: remove extern from function declarations using spatch Denton Liu
2019-04-15 19:19     ` Thomas Gummerer
2019-04-15 18:24   ` [PATCH v2 2/3] *.[ch]: remove extern from function declarations using sed Denton Liu
2019-04-15 18:24   ` [PATCH v2 3/3] cocci: prevent extern function declarations Denton Liu
2019-04-17  7:58   ` [PATCH v3 0/4] remove extern from " Denton Liu
2019-04-17  7:58     ` [PATCH v3 1/4] *.[ch]: remove extern from function declarations using spatch Denton Liu
2019-04-17  7:58     ` [PATCH v3 2/4] *.[ch]: remove extern from function declarations using sed Denton Liu
2019-04-17  7:58     ` [PATCH v3 3/4] *.[ch]: manually align parameter lists Denton Liu
2019-04-17  7:58     ` [PATCH v3 4/4] cocci: prevent extern function declarations Denton Liu
2019-04-22  5:44     ` [PATCH] cache.h: fix mismerge of 'dl/no-extern-in-func-decl' Denton Liu
2019-04-22  6:30       ` Junio C Hamano
2019-04-22 11:19         ` Junio C Hamano
2019-04-22 21:49     ` [PATCH v3 0/4] remove extern from function declarations Jeff King
2019-04-25 12:07       ` SZEDER Gábor
2019-04-25 18:05         ` Denton Liu
2019-04-30 23:21         ` Johannes Schindelin
2019-05-01 10:01           ` Denton Liu
2019-05-01 18:56             ` Jeff King
2019-05-02  0:04             ` SZEDER Gábor
2019-05-03  9:32               ` Johannes Schindelin
2019-05-03 14:42                 ` SZEDER Gábor
2019-05-03 14:58                   ` SZEDER Gábor
2019-05-03 17:45                   ` Jeff King
2019-05-03 18:44                     ` SZEDER Gábor
2019-05-05  5:28                     ` Junio C Hamano
2019-05-05 18:09                       ` Jacob Keller
2019-05-05 18:08                     ` Jacob Keller
2019-05-06  5:11                       ` [PATCH] coccicheck: optionally process every source file at once Jeff King
2019-05-06  9:34                         ` Duy Nguyen
2019-05-06 23:43                           ` [PATCH] coccicheck: optionally batch spatch invocations Jeff King
2019-05-07  1:41                             ` Jacob Keller
2019-05-07  2:04                               ` Jeff King
2019-05-07  2:42                             ` Junio C Hamano
2019-05-07  2:55                               ` Jeff King
2019-05-07  3:04                                 ` Jacob Keller
2019-05-07  4:52                                 ` Junio C Hamano
2019-05-08  7:07                                   ` Jeff King
2019-05-08 12:36                                     ` Denton Liu
2019-05-08 22:39                                       ` Jeff King
2019-05-07 10:20                             ` Duy Nguyen
2019-05-07 11:19                             ` SZEDER Gábor
2021-03-02 20:51                             ` [PATCH] Makefile: fix bugs in coccicheck and speed it up Ævar Arnfjörð Bjarmason
2021-03-03  9:43                               ` Denton Liu
2021-03-03 11:45                               ` Ævar Arnfjörð Bjarmason
2021-03-04 23:18                               ` Junio C Hamano
2021-03-05 11:17                                 ` Ævar Arnfjörð Bjarmason
2021-03-05 10:24                               ` Jeff King
2021-03-05 17:20                                 ` Ævar Arnfjörð Bjarmason
2021-03-06 10:59                                   ` Jeff King
2021-03-05 17:07                               ` [PATCH v2 0/4] Makefile/coccicheck: fix bugs " Ævar Arnfjörð Bjarmason
2021-03-05 19:10                                 ` René Scharfe.
     [not found]                                   ` <xmqqim659u57.fsf@gitster.c.googlers.com>
2021-03-06 11:26                                     ` René Scharfe.
2021-03-06 12:43                                       ` René Scharfe.
     [not found]                                       ` <xmqqft16914r.fsf@gitster.c.googlers.com>
2021-03-13 16:10                                         ` René Scharfe.
2021-03-06 17:27                                   ` Ævar Arnfjörð Bjarmason
2021-03-06 17:41                                     ` René Scharfe.
2021-03-06 17:52                                       ` Ævar Arnfjörð Bjarmason
2021-03-06 19:08                                         ` René Scharfe.
2021-03-05 17:07                               ` [PATCH v2 1/4] Makefile/coccicheck: add comment heading for all SPATCH flags Ævar Arnfjörð Bjarmason
2021-03-05 17:07                               ` [PATCH v2 2/4] Makefile/coccicheck: speed up and fix bug with duplicate hunks Ævar Arnfjörð Bjarmason
2021-03-06 10:45                                 ` Jeff King
2021-03-06 19:29                                   ` Ævar Arnfjörð Bjarmason
2021-03-05 17:07                               ` [PATCH v2 3/4] Makefile/coccicheck: allow for setting xargs concurrency Ævar Arnfjörð Bjarmason
2021-03-06 10:51                                 ` Jeff King
2021-03-05 17:07                               ` [PATCH v2 4/4] Makefile/coccicheck: set SPATCH_BATCH_SIZE to 8 Ævar Arnfjörð Bjarmason
2021-03-06 19:25                                 ` [PATCH v2 5/4] Makefile/coccicheck: use --include-headers-for-types Ævar Arnfjörð Bjarmason
2021-03-18 20:49                                   ` SZEDER Gábor
2021-03-19 10:32                                     ` Ævar Arnfjörð Bjarmason
2021-03-22 12:11                                   ` [PATCH v4 0/4] Makefile/coccicheck: fix bugs and speed it up Ævar Arnfjörð Bjarmason
2021-03-22 12:11                                     ` [PATCH v4 1/4] Makefile/coccicheck: add comment heading for all SPATCH flags Ævar Arnfjörð Bjarmason
2021-03-22 18:04                                       ` René Scharfe.
2021-03-22 12:11                                     ` [PATCH v4 2/4] Makefile/coccicheck: speed up and fix bug with duplicate hunks Ævar Arnfjörð Bjarmason
2021-03-22 18:05                                       ` René Scharfe.
2021-03-24 19:19                                         ` Jeff King
2021-03-22 19:09                                       ` Junio C Hamano
2021-03-22 12:11                                     ` Ævar Arnfjörð Bjarmason [this message]
2021-03-24 19:26                                       ` [PATCH v4 3/4] Makefile/coccicheck: allow for setting xargs concurrency Jeff King
2021-03-25  2:29                                         ` Ævar Arnfjörð Bjarmason
2021-03-26  4:11                                           ` Jeff King
2021-03-22 12:11                                     ` [PATCH v4 4/4] Makefile/coccicheck: set SPATCH_BATCH_SIZE to 8 Ævar Arnfjörð Bjarmason
2021-03-22 18:05                                       ` René Scharfe.
2021-03-24 19:27                                         ` Jeff King
2021-03-27 17:43                                     ` [PATCH v4 0/4] Makefile/coccicheck: fix bugs and speed it up Junio C Hamano
2021-03-27 19:46                                       ` Ævar Arnfjörð Bjarmason
2019-05-03  9:40               ` [PATCH v3 0/4] remove extern from function declarations Denton Liu
2019-04-23 23:40   ` [PATCH v4 " Denton Liu
2019-04-23 23:40     ` [PATCH v4 1/4] *.[ch]: remove extern from function declarations using spatch Denton Liu
2019-04-23 23:40     ` [PATCH v4 2/4] *.[ch]: remove extern from function declarations using sed Denton Liu
2019-04-24  4:56       ` Junio C Hamano
2019-04-25 19:00         ` Denton Liu
2019-04-23 23:40     ` [PATCH v4 3/4] *.[ch]: manually align parameter lists Denton Liu
2019-04-23 23:40     ` [PATCH v4 4/4] cocci: prevent extern function declarations Denton Liu
2019-04-29  8:28     ` [PATCH v5 0/3] *** SUBJECT HERE *** Denton Liu
2019-04-29  8:28       ` [PATCH v5 1/3] *.[ch]: remove extern from function declarations using spatch Denton Liu
2019-04-29  8:28       ` [PATCH v5 2/3] *.[ch]: remove extern from function declarations using sed Denton Liu
2019-04-29  8:28       ` [PATCH v5 3/3] *.[ch]: manually align parameter lists Denton Liu
2019-04-29  8:30       ` [PATCH v5 0/3] *** SUBJECT HERE *** Denton Liu
2019-05-06 11:03         ` Ævar Arnfjörð Bjarmason
2019-05-06 15:34           ` Denton Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9d5814dacdc281389c4cb163ddbe4b749e6c0852.1616414951.git.avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=l.s.r@web.de \
    --cc=peff@peff.net \
    --cc=szeder.dev@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).