* [Buildroot] [PATCH 0/1] Fix libnss parallel build failure
@ 2021-09-21 23:59 Giulio Benetti
2021-09-21 23:59 ` [Buildroot] [PATCH 1/1] package/libnss: fix " Giulio Benetti
2021-09-23 16:50 ` [Buildroot] [PATCH 0/1] Fix libnss " Giulio Benetti
0 siblings, 2 replies; 9+ messages in thread
From: Giulio Benetti @ 2021-09-21 23:59 UTC (permalink / raw)
To: buildroot; +Cc: Joseph Kogut, Giulio Benetti
I ask to accept this patch even if I couldn't test it. Unfortunately I
can't reproduce the build failure but as explained in commit log that
should be the way. Only this libnss parallel build bug is left and it's
only a bunch of symlink creation, it's not compiling/linking related.
Any further suggestion is accepted anyway.
Best regards
Giulio Benetti (1):
package/libnss: fix parallel build failure
...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
--
2.25.1
_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-09-21 23:59 [Buildroot] [PATCH 0/1] Fix libnss parallel build failure Giulio Benetti
@ 2021-09-21 23:59 ` Giulio Benetti
2021-10-06 19:20 ` Arnout Vandecappelle
2021-09-23 16:50 ` [Buildroot] [PATCH 0/1] Fix libnss " Giulio Benetti
1 sibling, 1 reply; 9+ messages in thread
From: Giulio Benetti @ 2021-09-21 23:59 UTC (permalink / raw)
To: buildroot; +Cc: Joseph Kogut, Giulio Benetti
As explained in the patch itself(pending here[1]), I couldn't reproduce
the failure but it seems to be right. Basically I'm making not parallel
every 'nsinstall -R' command that is where it fails.
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
Fixes:
http://autobuild.buildroot.net/results/a895ed7aa48edc5cd9ac8bbbd689433a98c9c057/
Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
---
...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
diff --git a/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
new file mode 100644
index 0000000000..7ac340f1ca
--- /dev/null
+++ b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
@@ -0,0 +1,34 @@
+From 7ba4d627212507f876dc112e89c68e82e1c50188 Mon Sep 17 00:00:00 2001
+From: Giulio Benetti <giulio.benetti@benettiengineering.com>
+Date: Wed, 22 Sep 2021 01:40:31 +0200
+Subject: [PATCH] Bug 1731911 - Fix nsinstall parallel failure
+
+Fix nsinstall race condition when building using Makefile with parallel
+options on. I've end up making $(1) recipe in copy_varlist_into_dir_RULE
+.NOTPARALLEL. I can't test it that much since it's very very rare, but
+it seems to be the correct way. This way every copy_varlist_into_dir_RULE
+called should occur not parallel and I think that this is acceptable
+since it only create symlinks that takes very few. Further improving is
+possible of course.
+
+Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
+---
+ nss/coreconf/rules.mk | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/nss/coreconf/rules.mk b/nss/coreconf/rules.mk
+index dc72e9cef..a341b8385 100644
+--- a/nss/coreconf/rules.mk
++++ b/nss/coreconf/rules.mk
+@@ -377,6 +377,8 @@ $(3)/%: %
+ $$(INSTALL) -m 444 $$^ $(3)
+
+ $(1): $$(addprefix $(3)/,$$($(2))) | $(3)/d
++
++.NOTPARALLEL: $(1)
+ endif
+ else
+ $(1):
+--
+2.25.1
+
--
2.25.1
_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 0/1] Fix libnss parallel build failure
2021-09-21 23:59 [Buildroot] [PATCH 0/1] Fix libnss parallel build failure Giulio Benetti
2021-09-21 23:59 ` [Buildroot] [PATCH 1/1] package/libnss: fix " Giulio Benetti
@ 2021-09-23 16:50 ` Giulio Benetti
2021-10-04 23:22 ` Giulio Benetti
1 sibling, 1 reply; 9+ messages in thread
From: Giulio Benetti @ 2021-09-23 16:50 UTC (permalink / raw)
To: buildroot; +Cc: Joseph Kogut
Hi All,
On 9/22/21 1:59 AM, Giulio Benetti wrote:
> I ask to accept this patch even if I couldn't test it. Unfortunately I
> can't reproduce the build failure but as explained in commit log that
> should be the way. Only this libnss parallel build bug is left and it's
> only a bunch of symlink creation, it's not compiling/linking related.
> Any further suggestion is accepted anyway.
libnss is building fine since 3 days while using the same machine for
building other heavy stuff or for doing nothing than compiling libnss,
so it seems promising.
Best regards
--
Giulio Benetti
Benetti Engineering sas
> Best regards
>
> Giulio Benetti (1):
> package/libnss: fix parallel build failure
>
> ...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
> 1 file changed, 34 insertions(+)
> create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>
_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 0/1] Fix libnss parallel build failure
2021-09-23 16:50 ` [Buildroot] [PATCH 0/1] Fix libnss " Giulio Benetti
@ 2021-10-04 23:22 ` Giulio Benetti
0 siblings, 0 replies; 9+ messages in thread
From: Giulio Benetti @ 2021-10-04 23:22 UTC (permalink / raw)
To: buildroot; +Cc: Joseph Kogut
On 9/23/21 6:50 PM, Giulio Benetti wrote:
> Hi All,
>
> On 9/22/21 1:59 AM, Giulio Benetti wrote:
>> I ask to accept this patch even if I couldn't test it. Unfortunately I
>> can't reproduce the build failure but as explained in commit log that
>> should be the way. Only this libnss parallel build bug is left and it's
>> only a bunch of symlink creation, it's not compiling/linking related.
>> Any further suggestion is accepted anyway.
>
> libnss is building fine since 3 days while using the same machine for
> building other heavy stuff or for doing nothing than compiling libnss,
> so it seems promising.
It built for 10 days with no problems, so kindly ping because it still
sporadically fails.
Best regards
--
Giulio Benetti
Benetti Engineering sas
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-09-21 23:59 ` [Buildroot] [PATCH 1/1] package/libnss: fix " Giulio Benetti
@ 2021-10-06 19:20 ` Arnout Vandecappelle
2021-10-19 17:33 ` Giulio Benetti
0 siblings, 1 reply; 9+ messages in thread
From: Arnout Vandecappelle @ 2021-10-06 19:20 UTC (permalink / raw)
To: Giulio Benetti, buildroot; +Cc: Joseph Kogut
On 22/09/2021 01:59, Giulio Benetti wrote:
> As explained in the patch itself(pending here[1]), I couldn't reproduce
> the failure but it seems to be right. Basically I'm making not parallel
> every 'nsinstall -R' command that is where it fails.
>
> [1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>
> Fixes:
> http://autobuild.buildroot.net/results/a895ed7aa48edc5cd9ac8bbbd689433a98c9c057/
>
> Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
> ---
> ...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
> 1 file changed, 34 insertions(+)
> create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>
> diff --git a/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
> new file mode 100644
> index 0000000000..7ac340f1ca
> --- /dev/null
> +++ b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
> @@ -0,0 +1,34 @@
> +From 7ba4d627212507f876dc112e89c68e82e1c50188 Mon Sep 17 00:00:00 2001
> +From: Giulio Benetti <giulio.benetti@benettiengineering.com>
> +Date: Wed, 22 Sep 2021 01:40:31 +0200
> +Subject: [PATCH] Bug 1731911 - Fix nsinstall parallel failure
> +
> +Fix nsinstall race condition when building using Makefile with parallel
> +options on. I've end up making $(1) recipe in copy_varlist_into_dir_RULE
> +.NOTPARALLEL. I can't test it that much since it's very very rare, but
> +it seems to be the correct way. This way every copy_varlist_into_dir_RULE
> +called should occur not parallel and I think that this is acceptable
> +since it only create symlinks that takes very few. Further improving is
> +possible of course.
> +
> +Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
> +---
> + nss/coreconf/rules.mk | 2 ++
> + 1 file changed, 2 insertions(+)
> +
> +diff --git a/nss/coreconf/rules.mk b/nss/coreconf/rules.mk
> +index dc72e9cef..a341b8385 100644
> +--- a/nss/coreconf/rules.mk
> ++++ b/nss/coreconf/rules.mk
> +@@ -377,6 +377,8 @@ $(3)/%: %
> + $$(INSTALL) -m 444 $$^ $(3)
> +
> + $(1): $$(addprefix $(3)/,$$($(2))) | $(3)/d
> ++
> ++.NOTPARALLEL: $(1)
This doesn't make a whole lot of sense: .NOTPARALLEL is something that applies
to the Makefile as a whole, not to one specific target. I haven't looked in any
depth in the libnss build system, but unless it builds recursively, adding this
is exactly equivalent to adding
.NOTPARALLEL:
anywhere in the Makefiles. This is BTW what we do in Buildroot.
And in the end, that's the same as calling $(MAKE1) from Buildroot, so
patching is not really needed.
If you want to be a little smarter, you will have to manually "unroll" the
dependency, like:
$(1): | $(3)/d
$$(foreach tgt,$(2),$(MAKE) $(3)/$$($(tgt)))
(chances are I made some major mistake in this thing).
Or if it does build recursively, you can probably conditionally add
.NOTPARALLEL in some smart way.
Anyway, I've marked this patch as Changed Requested.
Regards,
Arnout
> + endif
> + else
> + $(1):
> +--
> +2.25.1
> +
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-10-06 19:20 ` Arnout Vandecappelle
@ 2021-10-19 17:33 ` Giulio Benetti
2021-10-19 17:59 ` Giulio Benetti
0 siblings, 1 reply; 9+ messages in thread
From: Giulio Benetti @ 2021-10-19 17:33 UTC (permalink / raw)
To: Arnout Vandecappelle; +Cc: Joseph Kogut, buildroot
[-- Attachment #1.1: Type: text/plain, Size: 4442 bytes --]
Hi Arnout,
> Il giorno 6 ott 2021, alle ore 21:20, Arnout Vandecappelle <arnout@mind.be> ha scritto:
>
>
>
>> On 22/09/2021 01:59, Giulio Benetti wrote:
>> As explained in the patch itself(pending here[1]), I couldn't reproduce
>> the failure but it seems to be right. Basically I'm making not parallel
>> every 'nsinstall -R' command that is where it fails.
>> [1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>> Fixes:
>> http://autobuild.buildroot.net/results/a895ed7aa48edc5cd9ac8bbbd689433a98c9c057/
>> Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
>> ---
>> ...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
>> 1 file changed, 34 insertions(+)
>> create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>> diff --git a/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>> new file mode 100644
>> index 0000000000..7ac340f1ca
>> --- /dev/null
>> +++ b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>> @@ -0,0 +1,34 @@
>> +From 7ba4d627212507f876dc112e89c68e82e1c50188 Mon Sep 17 00:00:00 2001
>> +From: Giulio Benetti <giulio.benetti@benettiengineering.com>
>> +Date: Wed, 22 Sep 2021 01:40:31 +0200
>> +Subject: [PATCH] Bug 1731911 - Fix nsinstall parallel failure
>> +
>> +Fix nsinstall race condition when building using Makefile with parallel
>> +options on. I've end up making $(1) recipe in copy_varlist_into_dir_RULE
>> +.NOTPARALLEL. I can't test it that much since it's very very rare, but
>> +it seems to be the correct way. This way every copy_varlist_into_dir_RULE
>> +called should occur not parallel and I think that this is acceptable
>> +since it only create symlinks that takes very few. Further improving is
>> +possible of course.
>> +
>> +Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
>> +---
>> + nss/coreconf/rules.mk | 2 ++
>> + 1 file changed, 2 insertions(+)
>> +
>> +diff --git a/nss/coreconf/rules.mk b/nss/coreconf/rules.mk
>> +index dc72e9cef..a341b8385 100644
>> +--- a/nss/coreconf/rules.mk
>> ++++ b/nss/coreconf/rules.mk
>> +@@ -377,6 +377,8 @@ $(3)/%: %
>> + $$(INSTALL) -m 444 $$^ $(3)
>> +
>> + $(1): $$(addprefix $(3)/,$$($(2))) | $(3)/d
>> ++
>> ++.NOTPARALLEL: $(1)
>
> This doesn't make a whole lot of sense: .NOTPARALLEL is something that applies to the Makefile as a whole, not to one specific target. I haven't looked in any depth in the libnss build system, but unless it builds recursively, adding this is exactly equivalent to adding
>
> .NOTPARALLEL:
As I’ve tested it’s not like this.
I read this from [1]:
‘’’
.NOTPARALLEL
If .NOTPARALLEL is mentioned as a target, then this invocation of make will be run serially, even if the ‘-j’ option is given. Any recursively invoked make command will still run recipes in parallel (unless its makefile also contains this target). Any prerequisites on this target are ignored.
‘’’
They talk about target and it is what I experience.
It’s a bit ambiguous honestly. In the past I’ve also
experienced it on a big Makefile where I had .bmp files to be converted with xxd and that gave parallel problems. So I’ve tagged it like:
.NOTPARALLEL: %.bmp
and that solved the problem.
Also they have just upstreamed the patch [2], so
we can test if it works committing this patch or
by bumping next version.
Then it’s up to you :-)
Best regards
Giulio Benetti
Benetti Engineering sas
[1]: https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
[2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>
> anywhere in the Makefiles. This is BTW what we do in Buildroot.
>
> And in the end, that's the same as calling $(MAKE1) from Buildroot, so patching is not really needed.
>
> If you want to be a little smarter, you will have to manually "unroll" the dependency, like:
>
> $(1): | $(3)/d
> $$(foreach tgt,$(2),$(MAKE) $(3)/$$($(tgt)))
>
> (chances are I made some major mistake in this thing).
>
> Or if it does build recursively, you can probably conditionally add .NOTPARALLEL in some smart way.
>
> Anyway, I've marked this patch as Changed Requested.
>
> Regards,
> Arnout
>
>
>> + endif
>> + else
>> + $(1):
>> +--
>> +2.25.1
>> +
[-- Attachment #1.2: Type: text/html, Size: 8994 bytes --]
[-- Attachment #2: Type: text/plain, Size: 150 bytes --]
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-10-19 17:33 ` Giulio Benetti
@ 2021-10-19 17:59 ` Giulio Benetti
2021-10-19 19:01 ` Arnout Vandecappelle
0 siblings, 1 reply; 9+ messages in thread
From: Giulio Benetti @ 2021-10-19 17:59 UTC (permalink / raw)
To: Arnout Vandecappelle; +Cc: Joseph Kogut, buildroot
[-- Attachment #1.1: Type: text/plain, Size: 4790 bytes --]
Or…
> Il giorno 19 ott 2021, alle ore 19:40, Giulio Benetti <giulio.benetti@benettiengineering.com> ha scritto:
>
> Hi Arnout,
>
>>> Il giorno 6 ott 2021, alle ore 21:20, Arnout Vandecappelle <arnout@mind.be> ha scritto:
>>>
>>
>>
>>> On 22/09/2021 01:59, Giulio Benetti wrote:
>>> As explained in the patch itself(pending here[1]), I couldn't reproduce
>>> the failure but it seems to be right. Basically I'm making not parallel
>>> every 'nsinstall -R' command that is where it fails.
>>> [1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>>> Fixes:
>>> http://autobuild.buildroot.net/results/a895ed7aa48edc5cd9ac8bbbd689433a98c9c057/
>>> Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
>>> ---
>>> ...31911-Fix-nsinstall-parallel-failure.patch | 34 +++++++++++++++++++
>>> 1 file changed, 34 insertions(+)
>>> create mode 100644 package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>>> diff --git a/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>>> new file mode 100644
>>> index 0000000000..7ac340f1ca
>>> --- /dev/null
>>> +++ b/package/libnss/0001-Bug-1731911-Fix-nsinstall-parallel-failure.patch
>>> @@ -0,0 +1,34 @@
>>> +From 7ba4d627212507f876dc112e89c68e82e1c50188 Mon Sep 17 00:00:00 2001
>>> +From: Giulio Benetti <giulio.benetti@benettiengineering.com>
>>> +Date: Wed, 22 Sep 2021 01:40:31 +0200
>>> +Subject: [PATCH] Bug 1731911 - Fix nsinstall parallel failure
>>> +
>>> +Fix nsinstall race condition when building using Makefile with parallel
>>> +options on. I've end up making $(1) recipe in copy_varlist_into_dir_RULE
>>> +.NOTPARALLEL. I can't test it that much since it's very very rare, but
>>> +it seems to be the correct way. This way every copy_varlist_into_dir_RULE
>>> +called should occur not parallel and I think that this is acceptable
>>> +since it only create symlinks that takes very few. Further improving is
>>> +possible of course.
>>> +
>>> +Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
>>> +---
>>> + nss/coreconf/rules.mk | 2 ++
>>> + 1 file changed, 2 insertions(+)
>>> +
>>> +diff --git a/nss/coreconf/rules.mk b/nss/coreconf/rules.mk
>>> +index dc72e9cef..a341b8385 100644
>>> +--- a/nss/coreconf/rules.mk
>>> ++++ b/nss/coreconf/rules.mk
>>> +@@ -377,6 +377,8 @@ $(3)/%: %
>>> + $$(INSTALL) -m 444 $$^ $(3)
>>> +
>>> + $(1): $$(addprefix $(3)/,$$($(2))) | $(3)/d
>>> ++
>>> ++.NOTPARALLEL: $(1)
>>
>> This doesn't make a whole lot of sense: .NOTPARALLEL is something that applies to the Makefile as a whole, not to one specific target. I haven't looked in any depth in the libnss build system, but unless it builds recursively, adding this is exactly equivalent to adding
>>
>> .NOTPARALLEL:
>
> As I’ve tested it’s not like this.
>
> I read this from [1]:
> ‘’’
> .NOTPARALLEL
> If .NOTPARALLEL is mentioned as a target, then this invocation of make will be run serially, even if the ‘-j’ option is given. Any recursively invoked make command will still run recipes in parallel (unless its makefile also contains this target). Any prerequisites on this target are ignored.
>
> ‘’’
>
> They talk about target and it is what I experience.
> It’s a bit ambiguous honestly. In the past I’ve also
> experienced it on a big Makefile where I had .bmp files to be converted with xxd and that gave parallel problems. So I’ve tagged it like:
>
> .NOTPARALLEL: %.bmp
>
> and that solved the problem.
…it is because in both cases they are called recursively.
Giulio
>
> Also they have just upstreamed the patch [2], so
> we can test if it works committing this patch or
> by bumping next version.
>
> Then it’s up to you :-)
>
> Best regards
> Giulio Benetti
> Benetti Engineering sas
>
> [1]: https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
> [2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>
>>
>> anywhere in the Makefiles. This is BTW what we do in Buildroot.
>>
>> And in the end, that's the same as calling $(MAKE1) from Buildroot, so patching is not really needed.
>>
>> If you want to be a little smarter, you will have to manually "unroll" the dependency, like:
>>
>> $(1): | $(3)/d
>> $$(foreach tgt,$(2),$(MAKE) $(3)/$$($(tgt)))
>>
>> (chances are I made some major mistake in this thing).
>>
>> Or if it does build recursively, you can probably conditionally add .NOTPARALLEL in some smart way.
>>
>> Anyway, I've marked this patch as Changed Requested.
>>
>> Regards,
>> Arnout
>>
>>
>>> + endif
>>> + else
>>> + $(1):
>>> +--
>>> +2.25.1
>>> +
[-- Attachment #1.2: Type: text/html, Size: 9491 bytes --]
[-- Attachment #2: Type: text/plain, Size: 150 bytes --]
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-10-19 17:59 ` Giulio Benetti
@ 2021-10-19 19:01 ` Arnout Vandecappelle
2021-10-19 19:38 ` Giulio Benetti
0 siblings, 1 reply; 9+ messages in thread
From: Arnout Vandecappelle @ 2021-10-19 19:01 UTC (permalink / raw)
To: Giulio Benetti; +Cc: Joseph Kogut, buildroot
On 19/10/2021 19:59, Giulio Benetti wrote:
> Or…
>
>> Il giorno 19 ott 2021, alle ore 19:40, Giulio Benetti
>> <giulio.benetti@benettiengineering.com> ha scritto:
>>
>> Hi Arnout,
>>
>>> Il giorno 6 ott 2021, alle ore 21:20, Arnout Vandecappelle <arnout@mind.be>
>>> ha scritto:
>>>
>>>
>>>
>>> On 22/09/2021 01:59, Giulio Benetti wrote:
[snip]
>>>> ++.NOTPARALLEL: $(1)
>>>
>>> This doesn't make a whole lot of sense: .NOTPARALLEL is something that
>>> applies to the Makefile as a whole, not to one specific target. I haven't
>>> looked in any depth in the libnss build system, but unless it builds
>>> recursively, adding this is exactly equivalent to adding
>>>
>>> .NOTPARALLEL:
>>
>> As I’ve tested it’s not like this.
>>
>> I read this from [1]:
>> ‘’’
>> |.NOTPARALLEL|
>>
>> If |.NOTPARALLEL| is mentioned as a target, then this invocation of
>> |make| will be run serially, even if the ‘-j’ option is given. Any
>> recursively invoked |make| command will still run recipes in parallel
>> (unless its makefile also contains this target). Any prerequisites on this
>> target are ignored.
>>
>> ‘’’
>>
>> They talk about target and it is what I experience.
>> It’s a bit ambiguous honestly.
"target" is the thing on the left-hand side of a colon. So in
.NOTPARALLEL: %.bmp
the target is .NOTPARALLEL. My point is that the thing on the right-hand side of
the colon is ignored. It makes no difference at all if you put %.bmp there, or
blablabla - when you use that Makefile, the -j option will be ignored.
So what you're doing is that you make the whole thing not run in parallel, not
just the build of whatever you put on the right-hand side.
>> In the past I’ve also
>> experienced it on a big Makefile where I had .bmp files to be converted with
>> xxd and that gave parallel problems. So I’ve tagged it like:
>>
>> .NOTPARALLEL: %.bmp
>>
>> and that solved the problem.
>
> …it is because in both cases they are called recursively.
Because it is called recursively, the higher-level make invocations are indeed
still run in parallel.
Therefore, indeed, it does make sense to put it inside the macro.
But it doesn't make sense to add a right-hand side.
Anyway, since it's been applied upstream, we can just ignore the issue.
Regards,
Arnout
>
> Giulio
>
>>
>> Also they have just upstreamed the patch [2], so
>> we can test if it works committing this patch or
>> by bumping next version.
>>
>> Then it’s up to you :-)
>>
>> Best regards
>> Giulio Benetti
>> Benetti Engineering sas
>>
>> [1]: https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
>> <https://www.gnu.org/software/make/manual/html_node/Special-Targets.html>
>> [2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1731911>
>>
>>>
>>> anywhere in the Makefiles. This is BTW what we do in Buildroot.
>>>
>>> And in the end, that's the same as calling $(MAKE1) from Buildroot, so
>>> patching is not really needed.
>>>
>>> If you want to be a little smarter, you will have to manually "unroll" the
>>> dependency, like:
>>>
>>> $(1): | $(3)/d
>>> $$(foreach tgt,$(2),$(MAKE) $(3)/$$($(tgt)))
>>>
>>> (chances are I made some major mistake in this thing).
>>>
>>> Or if it does build recursively, you can probably conditionally add
>>> .NOTPARALLEL in some smart way.
>>>
>>> Anyway, I've marked this patch as Changed Requested.
>>>
>>> Regards,
>>> Arnout
>>>
>>>
>>>> + endif
>>>> + else
>>>> + $(1):
>>>> +--
>>>> +2.25.1
>>>> +
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/libnss: fix parallel build failure
2021-10-19 19:01 ` Arnout Vandecappelle
@ 2021-10-19 19:38 ` Giulio Benetti
0 siblings, 0 replies; 9+ messages in thread
From: Giulio Benetti @ 2021-10-19 19:38 UTC (permalink / raw)
To: Arnout Vandecappelle; +Cc: Joseph Kogut, buildroot
Hi Arnout,
On 10/19/21 9:01 PM, Arnout Vandecappelle wrote:
>
>
> On 19/10/2021 19:59, Giulio Benetti wrote:
>> Or…
>>
>>> Il giorno 19 ott 2021, alle ore 19:40, Giulio Benetti
>>> <giulio.benetti@benettiengineering.com> ha scritto:
>>>
>>> Hi Arnout,
>>>
>>>> Il giorno 6 ott 2021, alle ore 21:20, Arnout Vandecappelle <arnout@mind.be>
>>>> ha scritto:
>>>>
>>>>
>>>>
>>>> On 22/09/2021 01:59, Giulio Benetti wrote:
>
> [snip]
>>>>> ++.NOTPARALLEL: $(1)
>>>>
>>>> This doesn't make a whole lot of sense: .NOTPARALLEL is something that
>>>> applies to the Makefile as a whole, not to one specific target. I haven't
>>>> looked in any depth in the libnss build system, but unless it builds
>>>> recursively, adding this is exactly equivalent to adding
>>>>
>>>> .NOTPARALLEL:
>>>
>>> As I’ve tested it’s not like this.
>>>
>>> I read this from [1]:
>>> ‘’’
>>> |.NOTPARALLEL|
>>>
>>> If |.NOTPARALLEL| is mentioned as a target, then this invocation of
>>> |make| will be run serially, even if the ‘-j’ option is given. Any
>>> recursively invoked |make| command will still run recipes in parallel
>>> (unless its makefile also contains this target). Any prerequisites on this
>>> target are ignored.
>>>
>>> ‘’’
>>>
>>> They talk about target and it is what I experience.
>>> It’s a bit ambiguous honestly.
>
> "target" is the thing on the left-hand side of a colon. So in
>
> .NOTPARALLEL: %.bmp
>
> the target is .NOTPARALLEL. My point is that the thing on the right-hand side of
> the colon is ignored. It makes no difference at all if you put %.bmp there, or
> blablabla - when you use that Makefile, the -j option will be ignored.
>
> So what you're doing is that you make the whole thing not run in parallel, not
> just the build of whatever you put on the right-hand side.
>
>
>>> In the past I’ve also
>>> experienced it on a big Makefile where I had .bmp files to be converted with
>>> xxd and that gave parallel problems. So I’ve tagged it like:
>>>
>>> .NOTPARALLEL: %.bmp
>>>
>>> and that solved the problem.
>>
>> …it is because in both cases they are called recursively.
>
> Because it is called recursively, the higher-level make invocations are indeed
> still run in parallel.
Ok, it's like I've written on the next e-mail(too late :-) ) that you've
already pointed me in the previous e-mail.
> Therefore, indeed, it does make sense to put it inside the macro.
>
> But it doesn't make sense to add a right-hand side.
Ok. Thank you for explaining me!
> Anyway, since it's been applied upstream, we can just ignore the issue.
Yes
Thanks again
Kind regards
Giulio
> Regards,
> Arnout
>
>
>>
>> Giulio
>>
>>>
>>> Also they have just upstreamed the patch [2], so
>>> we can test if it works committing this patch or
>>> by bumping next version.
>>>
>>> Then it’s up to you :-)
>>>
>>> Best regards
>>> Giulio Benetti
>>> Benetti Engineering sas
>>>
>>> [1]: https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
>>> <https://www.gnu.org/software/make/manual/html_node/Special-Targets.html>
>>> [2]: https://bugzilla.mozilla.org/show_bug.cgi?id=1731911
>>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1731911>
>>>
>>>>
>>>> anywhere in the Makefiles. This is BTW what we do in Buildroot.
>>>>
>>>> And in the end, that's the same as calling $(MAKE1) from Buildroot, so
>>>> patching is not really needed.
>>>>
>>>> If you want to be a little smarter, you will have to manually "unroll" the
>>>> dependency, like:
>>>>
>>>> $(1): | $(3)/d
>>>> $$(foreach tgt,$(2),$(MAKE) $(3)/$$($(tgt)))
>>>>
>>>> (chances are I made some major mistake in this thing).
>>>>
>>>> Or if it does build recursively, you can probably conditionally add
>>>> .NOTPARALLEL in some smart way.
>>>>
>>>> Anyway, I've marked this patch as Changed Requested.
>>>>
>>>> Regards,
>>>> Arnout
>>>>
>>>>
>>>>> + endif
>>>>> + else
>>>>> + $(1):
>>>>> +--
>>>>> +2.25.1
>>>>> +
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
>
--
Giulio Benetti
Benetti Engineering sas
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-10-19 19:38 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-21 23:59 [Buildroot] [PATCH 0/1] Fix libnss parallel build failure Giulio Benetti
2021-09-21 23:59 ` [Buildroot] [PATCH 1/1] package/libnss: fix " Giulio Benetti
2021-10-06 19:20 ` Arnout Vandecappelle
2021-10-19 17:33 ` Giulio Benetti
2021-10-19 17:59 ` Giulio Benetti
2021-10-19 19:01 ` Arnout Vandecappelle
2021-10-19 19:38 ` Giulio Benetti
2021-09-23 16:50 ` [Buildroot] [PATCH 0/1] Fix libnss " Giulio Benetti
2021-10-04 23:22 ` Giulio Benetti
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.