[PATCH] x86,build: Fix make -jN modules_install install

[PATCH] x86,build: Fix make -jN modules_install install

[Andy Lutomirski]

Every few months, I forget why I type:

$ sudo make -j12 modules_install && sudo make -j12 install

instead of just:

$ sudo make -j12 modules_install install

I try the latter, it appears to work, and then my machine won't boot
because dracut got confused.  This fixes it once and for all: if you
ask make to install modules and a kernel, you almost certainly want
the modules installed *first* so that your initramfs scripts can
find the modules.

Signed-off-by: Andy Lutomirski <luto@amacapital.net>
---
 arch/x86/Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 33f71b0..7280d28 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -240,6 +240,15 @@ PHONY += install
 install:
 	$(Q)$(MAKE) $(build)=$(boot) $@
 
+# If installing modules and a kernel, it's very likely that some initramfs
+# script associated with installing the kernel will reference the modules,
+# so make sure that modules are installed first.
+ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+  ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+    install: modules_install
+  endif
+endif
+
 PHONY += vdso_install
 vdso_install:
 	$(Q)$(MAKE) $(build)=arch/x86/vdso $@
-- 
1.9.3

Re: [PATCH] x86,build: Fix make -jN modules_install install

[Sam Ravnborg]

On Wed, Jun 11, 2014 at 12:41:57PM -0700, Andy Lutomirski wrote:
> Every few months, I forget why I type:
> 
> $ sudo make -j12 modules_install && sudo make -j12 install
> 
> instead of just:
> 
> $ sudo make -j12 modules_install install
> 
> I try the latter, it appears to work, and then my machine won't boot
> because dracut got confused.  This fixes it once and for all: if you
> ask make to install modules and a kernel, you almost certainly want
> the modules installed *first* so that your initramfs scripts can
> find the modules.

Is this problem x86 specific?

	Sam

Re: [PATCH] x86,build: Fix make -jN modules_install install

[Andy Lutomirski]

On Wed, Jun 11, 2014 at 12:44 PM, Sam Ravnborg <sam@ravnborg.org> wrote:
> On Wed, Jun 11, 2014 at 12:41:57PM -0700, Andy Lutomirski wrote:
>> Every few months, I forget why I type:
>>
>> $ sudo make -j12 modules_install && sudo make -j12 install
>>
>> instead of just:
>>
>> $ sudo make -j12 modules_install install
>>
>> I try the latter, it appears to work, and then my machine won't boot
>> because dracut got confused.  This fixes it once and for all: if you
>> ask make to install modules and a kernel, you almost certainly want
>> the modules installed *first* so that your initramfs scripts can
>> find the modules.
>
> Is this problem x86 specific?

I don't know.  But I also don't want to have 'make modules_install
install' on an arch without an install target appear to work, so I
don't know how to do it in the kbuild core.  That is, I don't want to
define an install target; I just want to add a dependency if the
target is already there.

Also, this patch has a repeated ifneq.  I'll fix it in v2.

>
>         Sam



-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: [PATCH] x86,build: Fix make -jN modules_install install

[Michal Marek]

Dne 11.6.2014 21:41, Andy Lutomirski napsal(a):
> Every few months, I forget why I type:
> 
> $ sudo make -j12 modules_install && sudo make -j12 install
> 
> instead of just:
> 
> $ sudo make -j12 modules_install install
> 
> I try the latter, it appears to work, and then my machine won't boot
> because dracut got confused.  This fixes it once and for all: if you
> ask make to install modules and a kernel, you almost certainly want
> the modules installed *first* so that your initramfs scripts can
> find the modules.
> 
> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
> ---
>  arch/x86/Makefile | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index 33f71b0..7280d28 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -240,6 +240,15 @@ PHONY += install
>  install:
>  	$(Q)$(MAKE) $(build)=$(boot) $@
>  
> +# If installing modules and a kernel, it's very likely that some initramfs
> +# script associated with installing the kernel will reference the modules,
> +# so make sure that modules are installed first.
> +ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> +  ifneq ($(filter modules_install,$(MAKECMDGOALS)),)

The two conditions are identical. Did you mean to check for "install" in
one of them?

Thanks,
Michal

Re: [PATCH] x86,build: Fix make -jN modules_install install

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 1:33 AM, Michal Marek <mmarek@suse.cz> wrote:
> Dne 11.6.2014 21:41, Andy Lutomirski napsal(a):
>> Every few months, I forget why I type:
>>
>> $ sudo make -j12 modules_install && sudo make -j12 install
>>
>> instead of just:
>>
>> $ sudo make -j12 modules_install install
>>
>> I try the latter, it appears to work, and then my machine won't boot
>> because dracut got confused.  This fixes it once and for all: if you
>> ask make to install modules and a kernel, you almost certainly want
>> the modules installed *first* so that your initramfs scripts can
>> find the modules.
>>
>> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
>> ---
>>  arch/x86/Makefile | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
>> index 33f71b0..7280d28 100644
>> --- a/arch/x86/Makefile
>> +++ b/arch/x86/Makefile
>> @@ -240,6 +240,15 @@ PHONY += install
>>  install:
>>       $(Q)$(MAKE) $(build)=$(boot) $@
>>
>> +# If installing modules and a kernel, it's very likely that some initramfs
>> +# script associated with installing the kernel will reference the modules,
>> +# so make sure that modules are installed first.
>> +ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> +  ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>
> The two conditions are identical. Did you mean to check for "install" in
> one of them?

Yes, although I think that the check for "install" is actually unnecessary.

Is there some way to do this in the core kbuild?

--Andy

>
> Thanks,
> Michal



-- 
Andy Lutomirski
AMA Capital Management, LLC

[PATCH] kbuild: Do not run modules_install and install in paralel

[Michal Marek]

Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>

With modular kernels, 'make install' is going to need the installed
modules at some point to generate the initramfs.

Signed-off-by: Michal Marek <mmarek@suse.cz>
---
 Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index 7680d7c..7e5e483 100644
--- a/Makefile
+++ b/Makefile
@@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
                 endif
         endif
 endif
+# install and module_install need also be processed one by one
+ifneq ($(filter install,$(MAKECMDGOALS)),)
+        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+	        mixed-targets := 1
+        endif
+endif
 
 ifeq ($(mixed-targets),1)
 # ===========================================================================
-- 
1.9.2

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Michal Marek]

Dne 13.6.2014 11:39, Michal Marek napsal(a):
> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
> 
> With modular kernels, 'make install' is going to need the installed
> modules at some point to generate the initramfs.
> 
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  Makefile | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 7680d7c..7e5e483 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>                  endif
>          endif
>  endif
> +# install and module_install need also be processed one by one
> +ifneq ($(filter install,$(MAKECMDGOALS)),)
> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> +	        mixed-targets := 1
> +        endif
> +endif

Note that this version does not enforce the ordering, it just avoids the
interleaved execution. It can be added if desired.

Michal

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Andy Lutomirski]

On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <mmarek@suse.cz> wrote:
> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>>
>> With modular kernels, 'make install' is going to need the installed
>> modules at some point to generate the initramfs.
>>
>> Signed-off-by: Michal Marek <mmarek@suse.cz>
>> ---
>>  Makefile | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index 7680d7c..7e5e483 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>                  endif
>>          endif
>>  endif
>> +# install and module_install need also be processed one by one
>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> +             mixed-targets := 1
>> +        endif
>> +endif
>
> Note that this version does not enforce the ordering, it just avoids the
> interleaved execution. It can be added if desired.

Hmm.  This will fix 'make modules_install install' but will not fix
'make install modules_install'.  I don't know how many people would
type the latter.

--Andy

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Michal Marek]

Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <mmarek@suse.cz> wrote:
>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>>>
>>> With modular kernels, 'make install' is going to need the installed
>>> modules at some point to generate the initramfs.
>>>
>>> Signed-off-by: Michal Marek <mmarek@suse.cz>
>>> ---
>>>  Makefile | 6 ++++++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 7680d7c..7e5e483 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>>                  endif
>>>          endif
>>>  endif
>>> +# install and module_install need also be processed one by one
>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>> +             mixed-targets := 1
>>> +        endif
>>> +endif
>>
>> Note that this version does not enforce the ordering, it just avoids the
>> interleaved execution. It can be added if desired.
> 
> Hmm.  This will fix 'make modules_install install' but will not fix
> 'make install modules_install'.  I don't know how many people would
> type the latter.

Resuming a thread from last month -- As I said, it does not enforce the
ordering. So it's not as bullet-proof as your patch, but it works on all
architectures. I'm thinking about adding the patch to kbuild.git for
3.17. But if you want to fix the ordering as well, feel free to change
the __build_one_by_one rule to do this.

Thanks,
Michal

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Andy Lutomirski]

On Fri, Jul 4, 2014 at 3:15 PM, Michal Marek <mmarek@suse.cz> wrote:
> Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
>> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <mmarek@suse.cz> wrote:
>>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>>> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>>>>
>>>> With modular kernels, 'make install' is going to need the installed
>>>> modules at some point to generate the initramfs.
>>>>
>>>> Signed-off-by: Michal Marek <mmarek@suse.cz>
>>>> ---
>>>>  Makefile | 6 ++++++
>>>>  1 file changed, 6 insertions(+)
>>>>
>>>> diff --git a/Makefile b/Makefile
>>>> index 7680d7c..7e5e483 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>>>                  endif
>>>>          endif
>>>>  endif
>>>> +# install and module_install need also be processed one by one
>>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>>> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>>> +             mixed-targets := 1
>>>> +        endif
>>>> +endif
>>>
>>> Note that this version does not enforce the ordering, it just avoids the
>>> interleaved execution. It can be added if desired.
>>
>> Hmm.  This will fix 'make modules_install install' but will not fix
>> 'make install modules_install'.  I don't know how many people would
>> type the latter.
>
> Resuming a thread from last month -- As I said, it does not enforce the
> ordering. So it's not as bullet-proof as your patch, but it works on all
> architectures. I'm thinking about adding the patch to kbuild.git for
> 3.17. But if you want to fix the ordering as well, feel free to change
> the __build_one_by_one rule to do this.

Sounds good to me.

Where's __build_one_by_one?

--Andy

>
> Thanks,
> Michal



-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Michal Marek]

On 2014-07-05 01:45, Andy Lutomirski wrote:
> On Fri, Jul 4, 2014 at 3:15 PM, Michal Marek <mmarek@suse.cz> wrote:
>> Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
>>> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <mmarek@suse.cz> wrote:
>>>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>>>> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>>>>>
>>>>> With modular kernels, 'make install' is going to need the installed
>>>>> modules at some point to generate the initramfs.
>>>>>
>>>>> Signed-off-by: Michal Marek <mmarek@suse.cz>
>>>>> ---
>>>>>  Makefile | 6 ++++++
>>>>>  1 file changed, 6 insertions(+)
>>>>>
>>>>> diff --git a/Makefile b/Makefile
>>>>> index 7680d7c..7e5e483 100644
>>>>> --- a/Makefile
>>>>> +++ b/Makefile
>>>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>>>>                  endif
>>>>>          endif
>>>>>  endif
>>>>> +# install and module_install need also be processed one by one
>>>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>>>> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>>>> +             mixed-targets := 1
>>>>> +        endif
>>>>> +endif
>>>>
>>>> Note that this version does not enforce the ordering, it just avoids the
>>>> interleaved execution. It can be added if desired.
>>>
>>> Hmm.  This will fix 'make modules_install install' but will not fix
>>> 'make install modules_install'.  I don't know how many people would
>>> type the latter.
>>
>> Resuming a thread from last month -- As I said, it does not enforce the
>> ordering. So it's not as bullet-proof as your patch, but it works on all
>> architectures. I'm thinking about adding the patch to kbuild.git for
>> 3.17. But if you want to fix the ordering as well, feel free to change
>> the __build_one_by_one rule to do this.
> 
> Sounds good to me.
> 
> Where's __build_one_by_one?

A few lines below in the main Makefile:

   513  ifeq ($(mixed-targets),1)
   514  #
======================================================================
=====
   515  # We're called with mixed targets (*config and build targets).
   516  # Handle them one by one.
   517
   518  PHONY += $(MAKECMDGOALS) __build_one_by_one
   519
   520  $(filter-out __build_one_by_one, $(MAKECMDGOALS)):
__build_one_by_one
   521          @:
   522
   523  __build_one_by_one:
   524          $(Q)set -e; \
   525          for i in $(MAKECMDGOALS); do \
   526                  $(MAKE) -f $(srctree)/Makefile $$i; \
   527          done

Michal

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Andy Lutomirski]

On Fri, Jun 13, 2014 at 2:39 AM, Michal Marek <mmarek@suse.cz> wrote:
> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>
> With modular kernels, 'make install' is going to need the installed
> modules at some point to generate the initramfs.
>
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  Makefile | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 7680d7c..7e5e483 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>                  endif
>          endif
>  endif
> +# install and module_install need also be processed one by one
> +ifneq ($(filter install,$(MAKECMDGOALS)),)
> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> +               mixed-targets := 1
> +        endif
> +endif
>
>  ifeq ($(mixed-targets),1)
>  # ===========================================================================
> --
> 1.9.2
>

This patch seems to have gotten lost.  Do you still like it?  If so,
can someone apply it, please?

--Andy

Re: [PATCH] kbuild: Do not run modules_install and install in paralel

[Michal Marek]

Dne 9.12.2015 v 22:34 Andy Lutomirski napsal(a):
> On Fri, Jun 13, 2014 at 2:39 AM, Michal Marek <mmarek@suse.cz> wrote:
>> Based on a x86-only patch by Andy Lutomirski <luto@amacapital.net>
>>
>> With modular kernels, 'make install' is going to need the installed
>> modules at some point to generate the initramfs.
>>
>> Signed-off-by: Michal Marek <mmarek@suse.cz>
>> ---
>>  Makefile | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index 7680d7c..7e5e483 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>                  endif
>>          endif
>>  endif
>> +# install and module_install need also be processed one by one
>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>> +        ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> +               mixed-targets := 1
>> +        endif
>> +endif
>>
>>  ifeq ($(mixed-targets),1)
>>  # ===========================================================================
>> --
>> 1.9.2
>>
> 
> This patch seems to have gotten lost.  Do you still like it?  If so,
> can someone apply it, please?

Oops. I will rebase it and apply to the kbuild tree.

Michal