[PATCH 0/5] move everyone over to thin archives

[PATCH 0/5] move everyone over to thin archives

[Nicholas Piggin]

This has gone through 0day several times now, and I think I've
weeded out all the build breakage. Following what Linus asked
for, I did not make it a user selectable option. I kept the
option there in case we need to do a depend on !ARCH for any
reason.

Please consider for the kbuild tree.

Nicholas Piggin (5):
  kbuild: thin archives final link close --whole-archives option
  kbuild: thin archives use P option to ar
  sh: thin archives fix linking
  x86/um: thin archives build fix
  kbuild: thin archives make default for all archs

 Documentation/process/changes.rst | 9 ++++-----
 arch/Kconfig                      | 2 +-
 arch/powerpc/Kconfig              | 8 --------
 arch/sh/Makefile                  | 2 ++
 arch/sh/kernel/vsyscall/Makefile  | 2 --
 arch/x86/um/vdso/Makefile         | 2 +-
 scripts/Makefile.build            | 8 ++++----
 scripts/link-vmlinux.sh           | 6 +++---
 8 files changed, 15 insertions(+), 24 deletions(-)

-- 
2.11.0

[PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

Close the --whole-archives option with --no-whole-archive. Some
architectures end up including additional .o and files multiple
times after this, and they get duplicate symbols when they are
brought under the --whole-archives option.

This matches more closely with the incremental final link.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 scripts/link-vmlinux.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index c80291319cb2..2a062ea130b5 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -63,7 +63,7 @@ modpost_link()
 	local objects
 
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-		objects="--whole-archive built-in.o"
+		objects="--whole-archive built-in.o --no-whole-archive"
 	else
 		objects="${KBUILD_VMLINUX_INIT}				\
 			--start-group					\
@@ -83,7 +83,7 @@ vmlinux_link()
 
 	if [ "${SRCARCH}" != "um" ]; then
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="--whole-archive built-in.o ${1}"
+			objects="--whole-archive built-in.o ${1} --no-whole-archive"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				--start-group				\
@@ -96,7 +96,7 @@ vmlinux_link()
 			-T ${lds} ${objects}
 	else
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="-Wl,--whole-archive built-in.o ${1}"
+			objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				-Wl,--start-group			\
-- 
2.11.0

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,


2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> Close the --whole-archives option with --no-whole-archive. Some
> architectures end up including additional .o and files multiple
> times after this, and they get duplicate symbols when they are
> brought under the --whole-archives option.

Which architectures have additional files after --no-whole-archive ?

I see this case only for ARCH = "um" in vmlinux_link()
where it adds some -l* options after --no-whole-archive.

I think it is good to close the --whole-archives everywhere.
So, the code looks OK to me, but I just wondered about the log.


> This matches more closely with the incremental final link.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  scripts/link-vmlinux.sh | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index c80291319cb2..2a062ea130b5 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -63,7 +63,7 @@ modpost_link()
>         local objects
>
>         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -               objects="--whole-archive built-in.o"
> +               objects="--whole-archive built-in.o --no-whole-archive"
>         else
>                 objects="${KBUILD_VMLINUX_INIT}                         \
>                         --start-group                                   \
> @@ -83,7 +83,7 @@ vmlinux_link()
>
>         if [ "${SRCARCH}" != "um" ]; then
>                 if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -                       objects="--whole-archive built-in.o ${1}"
> +                       objects="--whole-archive built-in.o ${1} --no-whole-archive"
>                 else
>                         objects="${KBUILD_VMLINUX_INIT}                 \
>                                 --start-group                           \
> @@ -96,7 +96,7 @@ vmlinux_link()
>                         -T ${lds} ${objects}
>         else
>                 if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -                       objects="-Wl,--whole-archive built-in.o ${1}"
> +                       objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
>                 else
>                         objects="${KBUILD_VMLINUX_INIT}                 \
>                                 -Wl,--start-group                       \







-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Mon, 19 Jun 2017 15:16:17 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > Close the --whole-archives option with --no-whole-archive. Some
> > architectures end up including additional .o and files multiple
> > times after this, and they get duplicate symbols when they are
> > brought under the --whole-archives option.  
> 
> Which architectures have additional files after --no-whole-archive ?
> 
> I see this case only for ARCH = "um" in vmlinux_link()
> where it adds some -l* options after --no-whole-archive.
> 
> I think it is good to close the --whole-archives everywhere.
> So, the code looks OK to me, but I just wondered about the log.

Actually a number of archs seemed to get build errors without this,
and they seem to come from libgcc perhaps.

binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
/c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
/c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
/c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
/c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'

/home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'

etc

m32r, parisc, xtensa seemed to be getting such errors. I wonder if
the linker implicitly adds some runtime libs at the end that get
caught up here. Either way I didn't look too far into it because this
fix seems obviously correct and solved the problem.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,


2017-06-19 15:51 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> On Mon, 19 Jun 2017 15:16:17 +0900
> Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>
>> Hi Nicholas,
>>
>>
>> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>> > Close the --whole-archives option with --no-whole-archive. Some
>> > architectures end up including additional .o and files multiple
>> > times after this, and they get duplicate symbols when they are
>> > brought under the --whole-archives option.
>>
>> Which architectures have additional files after --no-whole-archive ?
>>
>> I see this case only for ARCH = "um" in vmlinux_link()
>> where it adds some -l* options after --no-whole-archive.
>>
>> I think it is good to close the --whole-archives everywhere.
>> So, the code looks OK to me, but I just wondered about the log.
>
> Actually a number of archs seemed to get build errors without this,
> and they seem to come from libgcc perhaps.
>
> binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
>
> /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
>
> etc


Oops, I did not test such architectures.


> m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> the linker implicitly adds some runtime libs at the end that get
> caught up here. Either way I didn't look too far into it because this
> fix seems obviously correct and solved the problem.
>
> Thanks,


Which toolchains do you use?

I downloaded xtensa-linux- from this site:
https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/


I see a different error for xtensa.


I applied all of you 5 patches, but xtensa build still fails.

  LD      vmlinux.o
xtensa-linux-ld: internal error
/home/tony/buildall/src/binutils/ld/ldlang.c 6178


Could you check it?


-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Mon, 19 Jun 2017 17:14:22 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > On Mon, 19 Jun 2017 15:16:17 +0900
> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> >  
> >> Hi Nicholas,
> >>
> >>
> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:  
> >> > Close the --whole-archives option with --no-whole-archive. Some
> >> > architectures end up including additional .o and files multiple
> >> > times after this, and they get duplicate symbols when they are
> >> > brought under the --whole-archives option.  
> >>
> >> Which architectures have additional files after --no-whole-archive ?
> >>
> >> I see this case only for ARCH = "um" in vmlinux_link()
> >> where it adds some -l* options after --no-whole-archive.
> >>
> >> I think it is good to close the --whole-archives everywhere.
> >> So, the code looks OK to me, but I just wondered about the log.  
> >
> > Actually a number of archs seemed to get build errors without this,
> > and they seem to come from libgcc perhaps.
> >
> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
> >
> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
> >
> > etc  
> 
> 
> Oops, I did not test such architectures.
> 
> 
> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> > the linker implicitly adds some runtime libs at the end that get
> > caught up here. Either way I didn't look too far into it because this
> > fix seems obviously correct and solved the problem.
> >
> > Thanks,  
> 
> 
> Which toolchains do you use?

I just had them run through the kbuild 0day tests. Not sure exactly
what they use.

> I downloaded xtensa-linux- from this site:
> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
> 
> 
> I see a different error for xtensa.
> 
> 
> I applied all of you 5 patches, but xtensa build still fails.
> 
>   LD      vmlinux.o
> xtensa-linux-ld: internal error
> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
> 
> 
> Could you check it?

Ahh, the old ld: internal error... what ld version is that?

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,


2017-06-19 17:27 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> On Mon, 19 Jun 2017 17:14:22 +0900
> Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>
>> Hi Nicholas,
>>
>>
>> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>> > On Mon, 19 Jun 2017 15:16:17 +0900
>> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>> >
>> >> Hi Nicholas,
>> >>
>> >>
>> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>> >> > Close the --whole-archives option with --no-whole-archive. Some
>> >> > architectures end up including additional .o and files multiple
>> >> > times after this, and they get duplicate symbols when they are
>> >> > brought under the --whole-archives option.
>> >>
>> >> Which architectures have additional files after --no-whole-archive ?
>> >>
>> >> I see this case only for ARCH = "um" in vmlinux_link()
>> >> where it adds some -l* options after --no-whole-archive.
>> >>
>> >> I think it is good to close the --whole-archives everywhere.
>> >> So, the code looks OK to me, but I just wondered about the log.
>> >
>> > Actually a number of archs seemed to get build errors without this,
>> > and they seem to come from libgcc perhaps.
>> >
>> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
>> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
>> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
>> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
>> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
>> >
>> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
>> >
>> > etc
>>
>>
>> Oops, I did not test such architectures.
>>
>>
>> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
>> > the linker implicitly adds some runtime libs at the end that get
>> > caught up here. Either way I didn't look too far into it because this
>> > fix seems obviously correct and solved the problem.
>> >
>> > Thanks,
>>
>>
>> Which toolchains do you use?
>
> I just had them run through the kbuild 0day tests. Not sure exactly
> what they use.
>
>> I downloaded xtensa-linux- from this site:
>> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
>>
>>
>> I see a different error for xtensa.
>>
>>
>> I applied all of you 5 patches, but xtensa build still fails.
>>
>>   LD      vmlinux.o
>> xtensa-linux-ld: internal error
>> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
>>
>>
>> Could you check it?
>
> Ahh, the old ld: internal error... what ld version is that?
>


seems 2.24 according to the following:


masahiro@pug:~$ xtensa-linux-ld -v
GNU ld (GNU Binutils) 2.24





-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Mon, 19 Jun 2017 17:35:32 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-19 17:27 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > On Mon, 19 Jun 2017 17:14:22 +0900
> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> >  
> >> Hi Nicholas,
> >>
> >>
> >> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:  
> >> > On Mon, 19 Jun 2017 15:16:17 +0900
> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> >> >  
> >> >> Hi Nicholas,
> >> >>
> >> >>
> >> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:  
> >> >> > Close the --whole-archives option with --no-whole-archive. Some
> >> >> > architectures end up including additional .o and files multiple
> >> >> > times after this, and they get duplicate symbols when they are
> >> >> > brought under the --whole-archives option.  
> >> >>
> >> >> Which architectures have additional files after --no-whole-archive ?
> >> >>
> >> >> I see this case only for ARCH = "um" in vmlinux_link()
> >> >> where it adds some -l* options after --no-whole-archive.
> >> >>
> >> >> I think it is good to close the --whole-archives everywhere.
> >> >> So, the code looks OK to me, but I just wondered about the log.  
> >> >
> >> > Actually a number of archs seemed to get build errors without this,
> >> > and they seem to come from libgcc perhaps.
> >> >
> >> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> >> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> >> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> >> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> >> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
> >> >
> >> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
> >> >
> >> > etc  
> >>
> >>
> >> Oops, I did not test such architectures.
> >>
> >>  
> >> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> >> > the linker implicitly adds some runtime libs at the end that get
> >> > caught up here. Either way I didn't look too far into it because this
> >> > fix seems obviously correct and solved the problem.
> >> >
> >> > Thanks,  
> >>
> >>
> >> Which toolchains do you use?  
> >
> > I just had them run through the kbuild 0day tests. Not sure exactly
> > what they use.
> >  
> >> I downloaded xtensa-linux- from this site:
> >> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
> >>
> >>
> >> I see a different error for xtensa.
> >>
> >>
> >> I applied all of you 5 patches, but xtensa build still fails.
> >>
> >>   LD      vmlinux.o
> >> xtensa-linux-ld: internal error
> >> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
> >>
> >>
> >> Could you check it?  
> >
> > Ahh, the old ld: internal error... what ld version is that?
> >  
> 
> 
> seems 2.24 according to the following:
> 
> 
> masahiro@pug:~$ xtensa-linux-ld -v
> GNU ld (GNU Binutils) 2.24

Ah, thank you. It must have been that the 0day build (cc'ed) did
not return any error to be from the ld internal error.

This has led me to the true cause of the error which is the way
external archives are handled. After implementing the fix, I think
the lib.a size regression for thin archives should be solved as
well.

This patch should be added between patches 2 and 3 of this series.


kbuild: handle libs-y archives separately from built-in.o archives

The thin archives build currently puts all lib.a and built-in.o
files together and links them with --whole-archive.

This works because thin archives can recursively refer to thin
archives. However some architectures include libgcc.a, which may
not be a thin archive, or it may not be constructed with the "P"
option, in which case its contents do not get linked correctly.

So don't pull .a libs into the root built-in.o archive. These
libs should already have symbol tables and indexes built, so they
can be direct linker inputs. Move them out of the --whole-archive
option, which restore the conditional linking behaviour of lib.a
to thin archives builds.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 Documentation/kbuild/kbuild.txt |  8 +++++--
 Makefile                        |  7 +++---
 scripts/link-vmlinux.sh         | 47 ++++++++++++++++++++++++++++++++---------
 3 files changed, 47 insertions(+), 15 deletions(-)

diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
index 0ff6a466a05b..ac2363ea05c5 100644
--- a/Documentation/kbuild/kbuild.txt
+++ b/Documentation/kbuild/kbuild.txt
@@ -236,5 +236,9 @@ Files specified with KBUILD_VMLINUX_INIT are linked first.
 KBUILD_VMLINUX_MAIN
 --------------------------------------------------
 All object files for the main part of vmlinux.
-KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
-all the object files used to link vmlinux.
+
+KBUILD_VMLINUX_LIBS
+--------------------------------------------------
+All .a "lib" files for vmlinux.
+KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, and KBUILD_VMLINUX_LIBS together
+specify all the object files used to link vmlinux.
diff --git a/Makefile b/Makefile
index 470bd4d9513a..a145a537ad42 100644
--- a/Makefile
+++ b/Makefile
@@ -952,19 +952,20 @@ core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
 net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
 libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
-libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
+libs-y2		:= $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
 libs-y		:= $(libs-y1) $(libs-y2)
 virt-y		:= $(patsubst %/, %/built-in.o, $(virt-y))
 
 # Externally visible symbols (used by link-vmlinux.sh)
 export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
-export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(virt-y)
+export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
+export KBUILD_VMLINUX_LIBS := $(libs-y1)
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
 export LDFLAGS_vmlinux
 # used by scripts/pacmage/Makefile
 export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) arch Documentation include samples scripts tools)
 
-vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
+vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
 
 # Include targets which we want to execute sequentially if the rest of the
 # kernel build went well. If CONFIG_TRIM_UNUSED_KSYMS is set, this might be
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 2a062ea130b5..e7b7eee31538 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -3,9 +3,12 @@
 # link vmlinux
 #
 # vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
-# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
-# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
-# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
+# $(KBUILD_VMLINUX_MAIN) and $(KBUILD_VMLINUX_LIBS). Most are built-in.o files
+# from top-level directories in the kernel tree, others are specified in
+# arch/$(ARCH)/Makefile. Ordering when linking is important, and
+# $(KBUILD_VMLINUX_INIT) must be first. $(KBUILD_VMLINUX_LIBS) are archives
+# which are linked conditionally (not within --whole-archive), and do not
+# require symbol indexes added.
 #
 # vmlinux
 #   ^
@@ -16,6 +19,9 @@
 #   +--< $(KBUILD_VMLINUX_MAIN)
 #   |    +--< drivers/built-in.o mm/built-in.o + more
 #   |
+#   +--< $(KBUILD_VMLINUX_LIBS)
+#   |    +--< lib/lib.a + more
+#   |
 #   +-< ${kallsymso} (see description in KALLSYMS section)
 #
 # vmlinux version (uname -v) cannot be updated during normal
@@ -37,9 +43,10 @@ info()
 	fi
 }
 
-# Thin archive build here makes a final archive with
-# symbol table and indexes from vmlinux objects, which can be
-# used as input to linker.
+# Thin archive build here makes a final archive with symbol table and indexes
+# from vmlinux objects INIT and MAIN, which can be used as input to linker.
+# KBUILD_VMLINUX_LIBS archives should already have symbol table and indexes
+# added.
 #
 # Traditional incremental style of link does not require this step
 #
@@ -50,7 +57,7 @@ archive_builtin()
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
 		info AR built-in.o
 		rm -f built-in.o;
-		${AR} rcsT${KBUILD_ARFLAGS} built-in.o			\
+		${AR} rcsTP${KBUILD_ARFLAGS} built-in.o			\
 					${KBUILD_VMLINUX_INIT}		\
 					${KBUILD_VMLINUX_MAIN}
 	fi
@@ -63,11 +70,17 @@ modpost_link()
 	local objects
 
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-		objects="--whole-archive built-in.o --no-whole-archive"
+		objects="--whole-archive				\
+			built-in.o					\
+			--no-whole-archive				\
+			--start-group					\
+			${KBUILD_VMLINUX_LIBS}				\
+			--end-group"
 	else
 		objects="${KBUILD_VMLINUX_INIT}				\
 			--start-group					\
 			${KBUILD_VMLINUX_MAIN}				\
+			${KBUILD_VMLINUX_LIBS}				\
 			--end-group"
 	fi
 	${LD} ${LDFLAGS} -r -o ${1} ${objects}
@@ -83,11 +96,18 @@ vmlinux_link()
 
 	if [ "${SRCARCH}" != "um" ]; then
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="--whole-archive built-in.o ${1} --no-whole-archive"
+			objects="--whole-archive			\
+				built-in.o				\
+				--no-whole-archive			\
+				--start-group				\
+				${KBUILD_VMLINUX_LIBS}			\
+				--end-group				\
+				${1}"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				--start-group				\
 				${KBUILD_VMLINUX_MAIN}			\
+				${KBUILD_VMLINUX_LIBS}			\
 				--end-group				\
 				${1}"
 		fi
@@ -96,11 +116,18 @@ vmlinux_link()
 			-T ${lds} ${objects}
 	else
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
+			objects="-Wl,--whole-archive			\
+				built-in.o				\
+				-Wl,--no-whole-archive			\
+				-Wl,--start-group			\
+				${KBUILD_VMLINUX_LIBS}			\
+				-Wl,--end-group				\
+				${1}"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				-Wl,--start-group			\
 				${KBUILD_VMLINUX_MAIN}			\
+				${KBUILD_VMLINUX_LIBS}			\
 				-Wl,--end-group				\
 				${1}"
 		fi
-- 
2.11.0

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Josh Triplett]

On Tue, Jun 20, 2017 at 01:52:05AM +1000, Nicholas Piggin wrote:
> On Mon, 19 Jun 2017 17:35:32 +0900
> Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> 
> > Hi Nicholas,
> > 
> > 
> > 2017-06-19 17:27 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > > On Mon, 19 Jun 2017 17:14:22 +0900
> > > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> > >  
> > >> Hi Nicholas,
> > >>
> > >>
> > >> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:  
> > >> > On Mon, 19 Jun 2017 15:16:17 +0900
> > >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
> > >> >  
> > >> >> Hi Nicholas,
> > >> >>
> > >> >>
> > >> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:  
> > >> >> > Close the --whole-archives option with --no-whole-archive. Some
> > >> >> > architectures end up including additional .o and files multiple
> > >> >> > times after this, and they get duplicate symbols when they are
> > >> >> > brought under the --whole-archives option.  
> > >> >>
> > >> >> Which architectures have additional files after --no-whole-archive ?
> > >> >>
> > >> >> I see this case only for ARCH = "um" in vmlinux_link()
> > >> >> where it adds some -l* options after --no-whole-archive.
> > >> >>
> > >> >> I think it is good to close the --whole-archives everywhere.
> > >> >> So, the code looks OK to me, but I just wondered about the log.  
> > >> >
> > >> > Actually a number of archs seemed to get build errors without this,
> > >> > and they seem to come from libgcc perhaps.
> > >> >
> > >> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> > >> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> > >> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> > >> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> > >> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
> > >> >
> > >> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
> > >> >
> > >> > etc  
> > >>
> > >>
> > >> Oops, I did not test such architectures.
> > >>
> > >>  
> > >> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> > >> > the linker implicitly adds some runtime libs at the end that get
> > >> > caught up here. Either way I didn't look too far into it because this
> > >> > fix seems obviously correct and solved the problem.
> > >> >
> > >> > Thanks,  
> > >>
> > >>
> > >> Which toolchains do you use?  
> > >
> > > I just had them run through the kbuild 0day tests. Not sure exactly
> > > what they use.
> > >  
> > >> I downloaded xtensa-linux- from this site:
> > >> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
> > >>
> > >>
> > >> I see a different error for xtensa.
> > >>
> > >>
> > >> I applied all of you 5 patches, but xtensa build still fails.
> > >>
> > >>   LD      vmlinux.o
> > >> xtensa-linux-ld: internal error
> > >> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
> > >>
> > >>
> > >> Could you check it?  
> > >
> > > Ahh, the old ld: internal error... what ld version is that?
> > >  
> > 
> > 
> > seems 2.24 according to the following:
> > 
> > 
> > masahiro@pug:~$ xtensa-linux-ld -v
> > GNU ld (GNU Binutils) 2.24
> 
> Ah, thank you. It must have been that the 0day build (cc'ed) did
> not return any error to be from the ld internal error.
> 
> This has led me to the true cause of the error which is the way
> external archives are handled. After implementing the fix, I think
> the lib.a size regression for thin archives should be solved as
> well.

Thanks for addressing this!  Much appreciated.

> This patch should be added between patches 2 and 3 of this series.
> 
> 
> kbuild: handle libs-y archives separately from built-in.o archives
> 
> The thin archives build currently puts all lib.a and built-in.o
> files together and links them with --whole-archive.
> 
> This works because thin archives can recursively refer to thin
> archives. However some architectures include libgcc.a, which may
> not be a thin archive, or it may not be constructed with the "P"
> option, in which case its contents do not get linked correctly.
> 
> So don't pull .a libs into the root built-in.o archive. These
> libs should already have symbol tables and indexes built, so they
> can be direct linker inputs. Move them out of the --whole-archive
> option, which restore the conditional linking behaviour of lib.a
> to thin archives builds.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  Documentation/kbuild/kbuild.txt |  8 +++++--
>  Makefile                        |  7 +++---
>  scripts/link-vmlinux.sh         | 47 ++++++++++++++++++++++++++++++++---------
>  3 files changed, 47 insertions(+), 15 deletions(-)
> 
> diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
> index 0ff6a466a05b..ac2363ea05c5 100644
> --- a/Documentation/kbuild/kbuild.txt
> +++ b/Documentation/kbuild/kbuild.txt
> @@ -236,5 +236,9 @@ Files specified with KBUILD_VMLINUX_INIT are linked first.
>  KBUILD_VMLINUX_MAIN
>  --------------------------------------------------
>  All object files for the main part of vmlinux.
> -KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
> -all the object files used to link vmlinux.
> +
> +KBUILD_VMLINUX_LIBS
> +--------------------------------------------------
> +All .a "lib" files for vmlinux.
> +KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, and KBUILD_VMLINUX_LIBS together
> +specify all the object files used to link vmlinux.
> diff --git a/Makefile b/Makefile
> index 470bd4d9513a..a145a537ad42 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -952,19 +952,20 @@ core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
>  drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
>  net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
>  libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
> -libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
> +libs-y2		:= $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
>  libs-y		:= $(libs-y1) $(libs-y2)
>  virt-y		:= $(patsubst %/, %/built-in.o, $(virt-y))
>  
>  # Externally visible symbols (used by link-vmlinux.sh)
>  export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
> -export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(virt-y)
> +export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
> +export KBUILD_VMLINUX_LIBS := $(libs-y1)
>  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
>  export LDFLAGS_vmlinux
>  # used by scripts/pacmage/Makefile
>  export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) arch Documentation include samples scripts tools)
>  
> -vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
> +vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
>  
>  # Include targets which we want to execute sequentially if the rest of the
>  # kernel build went well. If CONFIG_TRIM_UNUSED_KSYMS is set, this might be
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 2a062ea130b5..e7b7eee31538 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -3,9 +3,12 @@
>  # link vmlinux
>  #
>  # vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
> -# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
> -# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
> -# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
> +# $(KBUILD_VMLINUX_MAIN) and $(KBUILD_VMLINUX_LIBS). Most are built-in.o files
> +# from top-level directories in the kernel tree, others are specified in
> +# arch/$(ARCH)/Makefile. Ordering when linking is important, and
> +# $(KBUILD_VMLINUX_INIT) must be first. $(KBUILD_VMLINUX_LIBS) are archives
> +# which are linked conditionally (not within --whole-archive), and do not
> +# require symbol indexes added.
>  #
>  # vmlinux
>  #   ^
> @@ -16,6 +19,9 @@
>  #   +--< $(KBUILD_VMLINUX_MAIN)
>  #   |    +--< drivers/built-in.o mm/built-in.o + more
>  #   |
> +#   +--< $(KBUILD_VMLINUX_LIBS)
> +#   |    +--< lib/lib.a + more
> +#   |
>  #   +-< ${kallsymso} (see description in KALLSYMS section)
>  #
>  # vmlinux version (uname -v) cannot be updated during normal
> @@ -37,9 +43,10 @@ info()
>  	fi
>  }
>  
> -# Thin archive build here makes a final archive with
> -# symbol table and indexes from vmlinux objects, which can be
> -# used as input to linker.
> +# Thin archive build here makes a final archive with symbol table and indexes
> +# from vmlinux objects INIT and MAIN, which can be used as input to linker.
> +# KBUILD_VMLINUX_LIBS archives should already have symbol table and indexes
> +# added.
>  #
>  # Traditional incremental style of link does not require this step
>  #
> @@ -50,7 +57,7 @@ archive_builtin()
>  	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
>  		info AR built-in.o
>  		rm -f built-in.o;
> -		${AR} rcsT${KBUILD_ARFLAGS} built-in.o			\
> +		${AR} rcsTP${KBUILD_ARFLAGS} built-in.o			\
>  					${KBUILD_VMLINUX_INIT}		\
>  					${KBUILD_VMLINUX_MAIN}
>  	fi
> @@ -63,11 +70,17 @@ modpost_link()
>  	local objects
>  
>  	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -		objects="--whole-archive built-in.o --no-whole-archive"
> +		objects="--whole-archive				\
> +			built-in.o					\
> +			--no-whole-archive				\
> +			--start-group					\
> +			${KBUILD_VMLINUX_LIBS}				\
> +			--end-group"
>  	else
>  		objects="${KBUILD_VMLINUX_INIT}				\
>  			--start-group					\
>  			${KBUILD_VMLINUX_MAIN}				\
> +			${KBUILD_VMLINUX_LIBS}				\
>  			--end-group"
>  	fi
>  	${LD} ${LDFLAGS} -r -o ${1} ${objects}
> @@ -83,11 +96,18 @@ vmlinux_link()
>  
>  	if [ "${SRCARCH}" != "um" ]; then
>  		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -			objects="--whole-archive built-in.o ${1} --no-whole-archive"
> +			objects="--whole-archive			\
> +				built-in.o				\
> +				--no-whole-archive			\
> +				--start-group				\
> +				${KBUILD_VMLINUX_LIBS}			\
> +				--end-group				\
> +				${1}"
>  		else
>  			objects="${KBUILD_VMLINUX_INIT}			\
>  				--start-group				\
>  				${KBUILD_VMLINUX_MAIN}			\
> +				${KBUILD_VMLINUX_LIBS}			\
>  				--end-group				\
>  				${1}"
>  		fi
> @@ -96,11 +116,18 @@ vmlinux_link()
>  			-T ${lds} ${objects}
>  	else
>  		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -			objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
> +			objects="-Wl,--whole-archive			\
> +				built-in.o				\
> +				-Wl,--no-whole-archive			\
> +				-Wl,--start-group			\
> +				${KBUILD_VMLINUX_LIBS}			\
> +				-Wl,--end-group				\
> +				${1}"
>  		else
>  			objects="${KBUILD_VMLINUX_INIT}			\
>  				-Wl,--start-group			\
>  				${KBUILD_VMLINUX_MAIN}			\
> +				${KBUILD_VMLINUX_LIBS}			\
>  				-Wl,--end-group				\
>  				${1}"
>  		fi
> -- 
> 2.11.0
> 

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,

2017-06-20 0:52 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>>
>> masahiro@pug:~$ xtensa-linux-ld -v
>> GNU ld (GNU Binutils) 2.24
>
> Ah, thank you. It must have been that the 0day build (cc'ed) did
> not return any error to be from the ld internal error.
>
> This has led me to the true cause of the error which is the way
> external archives are handled. After implementing the fix, I think
> the lib.a size regression for thin archives should be solved as
> well.


Thanks for figuring this out!


> This patch should be added between patches 2 and 3 of this series.
>

Done.


I applied all 6 patches to linux-kbuild/thin-ar.

I fixed up some:

[1] fix a typo  "tihs"  -> "this"
[2] reword the document as "The build system has, as of 4.13, switched
to using..."
[3] Add "P" to link_vmlinux.sh as well


Could you double-check if I did them correctly?




> --- a/Makefile
> +++ b/Makefile
> @@ -952,19 +952,20 @@ core-y            := $(patsubst %/, %/built-in.o, $(core-y))
>  drivers-y      := $(patsubst %/, %/built-in.o, $(drivers-y))
>  net-y          := $(patsubst %/, %/built-in.o, $(net-y))
>  libs-y1                := $(patsubst %/, %/lib.a, $(libs-y))
> -libs-y2                := $(patsubst %/, %/built-in.o, $(libs-y))
> +libs-y2                := $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
>  libs-y         := $(libs-y1) $(libs-y2)


After I applied this patch, I noticed one more thing.

With this patch, I think "libs-y" will be unnecessary.

If you ack me to fix-up locally, I will remove:
   libs-y         := $(libs-y1) $(libs-y2)




> @@ -50,7 +57,7 @@ archive_builtin()
>         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
>                 info AR built-in.o
>                 rm -f built-in.o;
> -               ${AR} rcsT${KBUILD_ARFLAGS} built-in.o                  \
> +               ${AR} rcsTP${KBUILD_ARFLAGS} built-in.o                 \
>                                         ${KBUILD_VMLINUX_INIT}          \
>                                         ${KBUILD_VMLINUX_MAIN}
>         fi

I moved this hunk to  2/6 "thin archives use P option to ar".


-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 10:17:11 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 2017-06-20 0:52 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> >>
> >> masahiro@pug:~$ xtensa-linux-ld -v
> >> GNU ld (GNU Binutils) 2.24  
> >
> > Ah, thank you. It must have been that the 0day build (cc'ed) did
> > not return any error to be from the ld internal error.
> >
> > This has led me to the true cause of the error which is the way
> > external archives are handled. After implementing the fix, I think
> > the lib.a size regression for thin archives should be solved as
> > well.  
> 
> 
> Thanks for figuring this out!
> 
> 
> > This patch should be added between patches 2 and 3 of this series.
> >  
> 
> Done.
> 
> 
> I applied all 6 patches to linux-kbuild/thin-ar.
> 
> I fixed up some:
> 
> [1] fix a typo  "tihs"  -> "this"
> [2] reword the document as "The build system has, as of 4.13, switched
> to using..."
> [3] Add "P" to link_vmlinux.sh as well
> 
> 
> Could you double-check if I did them correctly?

Yes these look good to me, thank you.

> > --- a/Makefile
> > +++ b/Makefile
> > @@ -952,19 +952,20 @@ core-y            := $(patsubst %/, %/built-in.o, $(core-y))
> >  drivers-y      := $(patsubst %/, %/built-in.o, $(drivers-y))
> >  net-y          := $(patsubst %/, %/built-in.o, $(net-y))
> >  libs-y1                := $(patsubst %/, %/lib.a, $(libs-y))
> > -libs-y2                := $(patsubst %/, %/built-in.o, $(libs-y))
> > +libs-y2                := $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
> >  libs-y         := $(libs-y1) $(libs-y2)  
> 
> 
> After I applied this patch, I noticed one more thing.
> 
> With this patch, I think "libs-y" will be unnecessary.
> 
> If you ack me to fix-up locally, I will remove:
>    libs-y         := $(libs-y1) $(libs-y2)

Yes I suppose that can be removed.

I wonder if the names could be a bit more descriptive?

libs-y-liba
libs-y-builtin

?

> > @@ -50,7 +57,7 @@ archive_builtin()
> >         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> >                 info AR built-in.o
> >                 rm -f built-in.o;
> > -               ${AR} rcsT${KBUILD_ARFLAGS} built-in.o                  \
> > +               ${AR} rcsTP${KBUILD_ARFLAGS} built-in.o                 \
> >                                         ${KBUILD_VMLINUX_INIT}          \
> >                                         ${KBUILD_VMLINUX_MAIN}
> >         fi  
> 
> I moved this hunk to  2/6 "thin archives use P option to ar".

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,


2017-06-21 11:47 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> I wonder if the names could be a bit more descriptive?
>
> libs-y-liba
> libs-y-builtin
>
> ?

I do not have a strong opinion.  Either way sounds OK to me.
If you like, I can fix it up.


I think you mentioned LD_DEAD_CODE_DATA_ELIMINATION is under way for
ARM and x86.
If we can switch all architectures, lib.a will turn into built-in.o.  Correct?
(I am not sure how difficult the global conversion is.)




BTW, I saw abuse of lib.a in
https://patchwork.kernel.org/patch/9768439/

I see it in linux-next.

commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
Author: Stephen Boyd <sboyd@codeaurora.org>
Date:   Fri Jun 2 15:30:06 2017 -0700

    clk: sunxi-ng: Move all clock types to a library




Now drivers/clk/sunxi-ng/lib.a
will go into thin archives.
The result might be different from what they expect...



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 12:29:33 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-21 11:47 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > I wonder if the names could be a bit more descriptive?
> >
> > libs-y-liba
> > libs-y-builtin
> >
> > ?  
> 
> I do not have a strong opinion.  Either way sounds OK to me.
> If you like, I can fix it up.

I have nothing against it if you want to clean it up. You're the
maintainer so whatever you prefer :)

> I think you mentioned LD_DEAD_CODE_DATA_ELIMINATION is under way for
> ARM and x86.

Yes it seems to be quite successful. ARM and MIPS people are looking
at it, I have it working for powerpc and x86 quite easily. I will
send patches to arch maintainers if we get these thin archives changes
merged in the next window.

> If we can switch all architectures, lib.a will turn into built-in.o.  Correct?
> (I am not sure how difficult the global conversion is.)

Yes I think it should be no longer necessary.

The only problem with LD_DEAD_CODE_DATA_ELIMINATION is that it overloads
elf section names to pass data to the linker, so it's not 100% transparent.
In practice it's been very minor changes. On balance it will end up being
better than a whole lot of conditional compiling and linking workarounds
such as the one you mention below.

> BTW, I saw abuse of lib.a in
> https://patchwork.kernel.org/patch/9768439/
> 
> I see it in linux-next.
> 
> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> Author: Stephen Boyd <sboyd@codeaurora.org>
> Date:   Fri Jun 2 15:30:06 2017 -0700
> 
>     clk: sunxi-ng: Move all clock types to a library
> 
> 
> 
> 
> Now drivers/clk/sunxi-ng/lib.a
> will go into thin archives.
> The result might be different from what they expect...

Yes I see. With thin archives, that is just going to cause the
same behaviour as built-in.o (everything will be linked). So the
build should not break, but they won't get savings.

Does it even save space with incremental linking? If the lib.a gets
linked into drivers/built-in.o, I wonder what happens then?

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 12:29:33 +0900
> Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

>> BTW, I saw abuse of lib.a in
>> https://patchwork.kernel.org/patch/9768439/
>>
>> I see it in linux-next.
>>
>> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> Author: Stephen Boyd <sboyd@codeaurora.org>
>> Date:   Fri Jun 2 15:30:06 2017 -0700
>>
>>     clk: sunxi-ng: Move all clock types to a library
>>
>>
>>
>>
>> Now drivers/clk/sunxi-ng/lib.a
>> will go into thin archives.
>> The result might be different from what they expect...
>
> Yes I see. With thin archives, that is just going to cause the
> same behaviour as built-in.o (everything will be linked). So the
> build should not break, but they won't get savings.
>
> Does it even save space with incremental linking? If the lib.a gets
> linked into drivers/built-in.o, I wonder what happens then?

Ah, too bad. I thought we had found a way to use a library correctly
here, but I just verified that indeed all the just gets linked into built-in.o

I played around with it some more now, but without success: if I
build sunxi-ng as a loadable module (using a few modifications),
then the unneeded objects from lib.a are dropped as I had hoped,
but for built-in code we now always include everything.

I suppose that we can ignore this once we get
LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
until then, we have a code size regression. Any other ideas for
how this could be solved? We used to have a Kconfig symbol
for each file, but those would frequently get out of sync.

         Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 09:15:09 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 12:29:33 +0900
> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:  
> 
> >> BTW, I saw abuse of lib.a in
> >> https://patchwork.kernel.org/patch/9768439/
> >>
> >> I see it in linux-next.
> >>
> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> Author: Stephen Boyd <sboyd@codeaurora.org>
> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >>
> >>     clk: sunxi-ng: Move all clock types to a library
> >>
> >>
> >>
> >>
> >> Now drivers/clk/sunxi-ng/lib.a
> >> will go into thin archives.
> >> The result might be different from what they expect...  
> >
> > Yes I see. With thin archives, that is just going to cause the
> > same behaviour as built-in.o (everything will be linked). So the
> > build should not break, but they won't get savings.
> >
> > Does it even save space with incremental linking? If the lib.a gets
> > linked into drivers/built-in.o, I wonder what happens then?  
> 
> Ah, too bad. I thought we had found a way to use a library correctly
> here, but I just verified that indeed all the just gets linked into built-in.o
> 
> I played around with it some more now, but without success: if I
> build sunxi-ng as a loadable module (using a few modifications),
> then the unneeded objects from lib.a are dropped as I had hoped,
> but for built-in code we now always include everything.
> 
> I suppose that we can ignore this once we get
> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> until then, we have a code size regression.

I didn't follow the thread there, is it a regression caused by
thin archives, or just by removing the Kconfig symbol from each
file?

> Any other ideas for
> how this could be solved? We used to have a Kconfig symbol
> for each file, but those would frequently get out of sync.

Not really sure. Possibly the way to do it would be to plumb
lib-y through drivers/ as well (like obj-y is) so they can be
linked on their own for the final vmlinux link.

But that may be quite a bit of churn, so it may be better to
hold such changes until we see how LD_DEAD_CODE_DATA_ELIMINATION
works out.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 09:15:09 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> > On Wed, 21 Jun 2017 12:29:33 +0900
>> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>>
>> >> BTW, I saw abuse of lib.a in
>> >> https://patchwork.kernel.org/patch/9768439/
>> >>
>> >> I see it in linux-next.
>> >>
>> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> >> Author: Stephen Boyd <sboyd@codeaurora.org>
>> >> Date:   Fri Jun 2 15:30:06 2017 -0700
>> >>
>> >>     clk: sunxi-ng: Move all clock types to a library
>> >>
>> >>
>> >>
>> >>
>> >> Now drivers/clk/sunxi-ng/lib.a
>> >> will go into thin archives.
>> >> The result might be different from what they expect...
>> >
>> > Yes I see. With thin archives, that is just going to cause the
>> > same behaviour as built-in.o (everything will be linked). So the
>> > build should not break, but they won't get savings.
>> >
>> > Does it even save space with incremental linking? If the lib.a gets
>> > linked into drivers/built-in.o, I wonder what happens then?
>>
>> Ah, too bad. I thought we had found a way to use a library correctly
>> here, but I just verified that indeed all the just gets linked into built-in.o
>>
>> I played around with it some more now, but without success: if I
>> build sunxi-ng as a loadable module (using a few modifications),
>> then the unneeded objects from lib.a are dropped as I had hoped,
>> but for built-in code we now always include everything.
>>
>> I suppose that we can ignore this once we get
>> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
>> until then, we have a code size regression.
>
> I didn't follow the thread there, is it a regression caused by
> thin archives, or just by removing the Kconfig symbol from each
> file?

I thought it was the latter, but actually it only happens with thin
archives, so we are fine as long as we enable THIN_ARCHIVES
and LD_DEAD_CODE_DATA_ELIMINATION at the same time
on ARM.

        Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 12:21:16 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 09:15:09 +0200
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> > On Wed, 21 Jun 2017 12:29:33 +0900
> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:  
> >>  
> >> >> BTW, I saw abuse of lib.a in
> >> >> https://patchwork.kernel.org/patch/9768439/
> >> >>
> >> >> I see it in linux-next.
> >> >>
> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> >> Author: Stephen Boyd <sboyd@codeaurora.org>
> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >> >>
> >> >>     clk: sunxi-ng: Move all clock types to a library
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Now drivers/clk/sunxi-ng/lib.a
> >> >> will go into thin archives.
> >> >> The result might be different from what they expect...  
> >> >
> >> > Yes I see. With thin archives, that is just going to cause the
> >> > same behaviour as built-in.o (everything will be linked). So the
> >> > build should not break, but they won't get savings.
> >> >
> >> > Does it even save space with incremental linking? If the lib.a gets
> >> > linked into drivers/built-in.o, I wonder what happens then?  
> >>
> >> Ah, too bad. I thought we had found a way to use a library correctly
> >> here, but I just verified that indeed all the just gets linked into built-in.o
> >>
> >> I played around with it some more now, but without success: if I
> >> build sunxi-ng as a loadable module (using a few modifications),
> >> then the unneeded objects from lib.a are dropped as I had hoped,
> >> but for built-in code we now always include everything.
> >>
> >> I suppose that we can ignore this once we get
> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> >> until then, we have a code size regression.  
> >
> > I didn't follow the thread there, is it a regression caused by
> > thin archives, or just by removing the Kconfig symbol from each
> > file?  
> 
> I thought it was the latter, but actually it only happens with thin
> archives, 

Is this including these changes now in the kbuild tree?

I can take a look at ARM and try to get it at least to parity with
incremental link. Any particular config options required?

> so we are fine as long as we enable THIN_ARCHIVES
> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
> on ARM.

Well the current proposal is to unconditionally enable it for all archs
for 4.13. After that I'll submit patches to x86 and powerpc arch
maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
you will do ARM and there have been MIPS guys looking at it too.

That leaves a window of one release. ARM could unselect thin archives if
necessary but I think it would be much better to enable it and flush out
any toolchain and build issues before doing the LD_DCDE option. Disabling
should be a last resort if we can't fix something in time for release.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 12:21:16 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> > On Wed, 21 Jun 2017 09:15:09 +0200
>> > Arnd Bergmann <arnd@arndb.de> wrote:
>> >
>> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> >> > On Wed, 21 Jun 2017 12:29:33 +0900
>> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>> >>
>> >> >> BTW, I saw abuse of lib.a in
>> >> >> https://patchwork.kernel.org/patch/9768439/
>> >> >>
>> >> >> I see it in linux-next.
>> >> >>
>> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> >> >> Author: Stephen Boyd <sboyd@codeaurora.org>
>> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
>> >> >>
>> >> >>     clk: sunxi-ng: Move all clock types to a library
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> Now drivers/clk/sunxi-ng/lib.a
>> >> >> will go into thin archives.
>> >> >> The result might be different from what they expect...
>> >> >
>> >> > Yes I see. With thin archives, that is just going to cause the
>> >> > same behaviour as built-in.o (everything will be linked). So the
>> >> > build should not break, but they won't get savings.
>> >> >
>> >> > Does it even save space with incremental linking? If the lib.a gets
>> >> > linked into drivers/built-in.o, I wonder what happens then?
>> >>
>> >> Ah, too bad. I thought we had found a way to use a library correctly
>> >> here, but I just verified that indeed all the just gets linked into built-in.o
>> >>
>> >> I played around with it some more now, but without success: if I
>> >> build sunxi-ng as a loadable module (using a few modifications),
>> >> then the unneeded objects from lib.a are dropped as I had hoped,
>> >> but for built-in code we now always include everything.
>> >>
>> >> I suppose that we can ignore this once we get
>> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
>> >> until then, we have a code size regression.
>> >
>> > I didn't follow the thread there, is it a regression caused by
>> > thin archives, or just by removing the Kconfig symbol from each
>> > file?
>>
>> I thought it was the latter, but actually it only happens with thin
>> archives,
>
> Is this including these changes now in the kbuild tree?

I'm building on top of yesterday's linux-next at the moment,
with a number of my own patches applied

> I can take a look at ARM and try to get it at least to parity with
> incremental link. Any particular config options required?

This is the patch I am testing with:

https://pastebin.com/HQuhCEmK

I have not looked at that in a while, no idea if it works, or
if it has known problems.

I last posted the patch in March for discussion:

https://patchwork.kernel.org/patch/9626207/

>> so we are fine as long as we enable THIN_ARCHIVES
>> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
>> on ARM.
>
> Well the current proposal is to unconditionally enable it for all archs
> for 4.13. After that I'll submit patches to x86 and powerpc arch
> maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
> you will do ARM and there have been MIPS guys looking at it too.
>
> That leaves a window of one release. ARM could unselect thin archives if
> necessary but I think it would be much better to enable it and flush out
> any toolchain and build issues before doing the LD_DCDE option. Disabling
> should be a last resort if we can't fix something in time for release.

I see.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 12:49 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> On Wed, 21 Jun 2017 12:21:16 +0200
>> Arnd Bergmann <arnd@arndb.de> wrote:

>>> so we are fine as long as we enable THIN_ARCHIVES
>>> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
>>> on ARM.
>>
>> Well the current proposal is to unconditionally enable it for all archs
>> for 4.13. After that I'll submit patches to x86 and powerpc arch
>> maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
>> you will do ARM and there have been MIPS guys looking at it too.
>>
>> That leaves a window of one release. ARM could unselect thin archives if
>> necessary but I think it would be much better to enable it and flush out
>> any toolchain and build issues before doing the LD_DCDE option. Disabling
>> should be a last resort if we can't fix something in time for release.
>
> I see.

[hit 'send' too early]

We can always revert 06e226c7fb23 ("clk: sunxi-ng: Move all clock types to a
library"), it's also scheduled for 4.13 and has not been in a release yet.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 12:49:10 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 12:21:16 +0200
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> > On Wed, 21 Jun 2017 09:15:09 +0200
> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> >> >  
> >> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> >> > On Wed, 21 Jun 2017 12:29:33 +0900
> >> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:  
> >> >>  
> >> >> >> BTW, I saw abuse of lib.a in
> >> >> >> https://patchwork.kernel.org/patch/9768439/
> >> >> >>
> >> >> >> I see it in linux-next.
> >> >> >>
> >> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> >> >> Author: Stephen Boyd <sboyd@codeaurora.org>
> >> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >> >> >>
> >> >> >>     clk: sunxi-ng: Move all clock types to a library
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Now drivers/clk/sunxi-ng/lib.a
> >> >> >> will go into thin archives.
> >> >> >> The result might be different from what they expect...  
> >> >> >
> >> >> > Yes I see. With thin archives, that is just going to cause the
> >> >> > same behaviour as built-in.o (everything will be linked). So the
> >> >> > build should not break, but they won't get savings.
> >> >> >
> >> >> > Does it even save space with incremental linking? If the lib.a gets
> >> >> > linked into drivers/built-in.o, I wonder what happens then?  
> >> >>
> >> >> Ah, too bad. I thought we had found a way to use a library correctly
> >> >> here, but I just verified that indeed all the just gets linked into built-in.o
> >> >>
> >> >> I played around with it some more now, but without success: if I
> >> >> build sunxi-ng as a loadable module (using a few modifications),
> >> >> then the unneeded objects from lib.a are dropped as I had hoped,
> >> >> but for built-in code we now always include everything.
> >> >>
> >> >> I suppose that we can ignore this once we get
> >> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> >> >> until then, we have a code size regression.  
> >> >
> >> > I didn't follow the thread there, is it a regression caused by
> >> > thin archives, or just by removing the Kconfig symbol from each
> >> > file?  
> >>
> >> I thought it was the latter, but actually it only happens with thin
> >> archives,  
> >
> > Is this including these changes now in the kbuild tree?  
> 
> I'm building on top of yesterday's linux-next at the moment,
> with a number of my own patches applied
> 
> > I can take a look at ARM and try to get it at least to parity with
> > incremental link. Any particular config options required?  
> 
> This is the patch I am testing with:
> 
> https://pastebin.com/HQuhCEmK
> I have not looked at that in a while, no idea if it works, or
> if it has known problems.
> 
> I last posted the patch in March for discussion:
> 
> https://patchwork.kernel.org/patch/9626207/

Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
Just want to try getting thin archives up to a point where there are
no serious regressions first.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 12:49:10 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> > On Wed, 21 Jun 2017 12:21:16 +0200
>> > Arnd Bergmann <arnd@arndb.de> wrote:
>> >
>>
>> I'm building on top of yesterday's linux-next at the moment,
>> with a number of my own patches applied
>>
>> > I can take a look at ARM and try to get it at least to parity with
>> > incremental link. Any particular config options required?
>>
>> This is the patch I am testing with:
>>
>> https://pastebin.com/HQuhCEmK
>> I have not looked at that in a while, no idea if it works, or
>> if it has known problems.
>>
>> I last posted the patch in March for discussion:
>>
>> https://patchwork.kernel.org/patch/9626207/
>
> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> Just want to try getting thin archives up to a point where there are
> no serious regressions first.

For my build testing, I have now reverted most of my patch and only
left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

         Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 13:32:17 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 12:49:10 +0200
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> >> >  
> >>
> >> I'm building on top of yesterday's linux-next at the moment,
> >> with a number of my own patches applied
> >>  
> >> > I can take a look at ARM and try to get it at least to parity with
> >> > incremental link. Any particular config options required?  
> >>
> >> This is the patch I am testing with:
> >>
> >> https://pastebin.com/HQuhCEmK
> >> I have not looked at that in a while, no idea if it works, or
> >> if it has known problems.
> >>
> >> I last posted the patch in March for discussion:
> >>
> >> https://patchwork.kernel.org/patch/9626207/  
> >
> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> > Just want to try getting thin archives up to a point where there are
> > no serious regressions first.  
> 
> For my build testing, I have now reverted most of my patch and only
> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

Okay. Currently with the patches in the kbuild tree, thin archives
actually does better than incremental linking for ARM (defconfig).

11651574	6090952	 418616	18161142	1151df6	vmlinux.thinarc
11656118	6095208	 418952	18170278	11541a6	vmlinux.inc

In mainline, thin archives is sub-optimal for lib-y libraries (just
pulls them all in). But now that I test it, even that is better than
incremental linking for ARM:

11653926	6090888	 418620	18163434	11526ea	vmlinux.thinarc

I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
patch yet to check. What tree is it in? I wonder if I could convince you
to drop that for now and bring it up on linux-kernel/linux-kbuild? I
think it's quite reasonable to reduce Kconfig hell of very fine grained
conditional compilation/linking if we can make the toolchain do it for
us. So we should consider how to support it nicely rather than having
such hacks IMO.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 13:32:17 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> > On Wed, 21 Jun 2017 12:49:10 +0200
>> > Arnd Bergmann <arnd@arndb.de> wrote:
>> >
>> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> >> > On Wed, 21 Jun 2017 12:21:16 +0200
>> >> > Arnd Bergmann <arnd@arndb.de> wrote:
>> >> >
>> >>
>> >> I'm building on top of yesterday's linux-next at the moment,
>> >> with a number of my own patches applied
>> >>
>> >> > I can take a look at ARM and try to get it at least to parity with
>> >> > incremental link. Any particular config options required?
>> >>
>> >> This is the patch I am testing with:
>> >>
>> >> https://pastebin.com/HQuhCEmK
>> >> I have not looked at that in a while, no idea if it works, or
>> >> if it has known problems.
>> >>
>> >> I last posted the patch in March for discussion:
>> >>
>> >> https://patchwork.kernel.org/patch/9626207/
>> >
>> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
>> > Just want to try getting thin archives up to a point where there are
>> > no serious regressions first.
>>
>> For my build testing, I have now reverted most of my patch and only
>> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
>
> Okay. Currently with the patches in the kbuild tree, thin archives
> actually does better than incremental linking for ARM (defconfig).
>
> 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
>
> In mainline, thin archives is sub-optimal for lib-y libraries (just
> pulls them all in). But now that I test it, even that is better than
> incremental linking for ARM:
>
> 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
>
> I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> patch yet to check. What tree is it in? I wonder if I could convince you
> to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> think it's quite reasonable to reduce Kconfig hell of very fine grained
> conditional compilation/linking if we can make the toolchain do it for
> us. So we should consider how to support it nicely rather than having
> such hacks IMO.

[adding Maxime and Stephen to cc]

Stephen,

The hack has come to bite us after all, with Nick's proposed change to
CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
end up being linked into each kernel that enables the directory.

I'm out of other ideas for fixing it, so it seems better to revert back to
individual Kconfig options. Once we enable
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
next step after CONFIG_THIN_ARCHIVES), we can simply list
all files, and the linker will drop all unused functions by itself.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Stephen Boyd]

On 06/21, Arnd Bergmann wrote:
> On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 13:32:17 +0200
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >
> >> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> >> > On Wed, 21 Jun 2017 12:49:10 +0200
> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> >> >
> >> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> >> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> >> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> >> >> >
> >> >>
> >> >> I'm building on top of yesterday's linux-next at the moment,
> >> >> with a number of my own patches applied
> >> >>
> >> >> > I can take a look at ARM and try to get it at least to parity with
> >> >> > incremental link. Any particular config options required?
> >> >>
> >> >> This is the patch I am testing with:
> >> >>
> >> >> https://pastebin.com/HQuhCEmK
> >> >> I have not looked at that in a while, no idea if it works, or
> >> >> if it has known problems.
> >> >>
> >> >> I last posted the patch in March for discussion:
> >> >>
> >> >> https://patchwork.kernel.org/patch/9626207/
> >> >
> >> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> >> > Just want to try getting thin archives up to a point where there are
> >> > no serious regressions first.
> >>
> >> For my build testing, I have now reverted most of my patch and only
> >> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
> >
> > Okay. Currently with the patches in the kbuild tree, thin archives
> > actually does better than incremental linking for ARM (defconfig).
> >
> > 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> > 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
> >
> > In mainline, thin archives is sub-optimal for lib-y libraries (just
> > pulls them all in). But now that I test it, even that is better than
> > incremental linking for ARM:
> >
> > 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
> >
> > I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> > patch yet to check. What tree is it in? I wonder if I could convince you
> > to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> > think it's quite reasonable to reduce Kconfig hell of very fine grained
> > conditional compilation/linking if we can make the toolchain do it for
> > us. So we should consider how to support it nicely rather than having
> > such hacks IMO.
> 
> [adding Maxime and Stephen to cc]
> 
> Stephen,
> 
> The hack has come to bite us after all, with Nick's proposed change to
> CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
> end up being linked into each kernel that enables the directory.
> 
> I'm out of other ideas for fixing it, so it seems better to revert back to
> individual Kconfig options. Once we enable
> CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
> next step after CONFIG_THIN_ARCHIVES), we can simply list
> all files, and the linker will drop all unused functions by itself.
> 

Ok. Can you send a revert patch to the list with some information
on why we can't do the hack anymore and also Cc lkml/kbuild
lists? The commit is in linux-next now as commit 06e226c7fb23
(clk: sunxi-ng: Move all clock types to a library, 2017-06-02).

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 09:19:13 -0700
Stephen Boyd <sboyd@codeaurora.org> wrote:

> On 06/21, Arnd Bergmann wrote:
> > On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> > > On Wed, 21 Jun 2017 13:32:17 +0200
> > > Arnd Bergmann <arnd@arndb.de> wrote:
> > >  
> > >> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> > >> > On Wed, 21 Jun 2017 12:49:10 +0200
> > >> > Arnd Bergmann <arnd@arndb.de> wrote:
> > >> >  
> > >> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> > >> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> > >> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> > >> >> >  
> > >> >>
> > >> >> I'm building on top of yesterday's linux-next at the moment,
> > >> >> with a number of my own patches applied
> > >> >>  
> > >> >> > I can take a look at ARM and try to get it at least to parity with
> > >> >> > incremental link. Any particular config options required?  
> > >> >>
> > >> >> This is the patch I am testing with:
> > >> >>
> > >> >> https://pastebin.com/HQuhCEmK
> > >> >> I have not looked at that in a while, no idea if it works, or
> > >> >> if it has known problems.
> > >> >>
> > >> >> I last posted the patch in March for discussion:
> > >> >>
> > >> >> https://patchwork.kernel.org/patch/9626207/  
> > >> >
> > >> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> > >> > Just want to try getting thin archives up to a point where there are
> > >> > no serious regressions first.  
> > >>
> > >> For my build testing, I have now reverted most of my patch and only
> > >> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.  
> > >
> > > Okay. Currently with the patches in the kbuild tree, thin archives
> > > actually does better than incremental linking for ARM (defconfig).
> > >
> > > 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> > > 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
> > >
> > > In mainline, thin archives is sub-optimal for lib-y libraries (just
> > > pulls them all in). But now that I test it, even that is better than
> > > incremental linking for ARM:
> > >
> > > 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
> > >
> > > I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> > > patch yet to check. What tree is it in? I wonder if I could convince you
> > > to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> > > think it's quite reasonable to reduce Kconfig hell of very fine grained
> > > conditional compilation/linking if we can make the toolchain do it for
> > > us. So we should consider how to support it nicely rather than having
> > > such hacks IMO.  
> > 
> > [adding Maxime and Stephen to cc]
> > 
> > Stephen,
> > 
> > The hack has come to bite us after all, with Nick's proposed change to
> > CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
> > end up being linked into each kernel that enables the directory.
> > 
> > I'm out of other ideas for fixing it, so it seems better to revert back to
> > individual Kconfig options. Once we enable
> > CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
> > next step after CONFIG_THIN_ARCHIVES), we can simply list
> > all files, and the linker will drop all unused functions by itself.
> >   
> 
> Ok. Can you send a revert patch to the list with some information
> on why we can't do the hack anymore and also Cc lkml/kbuild
> lists? The commit is in linux-next now as commit 06e226c7fb23
> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> 

I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
with thin archives enabled and disabled with arm defconfig which ends up
setting CONFIG_SUNXI_CCU=y.

The patch makes no difference to vmlinux size whether using traditional
incremental link, or thin archives (there is a small difference between
inclink and thinarc but that's unrelated).

So this thin archives change does not break your patch, it's just that
it doesn't really do the right thing. I like the general idea, but we
need to work out how to make it properly supported by the build system.

With the patch applied, this is what the build system currently does:

arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o

This incremental link pulls in all your lib.a objects and links them
into built-in.o. They can no longer be selectively linked.

Thanks,
Nick

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 09:19:13 -0700
> Stephen Boyd <sboyd@codeaurora.org> wrote:
>> On 06/21, Arnd Bergmann wrote:
>>
>> Ok. Can you send a revert patch to the list with some information
>> on why we can't do the hack anymore and also Cc lkml/kbuild
>> lists? The commit is in linux-next now as commit 06e226c7fb23
>> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
>>
>
> I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> with thin archives enabled and disabled with arm defconfig which ends up
> setting CONFIG_SUNXI_CCU=y.
>
> The patch makes no difference to vmlinux size whether using traditional
> incremental link, or thin archives (there is a small difference between
> inclink and thinarc but that's unrelated).
>
> So this thin archives change does not break your patch, it's just that
> it doesn't really do the right thing. I like the general idea, but we
> need to work out how to make it properly supported by the build system.
>
> With the patch applied, this is what the build system currently does:
>
> arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
>
> This incremental link pulls in all your lib.a objects and links them
> into built-in.o. They can no longer be selectively linked.

I think the ARM defconfig actually needs all those objects because it
enables all the high-level drivers. You could try disabling e.g. all except
CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
unused.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2367 bytes --]

On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 09:19:13 -0700
> > Stephen Boyd <sboyd@codeaurora.org> wrote:
> >> On 06/21, Arnd Bergmann wrote:
> >>
> >> Ok. Can you send a revert patch to the list with some information
> >> on why we can't do the hack anymore and also Cc lkml/kbuild
> >> lists? The commit is in linux-next now as commit 06e226c7fb23
> >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> >>
> >
> > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> > with thin archives enabled and disabled with arm defconfig which ends up
> > setting CONFIG_SUNXI_CCU=y.
> >
> > The patch makes no difference to vmlinux size whether using traditional
> > incremental link, or thin archives (there is a small difference between
> > inclink and thinarc but that's unrelated).
> >
> > So this thin archives change does not break your patch, it's just that
> > it doesn't really do the right thing. I like the general idea, but we
> > need to work out how to make it properly supported by the build system.
> >
> > With the patch applied, this is what the build system currently does:
> >
> > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> >
> > This incremental link pulls in all your lib.a objects and links them
> > into built-in.o. They can no longer be selectively linked.
> 
> I think the ARM defconfig actually needs all those objects because it
> enables all the high-level drivers. You could try disabling e.g. all except
> CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> unused.

You can also disable MACH_SUN8I, it should disable a significant
number of the clocks driver.

Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Thu, 22 Jun 2017 08:18:38 +0200
Maxime Ripard <maxime.ripard@free-electrons.com> wrote:

> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> > > On Wed, 21 Jun 2017 09:19:13 -0700
> > > Stephen Boyd <sboyd@codeaurora.org> wrote:  
> > >> On 06/21, Arnd Bergmann wrote:
> > >>
> > >> Ok. Can you send a revert patch to the list with some information
> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> > >>  
> > >
> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> > > with thin archives enabled and disabled with arm defconfig which ends up
> > > setting CONFIG_SUNXI_CCU=y.
> > >
> > > The patch makes no difference to vmlinux size whether using traditional
> > > incremental link, or thin archives (there is a small difference between
> > > inclink and thinarc but that's unrelated).
> > >
> > > So this thin archives change does not break your patch, it's just that
> > > it doesn't really do the right thing. I like the general idea, but we
> > > need to work out how to make it properly supported by the build system.
> > >
> > > With the patch applied, this is what the build system currently does:
> > >
> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> > >
> > > This incremental link pulls in all your lib.a objects and links them
> > > into built-in.o. They can no longer be selectively linked.  
> > 
> > I think the ARM defconfig actually needs all those objects because it
> > enables all the high-level drivers. You could try disabling e.g. all except
> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> > unused.  
> 
> You can also disable MACH_SUN8I, it should disable a significant
> number of the clocks driver.

   text	   data	    bss	    dec	    hex	filename
11655078	6018736	 418944	18092758	11412d6	vmlinux.inclink
11655078	6018736	 418944	18092758	11412d6	vmlinux.inclink.clk
11650534	6010288	 418616	18079438	113dece	vmlinux.thinarc
11650534	6010288	 418616	18079438	113dece	vmlinux.thinarc.clk

Same result for me when compiling defconfig minus MACH_SUN8I, comparing
+/- the clock patch and thin archives. No size savings. As I explained
already, I really can't see how there could be any saving if the lib.a is
incrementally linked into built-in.o.

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

Hi Nicholas,

2017-06-23 0:50 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> On Thu, 22 Jun 2017 08:18:38 +0200
> Maxime Ripard <maxime.ripard@free-electrons.com> wrote:
>
>> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
>> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> > > On Wed, 21 Jun 2017 09:19:13 -0700
>> > > Stephen Boyd <sboyd@codeaurora.org> wrote:
>> > >> On 06/21, Arnd Bergmann wrote:
>> > >>
>> > >> Ok. Can you send a revert patch to the list with some information
>> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
>> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
>> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
>> > >>
>> > >
>> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
>> > > with thin archives enabled and disabled with arm defconfig which ends up
>> > > setting CONFIG_SUNXI_CCU=y.
>> > >
>> > > The patch makes no difference to vmlinux size whether using traditional
>> > > incremental link, or thin archives (there is a small difference between
>> > > inclink and thinarc but that's unrelated).
>> > >
>> > > So this thin archives change does not break your patch, it's just that
>> > > it doesn't really do the right thing. I like the general idea, but we
>> > > need to work out how to make it properly supported by the build system.
>> > >
>> > > With the patch applied, this is what the build system currently does:
>> > >
>> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
>> > >
>> > > This incremental link pulls in all your lib.a objects and links them
>> > > into built-in.o. They can no longer be selectively linked.
>> >
>> > I think the ARM defconfig actually needs all those objects because it
>> > enables all the high-level drivers. You could try disabling e.g. all except
>> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
>> > unused.
>>
>> You can also disable MACH_SUN8I, it should disable a significant
>> number of the clocks driver.
>
>    text    data     bss     dec     hex filename
> 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink
> 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink.clk
> 11650534        6010288  418616 18079438        113dece vmlinux.thinarc
> 11650534        6010288  418616 18079438        113dece vmlinux.thinarc.clk
>
> Same result for me when compiling defconfig minus MACH_SUN8I, comparing
> +/- the clock patch and thin archives. No size savings. As I explained
> already, I really can't see how there could be any saving if the lib.a is
> incrementally linked into built-in.o.

I think defconfig minus MACH_SUN8I
will give no difference to CONFIG_SUNXI_CCU*

As far as I tried,  minus MACH_SUN8I still enabled the following:
CONFIG_SUNXI_CCU=y
CONFIG_SUNXI_CCU_DIV=y
CONFIG_SUNXI_CCU_FRAC=y
CONFIG_SUNXI_CCU_GATE=y
CONFIG_SUNXI_CCU_MUX=y
CONFIG_SUNXI_CCU_MULT=y
CONFIG_SUNXI_CCU_PHASE=y
CONFIG_SUNXI_CCU_NK=y
CONFIG_SUNXI_CCU_NKM=y
CONFIG_SUNXI_CCU_NKMP=y
CONFIG_SUNXI_CCU_NM=y
CONFIG_SUNXI_CCU_MP=y

I think you need to disable some more MACH_SUN*I and
perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
then you will see difference in the result.





-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 4359 bytes --]

On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
> Hi Nicholas,
> 
> 2017-06-23 0:50 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > On Thu, 22 Jun 2017 08:18:38 +0200
> > Maxime Ripard <maxime.ripard@free-electrons.com> wrote:
> >
> >> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> >> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> >> > > On Wed, 21 Jun 2017 09:19:13 -0700
> >> > > Stephen Boyd <sboyd@codeaurora.org> wrote:
> >> > >> On 06/21, Arnd Bergmann wrote:
> >> > >>
> >> > >> Ok. Can you send a revert patch to the list with some information
> >> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
> >> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
> >> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> >> > >>
> >> > >
> >> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> >> > > with thin archives enabled and disabled with arm defconfig which ends up
> >> > > setting CONFIG_SUNXI_CCU=y.
> >> > >
> >> > > The patch makes no difference to vmlinux size whether using traditional
> >> > > incremental link, or thin archives (there is a small difference between
> >> > > inclink and thinarc but that's unrelated).
> >> > >
> >> > > So this thin archives change does not break your patch, it's just that
> >> > > it doesn't really do the right thing. I like the general idea, but we
> >> > > need to work out how to make it properly supported by the build system.
> >> > >
> >> > > With the patch applied, this is what the build system currently does:
> >> > >
> >> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> >> > >
> >> > > This incremental link pulls in all your lib.a objects and links them
> >> > > into built-in.o. They can no longer be selectively linked.
> >> >
> >> > I think the ARM defconfig actually needs all those objects because it
> >> > enables all the high-level drivers. You could try disabling e.g. all except
> >> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> >> > unused.
> >>
> >> You can also disable MACH_SUN8I, it should disable a significant
> >> number of the clocks driver.
> >
> >    text    data     bss     dec     hex filename
> > 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink
> > 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink.clk
> > 11650534        6010288  418616 18079438        113dece vmlinux.thinarc
> > 11650534        6010288  418616 18079438        113dece vmlinux.thinarc.clk
> >
> > Same result for me when compiling defconfig minus MACH_SUN8I, comparing
> > +/- the clock patch and thin archives. No size savings. As I explained
> > already, I really can't see how there could be any saving if the lib.a is
> > incrementally linked into built-in.o.
> 
> I think defconfig minus MACH_SUN8I
> will give no difference to CONFIG_SUNXI_CCU*
> 
> As far as I tried,  minus MACH_SUN8I still enabled the following:
> CONFIG_SUNXI_CCU=y
> CONFIG_SUNXI_CCU_DIV=y
> CONFIG_SUNXI_CCU_FRAC=y
> CONFIG_SUNXI_CCU_GATE=y
> CONFIG_SUNXI_CCU_MUX=y
> CONFIG_SUNXI_CCU_MULT=y
> CONFIG_SUNXI_CCU_PHASE=y
> CONFIG_SUNXI_CCU_NK=y
> CONFIG_SUNXI_CCU_NKM=y
> CONFIG_SUNXI_CCU_NKMP=y
> CONFIG_SUNXI_CCU_NM=y
> CONFIG_SUNXI_CCU_MP=y
> 
> I think you need to disable some more MACH_SUN*I and
> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
> then you will see difference in the result.

Ah, right, it's all selected by the rest.

I guess to get a meaningful example you could disable all the
SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
SUN8I_R_CCU.

Most of the clock types shouldn't be used but div and gates.

Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
<maxime.ripard@free-electrons.com> wrote:
> On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
>> As far as I tried,  minus MACH_SUN8I still enabled the following:
>> CONFIG_SUNXI_CCU=y
>> CONFIG_SUNXI_CCU_DIV=y
>> CONFIG_SUNXI_CCU_FRAC=y
>> CONFIG_SUNXI_CCU_GATE=y
>> CONFIG_SUNXI_CCU_MUX=y
>> CONFIG_SUNXI_CCU_MULT=y
>> CONFIG_SUNXI_CCU_PHASE=y
>> CONFIG_SUNXI_CCU_NK=y
>> CONFIG_SUNXI_CCU_NKM=y
>> CONFIG_SUNXI_CCU_NKMP=y
>> CONFIG_SUNXI_CCU_NM=y
>> CONFIG_SUNXI_CCU_MP=y
>>
>> I think you need to disable some more MACH_SUN*I and
>> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
>> then you will see difference in the result.
>
> Ah, right, it's all selected by the rest.
>
> I guess to get a meaningful example you could disable all the
> SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
> SUN8I_R_CCU.
>
> Most of the clock types shouldn't be used but div and gates.

Given that we appear to basically always pull in all (or most) of
the CCU parts anyway, and it's hard to even test with a subset
being disabled, I wonder how much we care about dropping
the unused objects at all:

Maybe the answer is that we just always build all of them into
the kernel and make only the SoC-specific objects configurable ;-)

Once we start linking with  --gc-sections, it will all be fine anyway.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Masahiro Yamada]

2017-06-24 0:04 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
> <maxime.ripard@free-electrons.com> wrote:
>> On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
>>> As far as I tried,  minus MACH_SUN8I still enabled the following:
>>> CONFIG_SUNXI_CCU=y
>>> CONFIG_SUNXI_CCU_DIV=y
>>> CONFIG_SUNXI_CCU_FRAC=y
>>> CONFIG_SUNXI_CCU_GATE=y
>>> CONFIG_SUNXI_CCU_MUX=y
>>> CONFIG_SUNXI_CCU_MULT=y
>>> CONFIG_SUNXI_CCU_PHASE=y
>>> CONFIG_SUNXI_CCU_NK=y
>>> CONFIG_SUNXI_CCU_NKM=y
>>> CONFIG_SUNXI_CCU_NKMP=y
>>> CONFIG_SUNXI_CCU_NM=y
>>> CONFIG_SUNXI_CCU_MP=y
>>>
>>> I think you need to disable some more MACH_SUN*I and
>>> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
>>> then you will see difference in the result.
>>
>> Ah, right, it's all selected by the rest.
>>
>> I guess to get a meaningful example you could disable all the
>> SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
>> SUN8I_R_CCU.
>>
>> Most of the clock types shouldn't be used but div and gates.
>
> Given that we appear to basically always pull in all (or most) of
> the CCU parts anyway, and it's hard to even test with a subset
> being disabled, I wonder how much we care about dropping
> the unused objects at all:
>
> Maybe the answer is that we just always build all of them into
> the kernel and make only the SoC-specific objects configurable ;-)
>
> Once we start linking with  --gc-sections, it will all be fine anyway.
>

I agree!




-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1815 bytes --]

On Fri, Jun 23, 2017 at 05:04:45PM +0200, Arnd Bergmann wrote:
> On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
> <maxime.ripard@free-electrons.com> wrote:
> > On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
> >> As far as I tried,  minus MACH_SUN8I still enabled the following:
> >> CONFIG_SUNXI_CCU=y
> >> CONFIG_SUNXI_CCU_DIV=y
> >> CONFIG_SUNXI_CCU_FRAC=y
> >> CONFIG_SUNXI_CCU_GATE=y
> >> CONFIG_SUNXI_CCU_MUX=y
> >> CONFIG_SUNXI_CCU_MULT=y
> >> CONFIG_SUNXI_CCU_PHASE=y
> >> CONFIG_SUNXI_CCU_NK=y
> >> CONFIG_SUNXI_CCU_NKM=y
> >> CONFIG_SUNXI_CCU_NKMP=y
> >> CONFIG_SUNXI_CCU_NM=y
> >> CONFIG_SUNXI_CCU_MP=y
> >>
> >> I think you need to disable some more MACH_SUN*I and
> >> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
> >> then you will see difference in the result.
> >
> > Ah, right, it's all selected by the rest.
> >
> > I guess to get a meaningful example you could disable all the
> > SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
> > SUN8I_R_CCU.
> >
> > Most of the clock types shouldn't be used but div and gates.
> 
> Given that we appear to basically always pull in all (or most) of
> the CCU parts anyway, and it's hard to even test with a subset
> being disabled, I wonder how much we care about dropping
> the unused objects at all:
> 
> Maybe the answer is that we just always build all of them into
> the kernel and make only the SoC-specific objects configurable ;-)

I was trying to avoid that, but maybe that's not worth it, and it's
our only solution.

> Once we start linking with  --gc-sections, it will all be fine anyway.

That works for me. Who wants to send the patch?

Thanks!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> On Wed, 21 Jun 2017 12:49:10 +0200
>> Arnd Bergmann <arnd@arndb.de> wrote:
>>
>>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>>> > On Wed, 21 Jun 2017 12:21:16 +0200
>>> > Arnd Bergmann <arnd@arndb.de> wrote:
>>> >
>>>
>>> I'm building on top of yesterday's linux-next at the moment,
>>> with a number of my own patches applied
>>>
>>> > I can take a look at ARM and try to get it at least to parity with
>>> > incremental link. Any particular config options required?
>>>
>>> This is the patch I am testing with:
>>>
>>> https://pastebin.com/HQuhCEmK
>>> I have not looked at that in a while, no idea if it works, or
>>> if it has known problems.
>>>
>>> I last posted the patch in March for discussion:
>>>
>>> https://patchwork.kernel.org/patch/9626207/
>>
>> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
>> Just want to try getting thin archives up to a point where there are
>> no serious regressions first.
>
> For my build testing, I have now reverted most of my patch and only
> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

I just got one build failure on a randconfig build with THIN_ARCHIVES
but without LD_DCDE:

/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
[0000000000808000,000000000096cc7f] overlaps section .text VMA
[0000000000080080,00000000008a8427]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
[00000000008a9000,0000000000ea216f] overlaps section .data VMA
[0000000000808000,000000000096cc7f]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
[000000000096cd00,000000000157896f] overlaps section .rodata VMA
[00000000008a9000,0000000000ea216f]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
[0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
[000000000096cd00,000000000157896f]

haven't spent any time analyzing it further, maybe you can see right
away what caused this. If not, you could try the .config file at
https://pastebin.com/raw/YTZKH9Xe

        Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 22:52:25 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> On Wed, 21 Jun 2017 12:49:10 +0200
> >> Arnd Bergmann <arnd@arndb.de> wrote:
> >>  
> >>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >>> > On Wed, 21 Jun 2017 12:21:16 +0200
> >>> > Arnd Bergmann <arnd@arndb.de> wrote:
> >>> >  
> >>>
> >>> I'm building on top of yesterday's linux-next at the moment,
> >>> with a number of my own patches applied
> >>>  
> >>> > I can take a look at ARM and try to get it at least to parity with
> >>> > incremental link. Any particular config options required?  
> >>>
> >>> This is the patch I am testing with:
> >>>
> >>> https://pastebin.com/HQuhCEmK
> >>> I have not looked at that in a while, no idea if it works, or
> >>> if it has known problems.
> >>>
> >>> I last posted the patch in March for discussion:
> >>>
> >>> https://patchwork.kernel.org/patch/9626207/  
> >>
> >> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> >> Just want to try getting thin archives up to a point where there are
> >> no serious regressions first.  
> >
> > For my build testing, I have now reverted most of my patch and only
> > left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.  
> 
> I just got one build failure on a randconfig build with THIN_ARCHIVES
> but without LD_DCDE:
> 
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
> [0000000000808000,000000000096cc7f] overlaps section .text VMA
> [0000000000080080,00000000008a8427]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
> [00000000008a9000,0000000000ea216f] overlaps section .data VMA
> [0000000000808000,000000000096cc7f]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
> [000000000096cd00,000000000157896f] overlaps section .rodata VMA
> [00000000008a9000,0000000000ea216f]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
> [0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
> [000000000096cd00,000000000157896f]
> 
> haven't spent any time analyzing it further, maybe you can see right
> away what caused this. If not, you could try the .config file at
> https://pastebin.com/raw/YTZKH9Xe

It's due to vmlinux-xip.lds.S explicitly putting sections inside each
other

        . = PAGE_OFFSET + TEXT_OFFSET;

Same config fails without thin archives for me.

If XIP can't place these dynamically, at least you should use an
ASSERT() linker script function if sizes overlap. E.g.,

    ASSERT(. <= PAGE_OFFSET + TEXT_OFFSET, "kernel too big");
    . = PAGE_OFFSET + TEXT_OFFSET;

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Arnd Bergmann]

On Wed, Jun 21, 2017 at 11:30 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> On Wed, 21 Jun 2017 22:52:25 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
>> >> On Wed, 21 Jun 2017 12:49:10 +0200
>> > For my build testing, I have now reverted most of my patch and only
>> > left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
>>
>> I just got one build failure on a randconfig build with THIN_ARCHIVES
>> but without LD_DCDE:
>>
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
>> [0000000000808000,000000000096cc7f] overlaps section .text VMA
>> [0000000000080080,00000000008a8427]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
>> [00000000008a9000,0000000000ea216f] overlaps section .data VMA
>> [0000000000808000,000000000096cc7f]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
>> [000000000096cd00,000000000157896f] overlaps section .rodata VMA
>> [00000000008a9000,0000000000ea216f]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
>> [0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
>> [000000000096cd00,000000000157896f]
>>
>> haven't spent any time analyzing it further, maybe you can see right
>> away what caused this. If not, you could try the .config file at
>> https://pastebin.com/raw/YTZKH9Xe
>
> It's due to vmlinux-xip.lds.S explicitly putting sections inside each
> other
>
>         . = PAGE_OFFSET + TEXT_OFFSET;
>
> Same config fails without thin archives for me.
>
> If XIP can't place these dynamically, at least you should use an
> ASSERT() linker script function if sizes overlap. E.g.,
>
>     ASSERT(. <= PAGE_OFFSET + TEXT_OFFSET, "kernel too big");
>     . = PAGE_OFFSET + TEXT_OFFSET;
>

Right, makes sense. The linker error actually shows this quite clearly,
though I wonder why I never saw that with my LD_DCDE patch.
It's possible that the sections were always small enough in that
case, but there is probably something else going on as well.

I'd like to end up with a version that lets me build randconfig kernels
without warnings or errors (probably keeping LD_DCDE enabled
is sufficient). I have to think about that some more, but for now
this doesn't seem like a reason against doing THIN_ARCHIVES
in mainline for ARM.

       Arnd

Re: [PATCH 1/5] kbuild: thin archives final link close --whole-archives option

[Nicholas Piggin]

On Wed, 21 Jun 2017 12:49:10 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 21 Jun 2017 12:21:16 +0200
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> > On Wed, 21 Jun 2017 09:15:09 +0200
> >> > Arnd Bergmann <arnd@arndb.de> wrote:
> >> >  
> >> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <npiggin@gmail.com> wrote:  
> >> >> > On Wed, 21 Jun 2017 12:29:33 +0900
> >> >> > Masahiro Yamada <yamada.masahiro@socionext.com> wrote:  
> >> >>  
> >> >> >> BTW, I saw abuse of lib.a in
> >> >> >> https://patchwork.kernel.org/patch/9768439/
> >> >> >>
> >> >> >> I see it in linux-next.
> >> >> >>
> >> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> >> >> Author: Stephen Boyd <sboyd@codeaurora.org>
> >> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >> >> >>
> >> >> >>     clk: sunxi-ng: Move all clock types to a library
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Now drivers/clk/sunxi-ng/lib.a
> >> >> >> will go into thin archives.
> >> >> >> The result might be different from what they expect...  
> >> >> >
> >> >> > Yes I see. With thin archives, that is just going to cause the
> >> >> > same behaviour as built-in.o (everything will be linked). So the
> >> >> > build should not break, but they won't get savings.
> >> >> >
> >> >> > Does it even save space with incremental linking? If the lib.a gets
> >> >> > linked into drivers/built-in.o, I wonder what happens then?  
> >> >>
> >> >> Ah, too bad. I thought we had found a way to use a library correctly
> >> >> here, but I just verified that indeed all the just gets linked into built-in.o
> >> >>
> >> >> I played around with it some more now, but without success: if I
> >> >> build sunxi-ng as a loadable module (using a few modifications),
> >> >> then the unneeded objects from lib.a are dropped as I had hoped,
> >> >> but for built-in code we now always include everything.
> >> >>
> >> >> I suppose that we can ignore this once we get
> >> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> >> >> until then, we have a code size regression.  
> >> >
> >> > I didn't follow the thread there, is it a regression caused by
> >> > thin archives, or just by removing the Kconfig symbol from each
> >> > file?  
> >>
> >> I thought it was the latter, but actually it only happens with thin
> >> archives,  
> >
> > Is this including these changes now in the kbuild tree?  
> 
> I'm building on top of yesterday's linux-next at the moment,
> with a number of my own patches applied
> 
> > I can take a look at ARM and try to get it at least to parity with
> > incremental link. Any particular config options required?  
> 
> This is the patch I am testing with:
> 
> https://pastebin.com/HQuhCEmK
> 
> I have not looked at that in a while, no idea if it works, or
> if it has known problems.
> 
> I last posted the patch in March for discussion:
> 
> https://patchwork.kernel.org/patch/9626207/

Your patch needs the KEEP annotations on the other exception table
bits to avoid that error.

After that it builds and no obvious sections or symbols are missing
but it would take more careful review.

diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index 7b9a7058c32c..218ab698970a 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -91,8 +91,8 @@ SECTIONS
                ARM_EXIT_DISCARD(EXIT_DATA)
                EXIT_CALL
 #ifndef CONFIG_MMU
-               *(.text.fixup)
-               *(__ex_table)
+               KEEP(*(.text.fixup))
+               KEEP(*(__ex_table))
 #endif
 #ifndef CONFIG_SMP_ON_UP
                *(.alt.smp.init)
@@ -144,7 +144,7 @@ SECTIONS
        __ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) {
                __start___ex_table = .;
 #ifdef CONFIG_MMU
-               *(__ex_table)
+               KEEP(*(__ex_table))
 #endif
                __stop___ex_table = .;
        }

[PATCH 2/5] kbuild: thin archives use P option to ar

[Nicholas Piggin]

The P option makes ar do full path name matching and can prevent ar
from discarding files with duplicate names in some cases of creating
thin archives from thin archives. The sh architecture in particular
loses some object files from its kernel/cpu/sh*/ directories without
this option.

This could be a bug in binutils ar, but the P option should not cause
any negative effects so it is safe to use to work around tihs with.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 scripts/Makefile.build | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 733e044fff8b..4a9a2cec0a1b 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -437,8 +437,8 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
 ifdef builtin-target
 
 ifdef CONFIG_THIN_ARCHIVES
-  cmd_make_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
-  cmd_make_empty_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
+  cmd_make_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
+  cmd_make_empty_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
   quiet_cmd_link_o_target = AR      $@
 else
   cmd_make_builtin = $(LD) $(ld_flags) -r -o
@@ -478,7 +478,7 @@ ifdef lib-target
 quiet_cmd_link_l_target = AR      $@
 
 ifdef CONFIG_THIN_ARCHIVES
-  cmd_link_l_target = rm -f $@; $(AR) rcsT$(KBUILD_ARFLAGS) $@ $(lib-y)
+  cmd_link_l_target = rm -f $@; $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(lib-y)
 else
   cmd_link_l_target = rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@ $(lib-y)
 endif
@@ -531,7 +531,7 @@ cmd_link_multi-link = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secana
 
 ifdef CONFIG_THIN_ARCHIVES
   quiet_cmd_link_multi-y = AR      $@
-  cmd_link_multi-y = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
+  cmd_link_multi-y = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
 else
   quiet_cmd_link_multi-y = LD      $@
   cmd_link_multi-y = $(cmd_link_multi-link)
-- 
2.11.0

Re: [PATCH 2/5] kbuild: thin archives use P option to ar

[Masahiro Yamada]

Hi Nicholas,


2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> The P option makes ar do full path name matching and can prevent ar
> from discarding files with duplicate names in some cases of creating
> thin archives from thin archives. The sh architecture in particular
> loses some object files from its kernel/cpu/sh*/ directories without
> this option.

After playing around with thin archives, I agree this is the right
thing to do.

Currently, sh is the only architecture that has this kind of issue
(arch/sh/kernel/cpu/fpu.c  vs  arch/sh/kernel/cpu/sh*/fpu.c),
but this could happen in any architecture, I think.


BTW, I see one more instance in archive_builtin() in scripts/link-vmlinux.sh.

We have no source file at the top directory, so it will work
with/without "P" for the top-level built-in.o

Either way seems OK to me.


> This could be a bug in binutils ar, but the P option should not cause
> any negative effects so it is safe to use to work around tihs with.


Is "tihs" a typo?



> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  scripts/Makefile.build | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 733e044fff8b..4a9a2cec0a1b 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -437,8 +437,8 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
>  ifdef builtin-target
>
>  ifdef CONFIG_THIN_ARCHIVES
> -  cmd_make_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
> -  cmd_make_empty_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
> +  cmd_make_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
> +  cmd_make_empty_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
>    quiet_cmd_link_o_target = AR      $@
>  else
>    cmd_make_builtin = $(LD) $(ld_flags) -r -o
> @@ -478,7 +478,7 @@ ifdef lib-target
>  quiet_cmd_link_l_target = AR      $@
>
>  ifdef CONFIG_THIN_ARCHIVES
> -  cmd_link_l_target = rm -f $@; $(AR) rcsT$(KBUILD_ARFLAGS) $@ $(lib-y)
> +  cmd_link_l_target = rm -f $@; $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(lib-y)
>  else
>    cmd_link_l_target = rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@ $(lib-y)
>  endif
> @@ -531,7 +531,7 @@ cmd_link_multi-link = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secana
>
>  ifdef CONFIG_THIN_ARCHIVES
>    quiet_cmd_link_multi-y = AR      $@
> -  cmd_link_multi-y = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
> +  cmd_link_multi-y = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
>  else
>    quiet_cmd_link_multi-y = LD      $@
>    cmd_link_multi-y = $(cmd_link_multi-link)
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 2/5] kbuild: thin archives use P option to ar

[Nicholas Piggin]

On Mon, 19 Jun 2017 15:17:39 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > The P option makes ar do full path name matching and can prevent ar
> > from discarding files with duplicate names in some cases of creating
> > thin archives from thin archives. The sh architecture in particular
> > loses some object files from its kernel/cpu/sh*/ directories without
> > this option.  
> 
> After playing around with thin archives, I agree this is the right
> thing to do.
> 
> Currently, sh is the only architecture that has this kind of issue
> (arch/sh/kernel/cpu/fpu.c  vs  arch/sh/kernel/cpu/sh*/fpu.c),
> but this could happen in any architecture, I think.
> 
> 
> BTW, I see one more instance in archive_builtin() in scripts/link-vmlinux.sh.
> 
> We have no source file at the top directory, so it will work
> with/without "P" for the top-level built-in.o
> 
> Either way seems OK to me.

Oh, we should probably add the P there for consistency. I can't see
any downside to it. Can you fold in the fix?

> > This could be a bug in binutils ar, but the P option should not cause
> > any negative effects so it is safe to use to work around tihs with.  
> 
> 
> Is "tihs" a typo?

Yes. Should be "this".

Thanks,
Nick

[PATCH 3/5] sh: thin archives fix linking

[Nicholas Piggin]

The VDSO symbols can't be linked into built-in.o when building with
thin archives, so change this to linking them into the final link.

Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: linux-sh@vger.kernel.org
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---

Arch maintainers please give an ack if we can take this through
the kbuild tree.

Thanks,
Nick

 arch/sh/Makefile                 | 2 ++
 arch/sh/kernel/vsyscall/Makefile | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/sh/Makefile b/arch/sh/Makefile
index 280bbff12102..a907366f0352 100644
--- a/arch/sh/Makefile
+++ b/arch/sh/Makefile
@@ -97,6 +97,8 @@ defaultimage-$(CONFIG_SH_7619_SOLUTION_ENGINE)	:= vmlinux
 boot := arch/sh/boot
 KBUILD_IMAGE		:= $(boot)/$(defaultimage-y)
 
+LDFLAGS_vmlinux		+= -R arch/sh/kernel/vsyscall/vsyscall-syms.o
+
 #
 # Choosing incompatible machines durings configuration will result in
 # error messages during linking.
diff --git a/arch/sh/kernel/vsyscall/Makefile b/arch/sh/kernel/vsyscall/Makefile
index 8f0ea5fc835c..b82d13eb8d30 100644
--- a/arch/sh/kernel/vsyscall/Makefile
+++ b/arch/sh/kernel/vsyscall/Makefile
@@ -27,8 +27,6 @@ $(obj)/vsyscall-%.so: $(src)/vsyscall.lds $(obj)/vsyscall-%.o FORCE
 # table and layout of the linked DSO.  With ld -R we can then refer to
 # these symbols in the kernel code rather than hand-coded addresses.
 extra-y += vsyscall-syms.o
-$(obj)/built-in.o: $(obj)/vsyscall-syms.o
-$(obj)/built-in.o: ld_flags += -R $(obj)/vsyscall-syms.o
 
 SYSCFLAGS_vsyscall-syms.o = -r
 $(obj)/vsyscall-syms.o: $(src)/vsyscall.lds \
-- 
2.11.0

[PATCH 3/5] sh: thin archives fix linking

[Nicholas Piggin]

The VDSO symbols can't be linked into built-in.o when building with
thin archives, so change this to linking them into the final link.

Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: linux-sh@vger.kernel.org
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---

Arch maintainers please give an ack if we can take this through
the kbuild tree.

Thanks,
Nick

 arch/sh/Makefile                 | 2 ++
 arch/sh/kernel/vsyscall/Makefile | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/sh/Makefile b/arch/sh/Makefile
index 280bbff12102..a907366f0352 100644
--- a/arch/sh/Makefile
+++ b/arch/sh/Makefile
@@ -97,6 +97,8 @@ defaultimage-$(CONFIG_SH_7619_SOLUTION_ENGINE)	:= vmlinux
 boot := arch/sh/boot
 KBUILD_IMAGE		:= $(boot)/$(defaultimage-y)
 
+LDFLAGS_vmlinux		+= -R arch/sh/kernel/vsyscall/vsyscall-syms.o
+
 #
 # Choosing incompatible machines durings configuration will result in
 # error messages during linking.
diff --git a/arch/sh/kernel/vsyscall/Makefile b/arch/sh/kernel/vsyscall/Makefile
index 8f0ea5fc835c..b82d13eb8d30 100644
--- a/arch/sh/kernel/vsyscall/Makefile
+++ b/arch/sh/kernel/vsyscall/Makefile
@@ -27,8 +27,6 @@ $(obj)/vsyscall-%.so: $(src)/vsyscall.lds $(obj)/vsyscall-%.o FORCE
 # table and layout of the linked DSO.  With ld -R we can then refer to
 # these symbols in the kernel code rather than hand-coded addresses.
 extra-y += vsyscall-syms.o
-$(obj)/built-in.o: $(obj)/vsyscall-syms.o
-$(obj)/built-in.o: ld_flags += -R $(obj)/vsyscall-syms.o
 
 SYSCFLAGS_vsyscall-syms.o = -r
 $(obj)/vsyscall-syms.o: $(src)/vsyscall.lds \
-- 
2.11.0

Re: [PATCH 3/5] sh: thin archives fix linking

[Masahiro Yamada]

2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> The VDSO symbols can't be linked into built-in.o when building with
> thin archives, so change this to linking them into the final link.
>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Rich Felker <dalias@libc.org>
> Cc: linux-sh@vger.kernel.org
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---

SH maintainers, any comments for this patch?


I confirmed this patch solved the build error,
but I do not have access to any SH hardware.

Some Acked-by will be very helpful.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 3/5] sh: thin archives fix linking

[Masahiro Yamada]

2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> The VDSO symbols can't be linked into built-in.o when building with
> thin archives, so change this to linking them into the final link.
>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Rich Felker <dalias@libc.org>
> Cc: linux-sh@vger.kernel.org
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---

SH maintainers, any comments for this patch?


I confirmed this patch solved the build error,
but I do not have access to any SH hardware.

Some Acked-by will be very helpful.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 3/5] sh: thin archives fix linking

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

On 06/19/2017 01:19 AM, Masahiro Yamada wrote:
> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>> The VDSO symbols can't be linked into built-in.o when building with
>> thin archives, so change this to linking them into the final link.
>>
>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>> Cc: Rich Felker <dalias@libc.org>
>> Cc: linux-sh@vger.kernel.org
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
> 
> SH maintainers, any comments for this patch?
> 
> 
> I confirmed this patch solved the build error,
> but I do not have access to any SH hardware.

You can build and boot the kernel under qemu for several architectures
(including sh4) by following the instructions on:

  https://github.com/landley/mkroot

I believe I last built -rc6?

You may need the attached patch to fix the serial port issue from:

  https://www.spinics.net/lists/linux-serial/msg26085.html

Which is still there because the kernel guys don't care that they broke
qemu, and the qemu guys are still using old kernel versions that work
fine. My patch just reverts the kernel change (which enabled hardware
buffer logic qemu doesn't implement).

Rob




[-- Attachment #2: qemu-sh4.patch --]
[-- Type: text/x-diff, Size: 479 bytes --]

diff --git a/drivers/tty/serial/sh-sci.c b/drivers/tty/serial/sh-sci.c
index 71707e8..a798def 100644
--- a/drivers/tty/serial/sh-sci.c
+++ b/drivers/tty/serial/sh-sci.c
@@ -2193,7 +2193,7 @@ static void sci_reset(struct uart_port *port)
 			setup_timer(&s->rx_fifo_timer, rx_fifo_timer_fn,
 				    (unsigned long)s);
 		} else {
-			if (port->type == PORT_SCIFA ||
+			if (1 || port->type == PORT_SCIFA ||
 			    port->type == PORT_SCIFB)
 				scif_set_rtrg(port, 1);
 			else

Re: [PATCH 3/5] sh: thin archives fix linking

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

On 06/19/2017 01:19 AM, Masahiro Yamada wrote:
> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
>> The VDSO symbols can't be linked into built-in.o when building with
>> thin archives, so change this to linking them into the final link.
>>
>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>> Cc: Rich Felker <dalias@libc.org>
>> Cc: linux-sh@vger.kernel.org
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
> 
> SH maintainers, any comments for this patch?
> 
> 
> I confirmed this patch solved the build error,
> but I do not have access to any SH hardware.

You can build and boot the kernel under qemu for several architectures
(including sh4) by following the instructions on:

  https://github.com/landley/mkroot

I believe I last built -rc6?

You may need the attached patch to fix the serial port issue from:

  https://www.spinics.net/lists/linux-serial/msg26085.html

Which is still there because the kernel guys don't care that they broke
qemu, and the qemu guys are still using old kernel versions that work
fine. My patch just reverts the kernel change (which enabled hardware
buffer logic qemu doesn't implement).

Rob




[-- Attachment #2: qemu-sh4.patch --]
[-- Type: text/x-diff, Size: 479 bytes --]

diff --git a/drivers/tty/serial/sh-sci.c b/drivers/tty/serial/sh-sci.c
index 71707e8..a798def 100644
--- a/drivers/tty/serial/sh-sci.c
+++ b/drivers/tty/serial/sh-sci.c
@@ -2193,7 +2193,7 @@ static void sci_reset(struct uart_port *port)
 			setup_timer(&s->rx_fifo_timer, rx_fifo_timer_fn,
 				    (unsigned long)s);
 		} else {
-			if (port->type == PORT_SCIFA ||
+			if (1 || port->type == PORT_SCIFA ||
 			    port->type == PORT_SCIFB)
 				scif_set_rtrg(port, 1);
 			else

[PATCH 4/5] x86/um: thin archives build fix

[Nicholas Piggin]

The linker does not like vdso-syms.lds in input archive files.
Make it an extra-y instead.

Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: user-mode-linux-devel@lists.sourceforge.net
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---

Arch maintainers please give an ack if we can take this through
the kbuild tree.

Thanks,
Nick

 arch/x86/um/vdso/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index d72dec406ccb..329406224330 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -53,7 +53,7 @@ CFLAGS_REMOVE_vdso-note.o = -pg -fprofile-arcs -ftest-coverage
 CFLAGS_REMOVE_um_vdso.o = -pg -fprofile-arcs -ftest-coverage
 
 targets += vdso-syms.lds
-obj-$(VDSO64-y)			+= vdso-syms.lds
+extra-$(VDSO64-y)			+= vdso-syms.lds
 
 #
 # Match symbols in the DSO that look like VDSO*; produce a file of constants.
-- 
2.11.0

Re: [PATCH 4/5] x86/um: thin archives build fix

[Masahiro Yamada]

2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> The linker does not like vdso-syms.lds in input archive files.
> Make it an extra-y instead.
>
> Cc: Jeff Dike <jdike@addtoit.com>
> Cc: Richard Weinberger <richard@nod.at>
> Cc: user-mode-linux-devel@lists.sourceforge.net
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>
> Arch maintainers please give an ack if we can take this through
> the kbuild tree.
>
> Thanks,
> Nick
>
>  arch/x86/um/vdso/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
> index d72dec406ccb..329406224330 100644
> --- a/arch/x86/um/vdso/Makefile
> +++ b/arch/x86/um/vdso/Makefile
> @@ -53,7 +53,7 @@ CFLAGS_REMOVE_vdso-note.o = -pg -fprofile-arcs -ftest-coverage
>  CFLAGS_REMOVE_um_vdso.o = -pg -fprofile-arcs -ftest-coverage
>
>  targets += vdso-syms.lds
> -obj-$(VDSO64-y)                        += vdso-syms.lds
> +extra-$(VDSO64-y)                      += vdso-syms.lds


I agree this line is weird, but where is vdso-syms.lds used?


I removed the following, but it still built successfully.


targets += vdso-syms.lds
obj-$(VDSO64-y)                        += vdso-syms.lds

$(obj)/%-syms.lds: $(obj)/%.so.dbg FORCE
       $(call if_changed,vdsosym)





-- 
Best Regards
Masahiro Yamada

[PATCH 5/5] kbuild: thin archives make default for all archs

[Nicholas Piggin]

Make thin archives build the default, but keep the config option
to allow exemptions if any breakage can't be quickly solved.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 Documentation/process/changes.rst | 9 ++++-----
 arch/Kconfig                      | 2 +-
 arch/powerpc/Kconfig              | 8 --------
 3 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index e25d63f8c0da..56fd0687bd4a 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -31,7 +31,7 @@ you probably needn't concern yourself with isdn4k-utils.
 ====================== ===============  ========================================
 GNU C                  3.2              gcc --version
 GNU make               3.81             make --version
-binutils               2.12             ld -v
+binutils               2.20             ld -v
 util-linux             2.10o            fdformat --version
 module-init-tools      0.9.10           depmod -V
 e2fsprogs              1.41.4           e2fsck -V
@@ -75,10 +75,9 @@ You will need GNU make 3.81 or later to build the kernel.
 Binutils
 --------
 
-Linux on IA-32 has recently switched from using ``as86`` to using ``gas`` for
-assembling the 16-bit boot code, removing the need for ``as86`` to compile
-your kernel.  This change does, however, mean that you need a recent
-release of binutils.
+The build system has recently switched to using thin archives (`ar T`) rather
+than incremental linking (`ld -r`) for built-in.o intermediate steps. This
+requires binutils 2.20 or newer.
 
 Perl
 ----
diff --git a/arch/Kconfig b/arch/Kconfig
index 6c00e5b00f8b..3abe581878d6 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -511,7 +511,7 @@ config CC_STACKPROTECTOR_STRONG
 endchoice
 
 config THIN_ARCHIVES
-	bool
+	def_bool y
 	help
 	  Select this if the architecture wants to use thin archives
 	  instead of ld -r to create the built-in.o files.
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index f7c8f9972f61..80d882a78426 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -472,14 +472,6 @@ config MPROFILE_KERNEL
 	depends on PPC64 && CPU_LITTLE_ENDIAN
 	def_bool !DISABLE_MPROFILE_KERNEL
 
-config USE_THIN_ARCHIVES
-	bool "Build the kernel using thin archives"
-	default n
-	select THIN_ARCHIVES
-	help
-	  Build the kernel using thin archives.
-	  If you're unsure say N.
-
 config IOMMU_HELPER
 	def_bool PPC64
 
-- 
2.11.0

Re: [PATCH 5/5] kbuild: thin archives make default for all archs

[Masahiro Yamada]

Hi Nicholas,

2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> Make thin archives build the default, but keep the config option
> to allow exemptions if any breakage can't be quickly solved.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  Documentation/process/changes.rst | 9 ++++-----
>  arch/Kconfig                      | 2 +-
>  arch/powerpc/Kconfig              | 8 --------
>  3 files changed, 5 insertions(+), 14 deletions(-)
>
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index e25d63f8c0da..56fd0687bd4a 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -31,7 +31,7 @@ you probably needn't concern yourself with isdn4k-utils.
>  ====================== ===============  ========================================
>  GNU C                  3.2              gcc --version
>  GNU make               3.81             make --version
> -binutils               2.12             ld -v
> +binutils               2.20             ld -v
>  util-linux             2.10o            fdformat --version
>  module-init-tools      0.9.10           depmod -V
>  e2fsprogs              1.41.4           e2fsck -V
> @@ -75,10 +75,9 @@ You will need GNU make 3.81 or later to build the kernel.
>  Binutils
>  --------
>
> -Linux on IA-32 has recently switched from using ``as86`` to using ``gas`` for
> -assembling the 16-bit boot code, removing the need for ``as86`` to compile
> -your kernel.  This change does, however, mean that you need a recent
> -release of binutils.
> +The build system has recently switched to using thin archives (`ar T`) rather
> +than incremental linking (`ld -r`) for built-in.o intermediate steps. This
> +requires binutils 2.20 or newer.

Sorry for my nit-picking.

If you see this document some years later,
"recently" may not be so recent.

Currently, this part is written like that already,
but perhaps is it better to reword it a bit?



-- 
Best Regards
Masahiro Yamada

Re: [PATCH 5/5] kbuild: thin archives make default for all archs

[Nicholas Piggin]

On Mon, 19 Jun 2017 15:22:14 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> Hi Nicholas,
> 
> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> > Make thin archives build the default, but keep the config option
> > to allow exemptions if any breakage can't be quickly solved.
> >
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >  Documentation/process/changes.rst | 9 ++++-----
> >  arch/Kconfig                      | 2 +-
> >  arch/powerpc/Kconfig              | 8 --------
> >  3 files changed, 5 insertions(+), 14 deletions(-)
> >
> > diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> > index e25d63f8c0da..56fd0687bd4a 100644
> > --- a/Documentation/process/changes.rst
> > +++ b/Documentation/process/changes.rst
> > @@ -31,7 +31,7 @@ you probably needn't concern yourself with isdn4k-utils.
> >  ====================== ===============  ========================================
> >  GNU C                  3.2              gcc --version
> >  GNU make               3.81             make --version
> > -binutils               2.12             ld -v
> > +binutils               2.20             ld -v
> >  util-linux             2.10o            fdformat --version
> >  module-init-tools      0.9.10           depmod -V
> >  e2fsprogs              1.41.4           e2fsck -V
> > @@ -75,10 +75,9 @@ You will need GNU make 3.81 or later to build the kernel.
> >  Binutils
> >  --------
> >
> > -Linux on IA-32 has recently switched from using ``as86`` to using ``gas`` for
> > -assembling the 16-bit boot code, removing the need for ``as86`` to compile
> > -your kernel.  This change does, however, mean that you need a recent
> > -release of binutils.
> > +The build system has recently switched to using thin archives (`ar T`) rather
> > +than incremental linking (`ld -r`) for built-in.o intermediate steps. This
> > +requires binutils 2.20 or newer.  
> 
> Sorry for my nit-picking.
> 
> If you see this document some years later,
> "recently" may not be so recent.
> 
> Currently, this part is written like that already,
> but perhaps is it better to reword it a bit?

Good point :)

We could say "The build system has, as of 4.13, switched to using..."

Thanks,
Nick

Re: [PATCH 0/5] move everyone over to thin archives

[Nicholas Piggin]

On Fri,  9 Jun 2017 15:24:12 +1000
Nicholas Piggin <npiggin@gmail.com> wrote:

> This has gone through 0day several times now, and I think I've
> weeded out all the build breakage. Following what Linus asked
> for, I did not make it a user selectable option. I kept the
> option there in case we need to do a depend on !ARCH for any
> reason.
> 
> Please consider for the kbuild tree.

Any thoughts on this?

Thanks,
Nick

Re: [PATCH 0/5] move everyone over to thin archives

[Masahiro Yamada]

Hi Nicholas,

2017-06-17 22:10 GMT+09:00 Nicholas Piggin <npiggin@gmail.com>:
> On Fri,  9 Jun 2017 15:24:12 +1000
> Nicholas Piggin <npiggin@gmail.com> wrote:
>
>> This has gone through 0day several times now, and I think I've
>> weeded out all the build breakage. Following what Linus asked
>> for, I did not make it a user selectable option. I kept the
>> option there in case we need to do a depend on !ARCH for any
>> reason.
>>
>> Please consider for the kbuild tree.
>
> Any thoughts on this?
>
> Thanks,
> Nick

Sorry for my late reply.

Basically, I think thin-archive is the right direction we should go.
(and I agree thin-archive and DCE can get in separately)

Could you check my comments on each patch?



-- 
Best Regards
Masahiro Yamada