* [PATCH] scripts: Fix incremental build header re-generation
@ 2021-03-01 14:22 Jeevan Shriram
2021-03-02 3:36 ` Masahiro Yamada
0 siblings, 1 reply; 4+ messages in thread
From: Jeevan Shriram @ 2021-03-01 14:22 UTC (permalink / raw)
To: linux-kbuild; +Cc: Jeevan Shriram, linux-kernel, tsoni
compile.h and autoconf.h are ignored when checking headers sha as they
are always re-generated for every kernel compilation. However,
these two headers are packaged into kheaders tar. During incremental
compilation of kernel, kheaders tar file is always generated and re-packaged
irrespective of the changes in headers.
Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f
Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org>
---
| 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
--git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index c1510f0..5499f72 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
if [ -f kernel/kheaders.md5 ] &&
[ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] &&
- [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] &&
- [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then
+ [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then
exit
fi
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] scripts: Fix incremental build header re-generation
2021-03-01 14:22 [PATCH] scripts: Fix incremental build header re-generation Jeevan Shriram
@ 2021-03-02 3:36 ` Masahiro Yamada
2021-03-18 2:54 ` Jeevan Shriram
0 siblings, 1 reply; 4+ messages in thread
From: Masahiro Yamada @ 2021-03-02 3:36 UTC (permalink / raw)
To: Jeevan Shriram
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List, Trilok Soni
On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote:
>
> compile.h and autoconf.h are ignored when checking headers sha as they
> are always re-generated for every kernel compilation. However,
> these two headers are packaged into kheaders tar. During incremental
> compilation of kernel, kheaders tar file is always generated and re-packaged
> irrespective of the changes in headers.
I do not see this problem.
Could you describe the steps to reproduce it, please?
> Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f
> Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org>
> ---
> kernel/gen_kheaders.sh | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index c1510f0..5499f72 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
> if [ -f kernel/kheaders.md5 ] &&
> [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] &&
> - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] &&
> - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then
> + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then
> exit
> fi
>
> --
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] scripts: Fix incremental build header re-generation
2021-03-02 3:36 ` Masahiro Yamada
@ 2021-03-18 2:54 ` Jeevan Shriram
2021-03-24 6:27 ` Masahiro Yamada
0 siblings, 1 reply; 4+ messages in thread
From: Jeevan Shriram @ 2021-03-18 2:54 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List, Trilok Soni
On 3/1/2021 7:36 PM, Masahiro Yamada wrote:
> On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote:
>> compile.h and autoconf.h are ignored when checking headers sha as they
>> are always re-generated for every kernel compilation. However,
>> these two headers are packaged into kheaders tar. During incremental
>> compilation of kernel, kheaders tar file is always generated and re-packaged
>> irrespective of the changes in headers.
>
> I do not see this problem.
> Could you describe the steps to reproduce it, please?
>
Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes.
I have added following log in gen_kheaders.sh script for confirming the hash differences.
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index b7425a0..ee542a0 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" |
# Any changes to this script will also cause a rebuild of the archive.
this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
+
+echo "Old Tar file $tarfile_md5"
+echo "New Tar file hash $this_file_md5"
+
if [ -f kernel/kheaders.md5 ] &&
[ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] &&
[ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
log output :
89306 19:29:02.109961 CHK kernel/kheaders_data.tar.xz
89307 19:29:02.109971 Old Tar file 2aa6990e4183c31a862951f4bcac037e
89308 19:29:02.109982 New Tar file hash ecf84e700c7cacfe8b35a0905859582d
>> Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f
>> Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org>
>> ---
>> kernel/gen_kheaders.sh | 3 +--
>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
>> index c1510f0..5499f72 100755
>> --- a/kernel/gen_kheaders.sh
>> +++ b/kernel/gen_kheaders.sh
>> @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
>> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
>> if [ -f kernel/kheaders.md5 ] &&
>> [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] &&
>> - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] &&
>> - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then
>> + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then
>> exit
>> fi
>>
>> --
>> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
>> a Linux Foundation Collaborative Project
>>
>
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH] scripts: Fix incremental build header re-generation
2021-03-18 2:54 ` Jeevan Shriram
@ 2021-03-24 6:27 ` Masahiro Yamada
0 siblings, 0 replies; 4+ messages in thread
From: Masahiro Yamada @ 2021-03-24 6:27 UTC (permalink / raw)
To: Jeevan Shriram
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List, Trilok Soni
On Thu, Mar 18, 2021 at 11:55 AM Jeevan Shriram <jshriram@codeaurora.org> wrote:
>
>
> On 3/1/2021 7:36 PM, Masahiro Yamada wrote:
> > On Mon, Mar 1, 2021 at 11:23 PM Jeevan Shriram <jshriram@codeaurora.org> wrote:
> >> compile.h and autoconf.h are ignored when checking headers sha as they
> >> are always re-generated for every kernel compilation. However,
> >> these two headers are packaged into kheaders tar. During incremental
> >> compilation of kernel, kheaders tar file is always generated and re-packaged
> >> irrespective of the changes in headers.
> >
> > I do not see this problem.
> > Could you describe the steps to reproduce it, please?
> >
> Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes.
> I have added following log in gen_kheaders.sh script for confirming the hash differences.
Of course they are different because you are comparing
the hashes of two different files.
>
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index b7425a0..ee542a0 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" |
> # Any changes to this script will also cause a rebuild of the archive.
> this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
> +
> +echo "Old Tar file $tarfile_md5"
This is the hash of kernel/kheaders_data.tar.xz
> +echo "New Tar file hash $this_file_md5"
This is the hash of kernel/gen_kheaders.sh
> +
> if [ -f kernel/kheaders.md5 ] &&
> [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] &&
> [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
>
> log output :
> 89306 19:29:02.109961 CHK kernel/kheaders_data.tar.xz
> 89307 19:29:02.109971 Old Tar file 2aa6990e4183c31a862951f4bcac037e
> 89308 19:29:02.109982 New Tar file hash ecf84e700c7cacfe8b35a0905859582d
I do not understand what your claim is.
CHK kernel/kheaders_data.tar.xz
is displayed when it is checking the hash in order to
determine if the tarball should be updated.
GEN kernel/kheaders_data.tar.xz
is displayed when the tarball is really updated.
[Incremental build with no change]
masahiro@oscar:~/ref/linux$ make -j24
DESCEND objtool
CALL scripts/atomic/check-atomics.sh
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
CHK kernel/kheaders_data.tar.xz
Kernel: arch/x86/boot/bzImage is ready (#2)
[Touch one header and rebuild]
masahiro@oscar:~/ref/linux$ touch include/uapi/drm/exynos_drm.h
masahiro@oscar:~/ref/linux$ make -j24
DESCEND objtool
CALL scripts/atomic/check-atomics.sh
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
CHK kernel/kheaders_data.tar.xz
GEN kernel/kheaders_data.tar.xz
CC kernel/kheaders.o
AR kernel/built-in.a
GEN .version
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
AR init/built-in.a
LD vmlinux.o
MODPOST vmlinux.symvers
MODINFO modules.builtin.modinfo
GEN modules.builtin
LD .tmp_vmlinux.kallsyms1
KSYMS .tmp_vmlinux.kallsyms1.S
AS .tmp_vmlinux.kallsyms1.S
LD .tmp_vmlinux.kallsyms2
KSYMS .tmp_vmlinux.kallsyms2.S
AS .tmp_vmlinux.kallsyms2.S
LD vmlinux
SORTTAB vmlinux
SYSMAP System.map
MODPOST Module.symvers
CC arch/x86/boot/version.o
VOFFSET arch/x86/boot/compressed/../voffset.h
OBJCOPY arch/x86/boot/compressed/vmlinux.bin
RELOCS arch/x86/boot/compressed/vmlinux.relocs
CC arch/x86/boot/compressed/kaslr.o
CC arch/x86/boot/compressed/misc.o
GZIP arch/x86/boot/compressed/vmlinux.bin.gz
MKPIGGY arch/x86/boot/compressed/piggy.S
AS arch/x86/boot/compressed/piggy.o
LD arch/x86/boot/compressed/vmlinux
ZOFFSET arch/x86/boot/zoffset.h
OBJCOPY arch/x86/boot/vmlinux.bin
AS arch/x86/boot/header.o
LD arch/x86/boot/setup.elf
OBJCOPY arch/x86/boot/setup.bin
BUILD arch/x86/boot/bzImage
Kernel: arch/x86/boot/bzImage is ready (#3)
> >> Change-Id: I7a64faebb81df44c32230b0fea1d6df09d7ce66f
> >> Signed-off-by: Jeevan Shriram <jshriram@codeaurora.org>
> >> ---
> >> kernel/gen_kheaders.sh | 3 +--
> >> 1 file changed, 1 insertion(+), 2 deletions(-)
> >>
> >> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> >> index c1510f0..5499f72 100755
> >> --- a/kernel/gen_kheaders.sh
> >> +++ b/kernel/gen_kheaders.sh
> >> @@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)"
> >> if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
> >> if [ -f kernel/kheaders.md5 ] &&
> >> [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] &&
> >> - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] &&
> >> - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then
> >> + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then
> >> exit
> >> fi
> >>
> >> --
> >> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> >> a Linux Foundation Collaborative Project
> >>
> >
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-03-24 6:29 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-01 14:22 [PATCH] scripts: Fix incremental build header re-generation Jeevan Shriram
2021-03-02 3:36 ` Masahiro Yamada
2021-03-18 2:54 ` Jeevan Shriram
2021-03-24 6:27 ` Masahiro Yamada
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).