linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [GIT PULL] Driver core patches for 5.2-rc1
@ 2019-05-07 17:59 Greg KH
  2019-05-07 20:50 ` pr-tracker-bot
  2019-05-09 20:47 ` Linus Torvalds
  0 siblings, 2 replies; 7+ messages in thread
From: Greg KH @ 2019-05-07 17:59 UTC (permalink / raw)
  To: Linus Torvalds, Andrew Morton; +Cc: linux-kernel

The following changes since commit 79a3aaa7b82e3106be97842dedfd8429248896e6:

  Linux 5.1-rc3 (2019-03-31 14:39:29 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core.git tags/driver-core-5.2-rc1

for you to fetch changes up to 70e16a620e075cb916644e06012766639b58b2fb:

  kobject: clean up the kobject add documentation a bit more (2019-05-03 08:26:51 +0200)

----------------------------------------------------------------
Driver core/kobject patches for 5.2-rc1

Here is the "big" set of driver core patches for 5.2-rc1

There are a number of ACPI patches in here as well, as Rafael said they
should go through this tree due to the driver core changes they
required.  They have all been acked by the ACPI developers.

There are also a number of small subsystem-specific changes in here, due
to some changes to the kobject core code.  Those too have all been acked
by the various subsystem maintainers.

As for content, it's pretty boring outside of the ACPI changes:
  - spdx cleanups
  - kobject documentation updates
  - default attribute groups for kobjects
  - other minor kobject/driver core fixes

All have been in linux-next for a while with no reported issues.

Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

----------------------------------------------------------------
Alison Schofield (1):
      acpi/hmat: Update acpi_hmat_type enum with ACPI_HMAT_TYPE_PROXIMITY

Andrea Parri (1):
      kernfs: fix barrier usage in __kernfs_new_node()

Andy Shevchenko (1):
      driver core: platform: Propagate error from insert_resource()

Bartosz Golaszewski (1):
      drivers: fix a typo in the kernel doc for devm_platform_ioremap_resource()

Borislav Petkov (1):
      driver core: Clarify which counterparts to use to device_add()

Christina Quast (1):
      fs: kernfs: Corrected spelling mistake

Colin Ian King (1):
      kobject: fix dereference before null check on kobj

Geert Uytterhoeven (1):
      driver: base: Disable CONFIG_UEVENT_HELPER by default

Greg Kroah-Hartman (6):
      device.h: reorganize struct device
      drivers: base: test: add proper SPDX identifier to Makefile
      drivers: base: firmware_loader: add proper SPDX identifiers on files that did not have them.
      drivers: base: power: add proper SPDX identifiers on files that did not have them.
      Revert "driver core: platform: Fix the usage of platform device name(pdev->name)"
      kobject: clean up the kobject add documentation a bit more

Joel Fernandes (Google) (2):
      Provide in-kernel headers to make extending kernel easier
      init/config: Do not select BUILD_BIN2C for IKCONFIG

John Garry (1):
      driver core: Postpone DMA tear-down until after devres release for probe failure

Jonathan Neuschäfer (1):
      firmware_loader: Fix a typo ("syfs" -> "sysfs")

Keith Busch (10):
      acpi: Create subtable parsing infrastructure
      acpi: Add HMAT to generic parsing tables
      acpi/hmat: Parse and report heterogeneous memory
      node: Link memory nodes to their compute nodes
      node: Add heterogenous memory access attributes
      node: Add memory-side caching attributes
      acpi/hmat: Register processor domain to its memory
      acpi/hmat: Register performance attributes
      acpi/hmat: Register memory side cache attributes
      doc/mm: New documentation for memory performance

Kimberly Brown (8):
      kobject: Add support for default attribute groups to kobj_type
      samples/kobject: Replace foo_ktype's default_attrs field with groups
      block: Replace all ktype default_attrs with groups
      net-sysfs: Replace ktype default_attrs field with groups
      irqdesc: Replace irq_kobj_type's default_attrs field with groups
      padata: Replace padata_attr_type default_attrs field with groups
      cpufreq: schedutil: Replace default_attrs field with groups
      livepatch: Replace klp_ktype_patch's default_attrs with groups

Lingutla Chandrasekhar (1):
      arch_topology: Make cpu_capacity sysfs node as read-only

Qian Cai (2):
      acpi/hmat: fix memory leaks in hmat_init()
      acpi/hmat: fix an uninitialized memory_target

Ronald Tschalär (1):
      debugfs: update documented return values of debugfs helpers

Tetsuo Handa (1):
      kobject: Don't trigger kobject_uevent(KOBJ_REMOVE) twice.

Tobin C. Harding (4):
      kobject: Improve docs for kobject_add/del
      kobject: Improve doc clarity kobject_init_and_add()
      kobject: Remove docstring reference to kset
      kobject: Fix kernel-doc comment first line

Venkata Narendra Kumar Gutta (1):
      driver core: platform: Fix the usage of platform device name(pdev->name)

zhong jiang (1):
      mm/memory_hotplug: Do not unlock when fails to take the device_hotplug_lock

 Documentation/ABI/stable/sysfs-devices-node     |  87 +++-
 Documentation/admin-guide/mm/numaperf.rst       | 169 ++++++
 Documentation/filesystems/debugfs.txt           |  16 +-
 arch/arm64/kernel/acpi_numa.c                   |   2 +-
 arch/arm64/kernel/smp.c                         |   4 +-
 arch/ia64/kernel/acpi.c                         |  14 +-
 arch/x86/kernel/acpi/boot.c                     |  36 +-
 block/blk-integrity.c                           |   3 +-
 block/blk-mq-sysfs.c                            |   8 +-
 block/blk-sysfs.c                               |   3 +-
 drivers/acpi/Kconfig                            |   1 +
 drivers/acpi/Makefile                           |   1 +
 drivers/acpi/hmat/Kconfig                       |  11 +
 drivers/acpi/hmat/Makefile                      |   1 +
 drivers/acpi/hmat/hmat.c                        | 666 ++++++++++++++++++++++++
 drivers/acpi/numa.c                             |  16 +-
 drivers/acpi/scan.c                             |   4 +-
 drivers/acpi/tables.c                           |  76 ++-
 drivers/base/Kconfig                            |   9 +-
 drivers/base/arch_topology.c                    |  36 +-
 drivers/base/core.c                             |   5 +
 drivers/base/dd.c                               |   5 +-
 drivers/base/firmware_loader/Kconfig            |   1 +
 drivers/base/firmware_loader/builtin/.gitignore |   1 +
 drivers/base/firmware_loader/fallback.c         |   6 +-
 drivers/base/memory.c                           |   2 +-
 drivers/base/node.c                             | 352 ++++++++++++-
 drivers/base/platform.c                         |  12 +-
 drivers/base/power/clock_ops.c                  |   3 +-
 drivers/base/power/common.c                     |   4 +-
 drivers/base/power/domain.c                     |   4 +-
 drivers/base/power/domain_governor.c            |   4 +-
 drivers/base/power/generic_ops.c                |   4 +-
 drivers/base/power/main.c                       |   4 +-
 drivers/base/power/qos.c                        |   6 +-
 drivers/base/power/runtime.c                    |   4 +-
 drivers/base/power/sysfs.c                      |   6 +-
 drivers/base/power/trace.c                      |   2 +-
 drivers/base/power/wakeirq.c                    |  15 +-
 drivers/base/power/wakeup.c                     |   4 +-
 drivers/base/test/Makefile                      |   1 +
 drivers/irqchip/irq-gic-v2m.c                   |   2 +-
 drivers/irqchip/irq-gic-v3-its-pci-msi.c        |   2 +-
 drivers/irqchip/irq-gic-v3-its-platform-msi.c   |   2 +-
 drivers/irqchip/irq-gic-v3-its.c                |   6 +-
 drivers/irqchip/irq-gic-v3.c                    |  10 +-
 drivers/irqchip/irq-gic.c                       |   4 +-
 drivers/mailbox/pcc.c                           |   2 +-
 fs/debugfs/file.c                               |  77 ++-
 fs/kernfs/dir.c                                 |   5 +-
 include/acpi/actbl1.h                           |   2 +-
 include/linux/acpi.h                            |   6 +-
 include/linux/device.h                          |  16 +-
 include/linux/kernfs.h                          |   2 +-
 include/linux/kobject.h                         |   3 +-
 include/linux/node.h                            |  71 +++
 init/Kconfig                                    |  11 +-
 kernel/.gitignore                               |   1 +
 kernel/Makefile                                 |  10 +
 kernel/gen_ikh_data.sh                          |  89 ++++
 kernel/irq/irqdesc.c                            |   3 +-
 kernel/kheaders.c                               |  74 +++
 kernel/livepatch/core.c                         |   3 +-
 kernel/padata.c                                 |   3 +-
 kernel/sched/cpufreq_schedutil.c                |   5 +-
 lib/kobject.c                                   |  93 ++--
 lib/kobject_uevent.c                            |  11 +-
 net/core/net-sysfs.c                            |   6 +-
 samples/kobject/kset-example.c                  |   3 +-
 69 files changed, 1855 insertions(+), 275 deletions(-)
 create mode 100644 Documentation/admin-guide/mm/numaperf.rst
 create mode 100644 drivers/acpi/hmat/Kconfig
 create mode 100644 drivers/acpi/hmat/Makefile
 create mode 100644 drivers/acpi/hmat/hmat.c
 create mode 100755 kernel/gen_ikh_data.sh
 create mode 100644 kernel/kheaders.c

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-07 17:59 [GIT PULL] Driver core patches for 5.2-rc1 Greg KH
@ 2019-05-07 20:50 ` pr-tracker-bot
  2019-05-09 20:47 ` Linus Torvalds
  1 sibling, 0 replies; 7+ messages in thread
From: pr-tracker-bot @ 2019-05-07 20:50 UTC (permalink / raw)
  To: Greg KH; +Cc: Linus Torvalds, Andrew Morton, linux-kernel

The pull request you sent on Tue, 7 May 2019 19:59:12 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core.git tags/driver-core-5.2-rc1

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/cf482a49af564a3044de3178ea28f10ad5921b38

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-07 17:59 [GIT PULL] Driver core patches for 5.2-rc1 Greg KH
  2019-05-07 20:50 ` pr-tracker-bot
@ 2019-05-09 20:47 ` Linus Torvalds
  2019-05-10  2:36   ` Joel Fernandes
  2019-05-10  6:20   ` Masahiro Yamada
  1 sibling, 2 replies; 7+ messages in thread
From: Linus Torvalds @ 2019-05-09 20:47 UTC (permalink / raw)
  To: Greg KH, Joel Fernandes (Google), Masahiro Yamada
  Cc: Andrew Morton, Linux List Kernel Mailing

[ Ok, this may look irrelevant to people, but I actually notice this
because I do quick rebuilds *all* the time, so the 30s vs 41s
difference is actually something I reacted to and then tried to figure
out... ]

On Tue, May 7, 2019 at 10:59 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> Joel Fernandes (Google) (2):
>       Provide in-kernel headers to make extending kernel easier

Joel and Masahiro,
 this commit does annoying things. It's a small thing, but it ends up
grating on my kernel rebuild times, so I hope somebody can take a look
at it..

Try building a kernel with no changes, and it shouldn't re-link.

HOWEVER.

If you re-make the config in between, the kernel/kheaders_data.tar.xz
is re-generated too. I think it checks timestamps despite having that
"CHK" phase that should verify just contents.

I think the kernel/config_data.gz rules do the same thing, judging by
the output.

I use "make allmodconfig" to re-generate the same kernel config, which
triggers this. The difference between "nothing changed" and "rerun
'make allmodconfig' and nothing _still_ should have changed" is quite
stark:

- nothing changed: rebuild in less than 30s

    [torvalds@i7 linux]$ time make -j32
      DESCEND  objtool
      CALL    scripts/atomic/check-atomics.sh
      CALL    scripts/checksyscalls.sh
      CHK     include/generated/compile.h
      CHK     kernel/kheaders_data.tar.xz
      Building modules, stage 2.
    Kernel: arch/x86/boot/bzImage is ready  (#9)
      MODPOST 7282 modules

    real        0m29.379s
    user        1m50.586s
    sys 0m41.047s

- do (the same) "make allmodconfig" in between, now rebuild time is
just over 41s:

    [torvalds@i7 linux]$ make allmodconfig

    [torvalds@i7 linux]$ time make -j32
    scripts/kconfig/conf  --syncconfig Kconfig
      DESCEND  objtool
      CALL    scripts/atomic/check-atomics.sh
      CALL    scripts/checksyscalls.sh
      CHK     include/generated/compile.h
      GZIP    kernel/config_data.gz
      CHK     kernel/kheaders_data.tar.xz
      CC [M]  kernel/configs.o
      GEN     kernel/kheaders_data.tar.xz
      CC [M]  kernel/kheaders.o
      Building modules, stage 2.
    Kernel: arch/x86/boot/bzImage is ready  (#9)
      MODPOST 7282 modules
      LD [M]  kernel/configs.ko
      LD [M]  kernel/kheaders.ko

    real        0m41.326s
    user        2m17.822s
    sys 0m54.561s

No, this isn't the end of the world, but if somebody sees a simple
solution to avoid that extra ten seconds, I'd appreciate it.

                  Linus

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-09 20:47 ` Linus Torvalds
@ 2019-05-10  2:36   ` Joel Fernandes
  2019-05-10  7:58     ` Masahiro Yamada
  2019-05-10  6:20   ` Masahiro Yamada
  1 sibling, 1 reply; 7+ messages in thread
From: Joel Fernandes @ 2019-05-10  2:36 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Greg KH, Masahiro Yamada, Andrew Morton, kernel-team,
	Linux List Kernel Mailing

On Thu, May 09, 2019 at 01:47:54PM -0700, Linus Torvalds wrote:
> [ Ok, this may look irrelevant to people, but I actually notice this
> because I do quick rebuilds *all* the time, so the 30s vs 41s
> difference is actually something I reacted to and then tried to figure
> out... ]
> 
> On Tue, May 7, 2019 at 10:59 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > Joel Fernandes (Google) (2):
> >       Provide in-kernel headers to make extending kernel easier
> 
> Joel and Masahiro,
>  this commit does annoying things. It's a small thing, but it ends up
> grating on my kernel rebuild times, so I hope somebody can take a look
> at it..
> 
> Try building a kernel with no changes, and it shouldn't re-link.
> 
> HOWEVER.
> 
> If you re-make the config in between, the kernel/kheaders_data.tar.xz
> is re-generated too. I think it checks timestamps despite having that
> "CHK" phase that should verify just contents.
> 
> I think the kernel/config_data.gz rules do the same thing, judging by
> the output.
> 
> I use "make allmodconfig" to re-generate the same kernel config, which
> triggers this. The difference between "nothing changed" and "rerun
> 'make allmodconfig' and nothing _still_ should have changed" is quite
> stark:
[snip]
> No, this isn't the end of the world, but if somebody sees a simple
> solution to avoid that extra ten seconds, I'd appreciate it.

Hi Linus,
The following patch should fix the issue. The patch depends on [1] though. So
that will have to be pulled first (which I believe Greg is going to pull soon
since it is in his pipeline, and Steven Rostedt already Acked it)
[1] https://lore.kernel.org/patchwork/patch/1070199/

For the below patch which fixes this issue, I have tested it and it fixes the
allmodconfig issue. Could you try it out as well? As mentioned above, the
patch at [1] should be applied first. Thanks a lot and let me know how it goes.

(I am going to be on a long haul flight shortly so I may not be available for
next 24-48 hours but will be there after, thanks).
---8<-----------------------

From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
Subject: [PATCH] gen_kheaders: Do not regenerate archive if config is not
 changed

Linus reported that allmodconfig config was causing the kheaders archive
to be regenerated even though the config is the same. This is due to the
fact that the generated config header files are rewritten even if they
were the same from a previous run.

To fix the issue, we ignore changes to these files and use md5sum on
auto.conf to determine if the config really changed. And regenerate the
header archive if it has.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 kernel/gen_kheaders.sh | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index 581b83534587..f621242037f4 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -33,7 +33,7 @@ arch/$SRCARCH/include/
 # Uncomment it for debugging.
 # iter=1
 # if [ ! -f /tmp/iter ]; then echo 1 > /tmp/iter;
-# else; 	iter=$(($(cat /tmp/iter) + 1)); fi
+# else 	iter=$(($(cat /tmp/iter) + 1)); fi
 # find $src_file_list -type f | xargs ls -lR > /tmp/src-ls-$iter
 # find $obj_file_list -type f | xargs ls -lR > /tmp/obj-ls-$iter
 
@@ -43,16 +43,27 @@ arch/$SRCARCH/include/
 pushd $kroot > /dev/null
 src_files_md5="$(find $src_file_list -type f                       |
 		grep -v "include/generated/compile.h"		   |
+		grep -v "include/generated/autoconf.h"		   |
+		grep -v "include/config/auto.conf"		   |
+		grep -v "include/config/auto.conf.cmd"		   |
+		grep -v "include/config/tristate.conf"		   |
 		xargs ls -lR | md5sum | cut -d ' ' -f1)"
 popd > /dev/null
 obj_files_md5="$(find $obj_file_list -type f                       |
 		grep -v "include/generated/compile.h"		   |
+		grep -v "include/generated/autoconf.h"		   |
+		grep -v "include/config/auto.conf"                 |
+		grep -v "include/config/auto.conf.cmd"		   |
+		grep -v "include/config/tristate.conf"		   |
 		xargs ls -lR | md5sum | cut -d ' ' -f1)"
 
+config_md5="$(md5sum include/config/auto.conf | cut -d ' ' -f1)"
+
 if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
 if [ -f kernel/kheaders.md5 ] &&
 	[ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] &&
 	[ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
+	[ "$(cat kernel/kheaders.md5|head -3|tail -1)" == "$config_md5" ] &&
 	[ "$(cat kernel/kheaders.md5|tail -1)" == "$tarfile_md5" ]; then
 		exit
 fi
@@ -82,8 +93,9 @@ find $cpio_dir -type f -print0 |
 
 tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
 
-echo "$src_files_md5" > kernel/kheaders.md5
+echo "$src_files_md5" >  kernel/kheaders.md5
 echo "$obj_files_md5" >> kernel/kheaders.md5
+echo "$config_md5"    >> kernel/kheaders.md5
 echo "$(md5sum $tarfile | cut -d ' ' -f1)" >> kernel/kheaders.md5
 
 rm -rf $cpio_dir
-- 
2.21.0.1020.gf2820cf01a-goog


^ permalink raw reply related	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-09 20:47 ` Linus Torvalds
  2019-05-10  2:36   ` Joel Fernandes
@ 2019-05-10  6:20   ` Masahiro Yamada
  1 sibling, 0 replies; 7+ messages in thread
From: Masahiro Yamada @ 2019-05-10  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Greg KH, Joel Fernandes (Google),
	Andrew Morton, Linux List Kernel Mailing

Hi Linus,


On Fri, May 10, 2019 at 5:50 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> [ Ok, this may look irrelevant to people, but I actually notice this
> because I do quick rebuilds *all* the time, so the 30s vs 41s
> difference is actually something I reacted to and then tried to figure
> out... ]
>
> On Tue, May 7, 2019 at 10:59 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > Joel Fernandes (Google) (2):
> >       Provide in-kernel headers to make extending kernel easier
>
> Joel and Masahiro,
>  this commit does annoying things. It's a small thing, but it ends up
> grating on my kernel rebuild times, so I hope somebody can take a look
> at it..
>
> Try building a kernel with no changes, and it shouldn't re-link.
>
> HOWEVER.
>
> If you re-make the config in between, the kernel/kheaders_data.tar.xz
> is re-generated too. I think it checks timestamps despite having that
> "CHK" phase that should verify just contents.
>
> I think the kernel/config_data.gz rules do the same thing, judging by
> the output.
>
> I use "make allmodconfig" to re-generate the same kernel config, which
> triggers this. The difference between "nothing changed" and "rerun
> 'make allmodconfig' and nothing _still_ should have changed" is quite
> stark:

Yeah, I have had this in my mind for a while.

If you run "make *config" and it happens to be creating
the exactly the same .config, Kconfig should not overwrite
it at all.

If you apply the following two,
I hope you will get the behavior you like.

https://patchwork.kernel.org/patch/10938255/
https://patchwork.kernel.org/patch/10938253/

(1/2 is just a cleanup because I am touching the
same hunk.)

Because I have not sent a Kconfig pull request yet
in the current MW, I will consider to merge them.

Thanks.


> - nothing changed: rebuild in less than 30s
>
>     [torvalds@i7 linux]$ time make -j32
>       DESCEND  objtool
>       CALL    scripts/atomic/check-atomics.sh
>       CALL    scripts/checksyscalls.sh
>       CHK     include/generated/compile.h
>       CHK     kernel/kheaders_data.tar.xz
>       Building modules, stage 2.
>     Kernel: arch/x86/boot/bzImage is ready  (#9)
>       MODPOST 7282 modules
>
>     real        0m29.379s
>     user        1m50.586s
>     sys 0m41.047s
>
> - do (the same) "make allmodconfig" in between, now rebuild time is
> just over 41s:
>
>     [torvalds@i7 linux]$ make allmodconfig
>
>     [torvalds@i7 linux]$ time make -j32
>     scripts/kconfig/conf  --syncconfig Kconfig
>       DESCEND  objtool
>       CALL    scripts/atomic/check-atomics.sh
>       CALL    scripts/checksyscalls.sh
>       CHK     include/generated/compile.h
>       GZIP    kernel/config_data.gz
>       CHK     kernel/kheaders_data.tar.xz
>       CC [M]  kernel/configs.o
>       GEN     kernel/kheaders_data.tar.xz
>       CC [M]  kernel/kheaders.o
>       Building modules, stage 2.
>     Kernel: arch/x86/boot/bzImage is ready  (#9)
>       MODPOST 7282 modules
>       LD [M]  kernel/configs.ko
>       LD [M]  kernel/kheaders.ko
>
>     real        0m41.326s
>     user        2m17.822s
>     sys 0m54.561s
>
> No, this isn't the end of the world, but if somebody sees a simple
> solution to avoid that extra ten seconds, I'd appreciate it.
>
>                   Linus



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-10  2:36   ` Joel Fernandes
@ 2019-05-10  7:58     ` Masahiro Yamada
  2019-05-10 19:04       ` Joel Fernandes
  0 siblings, 1 reply; 7+ messages in thread
From: Masahiro Yamada @ 2019-05-10  7:58 UTC (permalink / raw)
  To: Joel Fernandes
  Cc: Linus Torvalds, Greg KH, Andrew Morton, Cc: Android Kernel,
	Linux List Kernel Mailing

Hi Joel,

On Fri, May 10, 2019 at 11:38 AM Joel Fernandes <joel@joelfernandes.org> wrote:
>
> On Thu, May 09, 2019 at 01:47:54PM -0700, Linus Torvalds wrote:
> > [ Ok, this may look irrelevant to people, but I actually notice this
> > because I do quick rebuilds *all* the time, so the 30s vs 41s
> > difference is actually something I reacted to and then tried to figure
> > out... ]
> >
> > On Tue, May 7, 2019 at 10:59 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> > >
> > > Joel Fernandes (Google) (2):
> > >       Provide in-kernel headers to make extending kernel easier
> >
> > Joel and Masahiro,
> >  this commit does annoying things. It's a small thing, but it ends up
> > grating on my kernel rebuild times, so I hope somebody can take a look
> > at it..
> >
> > Try building a kernel with no changes, and it shouldn't re-link.
> >
> > HOWEVER.
> >
> > If you re-make the config in between, the kernel/kheaders_data.tar.xz
> > is re-generated too. I think it checks timestamps despite having that
> > "CHK" phase that should verify just contents.
> >
> > I think the kernel/config_data.gz rules do the same thing, judging by
> > the output.
> >
> > I use "make allmodconfig" to re-generate the same kernel config, which
> > triggers this. The difference between "nothing changed" and "rerun
> > 'make allmodconfig' and nothing _still_ should have changed" is quite
> > stark:
> [snip]
> > No, this isn't the end of the world, but if somebody sees a simple
> > solution to avoid that extra ten seconds, I'd appreciate it.
>
> Hi Linus,
> The following patch should fix the issue. The patch depends on [1] though. So
> that will have to be pulled first (which I believe Greg is going to pull soon
> since it is in his pipeline, and Steven Rostedt already Acked it)
> [1] https://lore.kernel.org/patchwork/patch/1070199/
>
> For the below patch which fixes this issue, I have tested it and it fixes the
> allmodconfig issue. Could you try it out as well? As mentioned above, the
> patch at [1] should be applied first. Thanks a lot and let me know how it goes.
>
> (I am going to be on a long haul flight shortly so I may not be available for
> next 24-48 hours but will be there after, thanks).
> ---8<-----------------------
>
> From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
> Subject: [PATCH] gen_kheaders: Do not regenerate archive if config is not
>  changed
>
> Linus reported that allmodconfig config was causing the kheaders archive
> to be regenerated even though the config is the same. This is due to the
> fact that the generated config header files are rewritten even if they
> were the same from a previous run.


I will fix the root cause in Kconfig anyway.


This patch is still useful in case
"one of Kconfig files is changed,
but the resulted configuration is still the same"

$ touch Kconfig
$ make -j8


But, you should simplify the code.


> To fix the issue, we ignore changes to these files and use md5sum on
> auto.conf to determine if the config really changed. And regenerate the
> header archive if it has.


Nope. This is really unnecessary.

When CONFIG_FOO_BAR is changed, include/config/foo/bar.h
should have already been touched by Kconfig,
so the change of the kernel configuration
is correctly detected.

If you want to know the this Kbuild magic,
read the comment block of scripts/basic/fixdep.c


Please remove config_md5 and correct the commit log.

Thanks.




> Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> ---
>  kernel/gen_kheaders.sh | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index 581b83534587..f621242037f4 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -33,7 +33,7 @@ arch/$SRCARCH/include/
>  # Uncomment it for debugging.
>  # iter=1
>  # if [ ! -f /tmp/iter ]; then echo 1 > /tmp/iter;
> -# else;        iter=$(($(cat /tmp/iter) + 1)); fi
> +# else         iter=$(($(cat /tmp/iter) + 1)); fi
>  # find $src_file_list -type f | xargs ls -lR > /tmp/src-ls-$iter
>  # find $obj_file_list -type f | xargs ls -lR > /tmp/obj-ls-$iter
>
> @@ -43,16 +43,27 @@ arch/$SRCARCH/include/
>  pushd $kroot > /dev/null
>  src_files_md5="$(find $src_file_list -type f                       |
>                 grep -v "include/generated/compile.h"              |
> +               grep -v "include/generated/autoconf.h"             |
> +               grep -v "include/config/auto.conf"                 |
> +               grep -v "include/config/auto.conf.cmd"             |
> +               grep -v "include/config/tristate.conf"             |
>                 xargs ls -lR | md5sum | cut -d ' ' -f1)"
>  popd > /dev/null
>  obj_files_md5="$(find $obj_file_list -type f                       |
>                 grep -v "include/generated/compile.h"              |
> +               grep -v "include/generated/autoconf.h"             |
> +               grep -v "include/config/auto.conf"                 |
> +               grep -v "include/config/auto.conf.cmd"             |
> +               grep -v "include/config/tristate.conf"             |
>                 xargs ls -lR | md5sum | cut -d ' ' -f1)"
>
> +config_md5="$(md5sum include/config/auto.conf | cut -d ' ' -f1)"
> +
>  if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
>  if [ -f kernel/kheaders.md5 ] &&
>         [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] &&
>         [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
> +       [ "$(cat kernel/kheaders.md5|head -3|tail -1)" == "$config_md5" ] &&
>         [ "$(cat kernel/kheaders.md5|tail -1)" == "$tarfile_md5" ]; then
>                 exit
>  fi
> @@ -82,8 +93,9 @@ find $cpio_dir -type f -print0 |
>
>  tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
>
> -echo "$src_files_md5" > kernel/kheaders.md5
> +echo "$src_files_md5" >  kernel/kheaders.md5
>  echo "$obj_files_md5" >> kernel/kheaders.md5
> +echo "$config_md5"    >> kernel/kheaders.md5
>  echo "$(md5sum $tarfile | cut -d ' ' -f1)" >> kernel/kheaders.md5
>
>  rm -rf $cpio_dir
> --
> 2.21.0.1020.gf2820cf01a-goog
>


--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [GIT PULL] Driver core patches for 5.2-rc1
  2019-05-10  7:58     ` Masahiro Yamada
@ 2019-05-10 19:04       ` Joel Fernandes
  0 siblings, 0 replies; 7+ messages in thread
From: Joel Fernandes @ 2019-05-10 19:04 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Linus Torvalds, Greg KH, Andrew Morton, Cc: Android Kernel,
	Linux List Kernel Mailing

On Fri, May 10, 2019 at 04:58:32PM +0900, Masahiro Yamada wrote:
> Hi Joel,
> 
> On Fri, May 10, 2019 at 11:38 AM Joel Fernandes <joel@joelfernandes.org> wrote:
> >
> > On Thu, May 09, 2019 at 01:47:54PM -0700, Linus Torvalds wrote:
> > > [ Ok, this may look irrelevant to people, but I actually notice this
> > > because I do quick rebuilds *all* the time, so the 30s vs 41s
> > > difference is actually something I reacted to and then tried to figure
> > > out... ]
> > >
> > > On Tue, May 7, 2019 at 10:59 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> > > >
> > > > Joel Fernandes (Google) (2):
> > > >       Provide in-kernel headers to make extending kernel easier
> > >
> > > Joel and Masahiro,
> > >  this commit does annoying things. It's a small thing, but it ends up
> > > grating on my kernel rebuild times, so I hope somebody can take a look
> > > at it..
> > >
> > > Try building a kernel with no changes, and it shouldn't re-link.
> > >
> > > HOWEVER.
> > >
> > > If you re-make the config in between, the kernel/kheaders_data.tar.xz
> > > is re-generated too. I think it checks timestamps despite having that
> > > "CHK" phase that should verify just contents.
> > >
> > > I think the kernel/config_data.gz rules do the same thing, judging by
> > > the output.
> > >
> > > I use "make allmodconfig" to re-generate the same kernel config, which
> > > triggers this. The difference between "nothing changed" and "rerun
> > > 'make allmodconfig' and nothing _still_ should have changed" is quite
> > > stark:
> > [snip]
> > > No, this isn't the end of the world, but if somebody sees a simple
> > > solution to avoid that extra ten seconds, I'd appreciate it.
> >
> > Hi Linus,
> > The following patch should fix the issue. The patch depends on [1] though. So
> > that will have to be pulled first (which I believe Greg is going to pull soon
> > since it is in his pipeline, and Steven Rostedt already Acked it)
> > [1] https://lore.kernel.org/patchwork/patch/1070199/
> >
> > For the below patch which fixes this issue, I have tested it and it fixes the
> > allmodconfig issue. Could you try it out as well? As mentioned above, the
> > patch at [1] should be applied first. Thanks a lot and let me know how it goes.
> >
> > (I am going to be on a long haul flight shortly so I may not be available for
> > next 24-48 hours but will be there after, thanks).
> > ---8<-----------------------
> >
> > From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
> > Subject: [PATCH] gen_kheaders: Do not regenerate archive if config is not
> >  changed
> >
> > Linus reported that allmodconfig config was causing the kheaders archive
> > to be regenerated even though the config is the same. This is due to the
> > fact that the generated config header files are rewritten even if they
> > were the same from a previous run.
> 
> 
> I will fix the root cause in Kconfig anyway.
> 
> 
> This patch is still useful in case
> "one of Kconfig files is changed,
> but the resulted configuration is still the same"
> 
> $ touch Kconfig
> $ make -j8
> 
> 
> But, you should simplify the code.
> 
> 
> > To fix the issue, we ignore changes to these files and use md5sum on
> > auto.conf to determine if the config really changed. And regenerate the
> > header archive if it has.
> 
> 
> Nope. This is really unnecessary.
> 
> When CONFIG_FOO_BAR is changed, include/config/foo/bar.h
> should have already been touched by Kconfig,
> so the change of the kernel configuration
> is correctly detected.
> 
> If you want to know the this Kbuild magic,
> read the comment block of scripts/basic/fixdep.c
> 
> 
> Please remove config_md5 and correct the commit log.

Agreed! Will remove the md5 check and keep the rest of the patch the same
(except the commit log).

thanks!

 - Joel


> 
> Thanks.
> 
> 
> 
> 
> > Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
> > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > ---
> >  kernel/gen_kheaders.sh | 16 ++++++++++++++--
> >  1 file changed, 14 insertions(+), 2 deletions(-)
> >
> > diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> > index 581b83534587..f621242037f4 100755
> > --- a/kernel/gen_kheaders.sh
> > +++ b/kernel/gen_kheaders.sh
> > @@ -33,7 +33,7 @@ arch/$SRCARCH/include/
> >  # Uncomment it for debugging.
> >  # iter=1
> >  # if [ ! -f /tmp/iter ]; then echo 1 > /tmp/iter;
> > -# else;        iter=$(($(cat /tmp/iter) + 1)); fi
> > +# else         iter=$(($(cat /tmp/iter) + 1)); fi
> >  # find $src_file_list -type f | xargs ls -lR > /tmp/src-ls-$iter
> >  # find $obj_file_list -type f | xargs ls -lR > /tmp/obj-ls-$iter
> >
> > @@ -43,16 +43,27 @@ arch/$SRCARCH/include/
> >  pushd $kroot > /dev/null
> >  src_files_md5="$(find $src_file_list -type f                       |
> >                 grep -v "include/generated/compile.h"              |
> > +               grep -v "include/generated/autoconf.h"             |
> > +               grep -v "include/config/auto.conf"                 |
> > +               grep -v "include/config/auto.conf.cmd"             |
> > +               grep -v "include/config/tristate.conf"             |
> >                 xargs ls -lR | md5sum | cut -d ' ' -f1)"
> >  popd > /dev/null
> >  obj_files_md5="$(find $obj_file_list -type f                       |
> >                 grep -v "include/generated/compile.h"              |
> > +               grep -v "include/generated/autoconf.h"             |
> > +               grep -v "include/config/auto.conf"                 |
> > +               grep -v "include/config/auto.conf.cmd"             |
> > +               grep -v "include/config/tristate.conf"             |
> >                 xargs ls -lR | md5sum | cut -d ' ' -f1)"
> >
> > +config_md5="$(md5sum include/config/auto.conf | cut -d ' ' -f1)"
> > +
> >  if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi
> >  if [ -f kernel/kheaders.md5 ] &&
> >         [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] &&
> >         [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
> > +       [ "$(cat kernel/kheaders.md5|head -3|tail -1)" == "$config_md5" ] &&
> >         [ "$(cat kernel/kheaders.md5|tail -1)" == "$tarfile_md5" ]; then
> >                 exit
> >  fi
> > @@ -82,8 +93,9 @@ find $cpio_dir -type f -print0 |
> >
> >  tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
> >
> > -echo "$src_files_md5" > kernel/kheaders.md5
> > +echo "$src_files_md5" >  kernel/kheaders.md5
> >  echo "$obj_files_md5" >> kernel/kheaders.md5
> > +echo "$config_md5"    >> kernel/kheaders.md5
> >  echo "$(md5sum $tarfile | cut -d ' ' -f1)" >> kernel/kheaders.md5
> >
> >  rm -rf $cpio_dir
> > --
> > 2.21.0.1020.gf2820cf01a-goog
> >
> 
> 
> --
> Best Regards
> Masahiro Yamada

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-05-10 19:04 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-07 17:59 [GIT PULL] Driver core patches for 5.2-rc1 Greg KH
2019-05-07 20:50 ` pr-tracker-bot
2019-05-09 20:47 ` Linus Torvalds
2019-05-10  2:36   ` Joel Fernandes
2019-05-10  7:58     ` Masahiro Yamada
2019-05-10 19:04       ` Joel Fernandes
2019-05-10  6:20   ` Masahiro Yamada

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).