lvs-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [GIT PULL] sysctl changes for v6.6-rc1
@ 2023-08-29 20:44 Luis Chamberlain
  2023-08-30  3:58 ` pr-tracker-bot
  2023-09-06 11:58 ` Alexey Gladkov
  0 siblings, 2 replies; 4+ messages in thread
From: Luis Chamberlain @ 2023-08-29 20:44 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Joel Granados, linux-fsdevel, rds-devel, David S. Miller,
	Florian Westphal, willy, Jan Karcher, Wen Gu, Simon Horman,
	Tony Lu, linux-wpan, Matthieu Baerts, Christian Borntraeger,
	mptcp, Heiko Carstens, Stefan Schmidt, Will Deacon,
	Julian Anastasov, netfilter-devel, Joerg Reuter, linux-kernel,
	Alexander Gordeev, linux-sctp, Xin Long, Herbert Xu, linux-hams

The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:

  Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1

for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:

  sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)

----------------------------------------------------------------
sysctl-6.6-rc1

Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
placings sysctls to their own sybsystem or file to help avoid merge conflicts.
Matthew Wilcox pointed out though that if we're going to do that we might as
well also *save* space while at it and try to remove the extra last sysctl
entry added at the end of each array, a sentintel, instead of bloating the
kernel by adding a new sentinel with each array moved.

Doing that was not so trivial, and has required slowing down the moves of
kernel/sysctl.c arrays and measuring the impact on size by each new move.

The complex part of the effort to help reduce the size of each sysctl is being
done by the patient work of el señor Don Joel Granados. A lot of this is truly
painful code refactoring and testing and then trying to measure the savings of
each move and removing the sentinels. Although Joel already has code which does
most of this work, experience with sysctl moves in the past shows is we need to
be careful due to the slew of odd build failures that are possible due to the
amount of random Kconfig options sysctls use.

To that end Joel's work is split by first addressing the major housekeeping
needed to remove the sentinels, which is part of this merge request. The rest
of the work to actually remove the sentinels will be done later in future
kernel releases.

At first I was only going to send his first 7 patches of his patch series,
posted 1 month ago, but in retrospect due to the testing the changes have
received in linux-next and the minor changes they make this goes with the
entire set of patches Joel had planned: just sysctl house keeping. There are
networking changes but these are part of the house keeping too.

The preliminary math is showing this will all help reduce the overall build
time size of the kernel and run time memory consumed by the kernel by about
~64 bytes per array where we are able to remove each sentinel in the future.
That also means there is no more bloating the kernel with the extra ~64 bytes
per array moved as no new sentinels are created.

Most of this has been in linux-next for about a month, the last 7 patches took
a minor refresh 2 week ago based on feedback.

----------------------------------------------------------------
Joel Granados (14):
      sysctl: Prefer ctl_table_header in proc_sysctl
      sysctl: Use ctl_table_header in list_for_each_table_entry
      sysctl: Add ctl_table_size to ctl_table_header
      sysctl: Add size argument to init_header
      sysctl: Add a size arg to __register_sysctl_table
      sysctl: Add size to register_sysctl
      sysctl: Add size arg to __register_sysctl_init
      sysctl: Add size to register_net_sysctl function
      ax.25: Update to register_net_sysctl_sz
      netfilter: Update to register_net_sysctl_sz
      networking: Update to register_net_sysctl_sz
      vrf: Update to register_net_sysctl_sz
      sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
      sysctl: Use ctl_table_size as stopping criteria for list macro

 arch/arm64/kernel/armv8_deprecated.c    |  2 +-
 arch/s390/appldata/appldata_base.c      |  2 +-
 drivers/net/vrf.c                       |  3 +-
 fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
 include/linux/sysctl.h                  | 31 +++++++++---
 include/net/ipv6.h                      |  2 +
 include/net/net_namespace.h             | 10 ++--
 ipc/ipc_sysctl.c                        |  4 +-
 ipc/mq_sysctl.c                         |  4 +-
 kernel/ucount.c                         |  5 +-
 net/ax25/sysctl_net_ax25.c              |  3 +-
 net/bridge/br_netfilter_hooks.c         |  3 +-
 net/core/neighbour.c                    |  8 ++-
 net/core/sysctl_net_core.c              |  3 +-
 net/ieee802154/6lowpan/reassembly.c     |  8 ++-
 net/ipv4/devinet.c                      |  3 +-
 net/ipv4/ip_fragment.c                  |  3 +-
 net/ipv4/route.c                        |  8 ++-
 net/ipv4/sysctl_net_ipv4.c              |  3 +-
 net/ipv4/xfrm4_policy.c                 |  3 +-
 net/ipv6/addrconf.c                     |  3 +-
 net/ipv6/icmp.c                         |  5 ++
 net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
 net/ipv6/reassembly.c                   |  3 +-
 net/ipv6/route.c                        |  9 ++++
 net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
 net/ipv6/xfrm6_policy.c                 |  3 +-
 net/mpls/af_mpls.c                      |  6 ++-
 net/mptcp/ctrl.c                        |  3 +-
 net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
 net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
 net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
 net/netfilter/nf_conntrack_standalone.c |  4 +-
 net/netfilter/nf_log.c                  |  7 +--
 net/rds/tcp.c                           |  3 +-
 net/sctp/sysctl.c                       |  4 +-
 net/smc/smc_sysctl.c                    |  3 +-
 net/sysctl_net.c                        | 26 +++++++---
 net/unix/sysctl_net_unix.c              |  3 +-
 net/xfrm/xfrm_sysctl.c                  |  8 ++-
 40 files changed, 222 insertions(+), 113 deletions(-)


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [GIT PULL] sysctl changes for v6.6-rc1
  2023-08-29 20:44 [GIT PULL] sysctl changes for v6.6-rc1 Luis Chamberlain
@ 2023-08-30  3:58 ` pr-tracker-bot
  2023-09-06 11:58 ` Alexey Gladkov
  1 sibling, 0 replies; 4+ messages in thread
From: pr-tracker-bot @ 2023-08-30  3:58 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: Linus Torvalds, Joel Granados, linux-fsdevel, rds-devel,
	David S. Miller, Florian Westphal, willy, Jan Karcher, Wen Gu,
	Simon Horman, Tony Lu, linux-wpan, Matthieu Baerts,
	Christian Borntraeger, mptcp, Heiko Carstens, Stefan Schmidt,
	Will Deacon, Julian Anastasov, netfilter-devel, Joerg Reuter,
	linux-kernel, Alexander Gordeev, linux-sctp, Xin Long, Herbert

The pull request you sent on Tue, 29 Aug 2023 13:44:55 -0700:

> git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/adfd671676c922bada16477eb68b5eb5f065addc

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [GIT PULL] sysctl changes for v6.6-rc1
  2023-08-29 20:44 [GIT PULL] sysctl changes for v6.6-rc1 Luis Chamberlain
  2023-08-30  3:58 ` pr-tracker-bot
@ 2023-09-06 11:58 ` Alexey Gladkov
  2023-09-06 12:35   ` Joel Granados
  1 sibling, 1 reply; 4+ messages in thread
From: Alexey Gladkov @ 2023-09-06 11:58 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: Linus Torvalds, Joel Granados, linux-fsdevel, rds-devel,
	David S. Miller, Florian Westphal, willy, Jan Karcher, Wen Gu,
	Simon Horman, Tony Lu, linux-wpan, Matthieu Baerts,
	Christian Borntraeger, mptcp, Heiko Carstens, Stefan Schmidt,
	Will Deacon, Julian Anastasov, netfilter-devel, Joerg Reuter,
	linux-kernel, A

On Tue, Aug 29, 2023 at 01:44:55PM -0700, Luis Chamberlain wrote:
> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> 
>   Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> 
> are available in the Git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1
> 
> for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:
> 
>   sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)
> 
> ----------------------------------------------------------------
> sysctl-6.6-rc1
> 
> Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
> placings sysctls to their own sybsystem or file to help avoid merge conflicts.
> Matthew Wilcox pointed out though that if we're going to do that we might as
> well also *save* space while at it and try to remove the extra last sysctl
> entry added at the end of each array, a sentintel, instead of bloating the
> kernel by adding a new sentinel with each array moved.
> 
> Doing that was not so trivial, and has required slowing down the moves of
> kernel/sysctl.c arrays and measuring the impact on size by each new move.
> 
> The complex part of the effort to help reduce the size of each sysctl is being
> done by the patient work of el señor Don Joel Granados. A lot of this is truly
> painful code refactoring and testing and then trying to measure the savings of
> each move and removing the sentinels. Although Joel already has code which does
> most of this work, experience with sysctl moves in the past shows is we need to
> be careful due to the slew of odd build failures that are possible due to the
> amount of random Kconfig options sysctls use.
> 
> To that end Joel's work is split by first addressing the major housekeeping
> needed to remove the sentinels, which is part of this merge request. The rest
> of the work to actually remove the sentinels will be done later in future
> kernel releases.

This is very interesting for me. I'm also refactoring sysctl based on
discussion with Linus a while ago.

Could you please add me to the Cc in the next patches?

> At first I was only going to send his first 7 patches of his patch series,
> posted 1 month ago, but in retrospect due to the testing the changes have
> received in linux-next and the minor changes they make this goes with the
> entire set of patches Joel had planned: just sysctl house keeping. There are
> networking changes but these are part of the house keeping too.
> 
> The preliminary math is showing this will all help reduce the overall build
> time size of the kernel and run time memory consumed by the kernel by about
> ~64 bytes per array where we are able to remove each sentinel in the future.
> That also means there is no more bloating the kernel with the extra ~64 bytes
> per array moved as no new sentinels are created.
> 
> Most of this has been in linux-next for about a month, the last 7 patches took
> a minor refresh 2 week ago based on feedback.
> 
> ----------------------------------------------------------------
> Joel Granados (14):
>       sysctl: Prefer ctl_table_header in proc_sysctl
>       sysctl: Use ctl_table_header in list_for_each_table_entry
>       sysctl: Add ctl_table_size to ctl_table_header
>       sysctl: Add size argument to init_header
>       sysctl: Add a size arg to __register_sysctl_table
>       sysctl: Add size to register_sysctl
>       sysctl: Add size arg to __register_sysctl_init
>       sysctl: Add size to register_net_sysctl function
>       ax.25: Update to register_net_sysctl_sz
>       netfilter: Update to register_net_sysctl_sz
>       networking: Update to register_net_sysctl_sz
>       vrf: Update to register_net_sysctl_sz
>       sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
>       sysctl: Use ctl_table_size as stopping criteria for list macro
> 
>  arch/arm64/kernel/armv8_deprecated.c    |  2 +-
>  arch/s390/appldata/appldata_base.c      |  2 +-
>  drivers/net/vrf.c                       |  3 +-
>  fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
>  include/linux/sysctl.h                  | 31 +++++++++---
>  include/net/ipv6.h                      |  2 +
>  include/net/net_namespace.h             | 10 ++--
>  ipc/ipc_sysctl.c                        |  4 +-
>  ipc/mq_sysctl.c                         |  4 +-
>  kernel/ucount.c                         |  5 +-
>  net/ax25/sysctl_net_ax25.c              |  3 +-
>  net/bridge/br_netfilter_hooks.c         |  3 +-
>  net/core/neighbour.c                    |  8 ++-
>  net/core/sysctl_net_core.c              |  3 +-
>  net/ieee802154/6lowpan/reassembly.c     |  8 ++-
>  net/ipv4/devinet.c                      |  3 +-
>  net/ipv4/ip_fragment.c                  |  3 +-
>  net/ipv4/route.c                        |  8 ++-
>  net/ipv4/sysctl_net_ipv4.c              |  3 +-
>  net/ipv4/xfrm4_policy.c                 |  3 +-
>  net/ipv6/addrconf.c                     |  3 +-
>  net/ipv6/icmp.c                         |  5 ++
>  net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
>  net/ipv6/reassembly.c                   |  3 +-
>  net/ipv6/route.c                        |  9 ++++
>  net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
>  net/ipv6/xfrm6_policy.c                 |  3 +-
>  net/mpls/af_mpls.c                      |  6 ++-
>  net/mptcp/ctrl.c                        |  3 +-
>  net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
>  net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
>  net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
>  net/netfilter/nf_conntrack_standalone.c |  4 +-
>  net/netfilter/nf_log.c                  |  7 +--
>  net/rds/tcp.c                           |  3 +-
>  net/sctp/sysctl.c                       |  4 +-
>  net/smc/smc_sysctl.c                    |  3 +-
>  net/sysctl_net.c                        | 26 +++++++---
>  net/unix/sysctl_net_unix.c              |  3 +-
>  net/xfrm/xfrm_sysctl.c                  |  8 ++-
>  40 files changed, 222 insertions(+), 113 deletions(-)

-- 
Rgrds, legion


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [GIT PULL] sysctl changes for v6.6-rc1
  2023-09-06 11:58 ` Alexey Gladkov
@ 2023-09-06 12:35   ` Joel Granados
  0 siblings, 0 replies; 4+ messages in thread
From: Joel Granados @ 2023-09-06 12:35 UTC (permalink / raw)
  To: Alexey Gladkov
  Cc: Luis Chamberlain, Linus Torvalds, Joel Granados, linux-fsdevel,
	rds-devel, David S. Miller, Florian Westphal, willy, Jan Karcher,
	Wen Gu, Simon Horman, Tony Lu, linux-wpan, Matthieu Baerts,
	Christian Borntraeger, mptcp, Heiko Carstens, Stefan Schmidt,
	Will Deacon, Julian Anastasov, netfilter-devel, Joerg Reuter

[-- Attachment #1: Type: text/plain, Size: 6752 bytes --]

On Wed, Sep 06, 2023 at 01:58:49PM +0200, Alexey Gladkov wrote:
> On Tue, Aug 29, 2023 at 01:44:55PM -0700, Luis Chamberlain wrote:
> > The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> > 
> >   Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> > 
> > are available in the Git repository at:
> > 
> >   git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1
> > 
> > for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:
> > 
> >   sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)
> > 
> > ----------------------------------------------------------------
> > sysctl-6.6-rc1
> > 
> > Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
> > placings sysctls to their own sybsystem or file to help avoid merge conflicts.
> > Matthew Wilcox pointed out though that if we're going to do that we might as
> > well also *save* space while at it and try to remove the extra last sysctl
> > entry added at the end of each array, a sentintel, instead of bloating the
> > kernel by adding a new sentinel with each array moved.
> > 
> > Doing that was not so trivial, and has required slowing down the moves of
> > kernel/sysctl.c arrays and measuring the impact on size by each new move.
> > 
> > The complex part of the effort to help reduce the size of each sysctl is being
> > done by the patient work of el señor Don Joel Granados. A lot of this is truly
> > painful code refactoring and testing and then trying to measure the savings of
> > each move and removing the sentinels. Although Joel already has code which does
> > most of this work, experience with sysctl moves in the past shows is we need to
> > be careful due to the slew of odd build failures that are possible due to the
> > amount of random Kconfig options sysctls use.
> > 
> > To that end Joel's work is split by first addressing the major housekeeping
> > needed to remove the sentinels, which is part of this merge request. The rest
> > of the work to actually remove the sentinels will be done later in future
> > kernel releases.
> 
> This is very interesting for me. I'm also refactoring sysctl based on
> discussion with Linus a while ago.
> 
> Could you please add me to the Cc in the next patches?
I just sent the next batch for this set for review. You can see it here
https://lore.kernel.org/all/20230906-jag-sysctl_remove_empty_elem_arch-v1-0-3935d4854248@samsung.com/
Will add you for the next ones.

Best

> 
> > At first I was only going to send his first 7 patches of his patch series,
> > posted 1 month ago, but in retrospect due to the testing the changes have
> > received in linux-next and the minor changes they make this goes with the
> > entire set of patches Joel had planned: just sysctl house keeping. There are
> > networking changes but these are part of the house keeping too.
> > 
> > The preliminary math is showing this will all help reduce the overall build
> > time size of the kernel and run time memory consumed by the kernel by about
> > ~64 bytes per array where we are able to remove each sentinel in the future.
> > That also means there is no more bloating the kernel with the extra ~64 bytes
> > per array moved as no new sentinels are created.
> > 
> > Most of this has been in linux-next for about a month, the last 7 patches took
> > a minor refresh 2 week ago based on feedback.
> > 
> > ----------------------------------------------------------------
> > Joel Granados (14):
> >       sysctl: Prefer ctl_table_header in proc_sysctl
> >       sysctl: Use ctl_table_header in list_for_each_table_entry
> >       sysctl: Add ctl_table_size to ctl_table_header
> >       sysctl: Add size argument to init_header
> >       sysctl: Add a size arg to __register_sysctl_table
> >       sysctl: Add size to register_sysctl
> >       sysctl: Add size arg to __register_sysctl_init
> >       sysctl: Add size to register_net_sysctl function
> >       ax.25: Update to register_net_sysctl_sz
> >       netfilter: Update to register_net_sysctl_sz
> >       networking: Update to register_net_sysctl_sz
> >       vrf: Update to register_net_sysctl_sz
> >       sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
> >       sysctl: Use ctl_table_size as stopping criteria for list macro
> > 
> >  arch/arm64/kernel/armv8_deprecated.c    |  2 +-
> >  arch/s390/appldata/appldata_base.c      |  2 +-
> >  drivers/net/vrf.c                       |  3 +-
> >  fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
> >  include/linux/sysctl.h                  | 31 +++++++++---
> >  include/net/ipv6.h                      |  2 +
> >  include/net/net_namespace.h             | 10 ++--
> >  ipc/ipc_sysctl.c                        |  4 +-
> >  ipc/mq_sysctl.c                         |  4 +-
> >  kernel/ucount.c                         |  5 +-
> >  net/ax25/sysctl_net_ax25.c              |  3 +-
> >  net/bridge/br_netfilter_hooks.c         |  3 +-
> >  net/core/neighbour.c                    |  8 ++-
> >  net/core/sysctl_net_core.c              |  3 +-
> >  net/ieee802154/6lowpan/reassembly.c     |  8 ++-
> >  net/ipv4/devinet.c                      |  3 +-
> >  net/ipv4/ip_fragment.c                  |  3 +-
> >  net/ipv4/route.c                        |  8 ++-
> >  net/ipv4/sysctl_net_ipv4.c              |  3 +-
> >  net/ipv4/xfrm4_policy.c                 |  3 +-
> >  net/ipv6/addrconf.c                     |  3 +-
> >  net/ipv6/icmp.c                         |  5 ++
> >  net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
> >  net/ipv6/reassembly.c                   |  3 +-
> >  net/ipv6/route.c                        |  9 ++++
> >  net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
> >  net/ipv6/xfrm6_policy.c                 |  3 +-
> >  net/mpls/af_mpls.c                      |  6 ++-
> >  net/mptcp/ctrl.c                        |  3 +-
> >  net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
> >  net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
> >  net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
> >  net/netfilter/nf_conntrack_standalone.c |  4 +-
> >  net/netfilter/nf_log.c                  |  7 +--
> >  net/rds/tcp.c                           |  3 +-
> >  net/sctp/sysctl.c                       |  4 +-
> >  net/smc/smc_sysctl.c                    |  3 +-
> >  net/sysctl_net.c                        | 26 +++++++---
> >  net/unix/sysctl_net_unix.c              |  3 +-
> >  net/xfrm/xfrm_sysctl.c                  |  8 ++-
> >  40 files changed, 222 insertions(+), 113 deletions(-)
> 
> -- 
> Rgrds, legion
> 

-- 

Joel Granados

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2023-09-06 12:35 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-29 20:44 [GIT PULL] sysctl changes for v6.6-rc1 Luis Chamberlain
2023-08-30  3:58 ` pr-tracker-bot
2023-09-06 11:58 ` Alexey Gladkov
2023-09-06 12:35   ` Joel Granados

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).