linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: x86@kernel.org, Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	Paolo Bonzini <pbonzini@redhat.com>, Wei Liu <wei.liu@kernel.org>,
	Arjan van de Ven <arjan@linux.intel.com>,
	Juergen Gross <jgross@suse.com>
Subject: [patch 00/58] x86/apic: Decrapification and static calls
Date: Tue, 18 Jul 2023 01:14:33 +0200 (CEST)	[thread overview]
Message-ID: <20230717223049.327865981@linutronix.de> (raw)

Hi!

While working on a sane topology evaluation mechanism, which addresses the
short-comings of the existing tragedy held together with duct-tape and
hay-wire, I ran into the issue that quite some of this tragedy is deeply
embedded in the APIC code and uses an impenetrable maze of callbacks which
might or might not be correct at the point where the CPUs are registered
via MPPARSE or ACPI/MADT.

This made me look deeper and the findings were anything but pretty.
Redundant per CPU variables, completely unused code, needless complexity
all over the place. The most amazing gem was:

	   physid_mask_t tmp;  	    // 32bytes on stack

	   apic->magic(&tmp, bit);  // Zeros tmp and sets bit
	   physids_or(real_map, real_map, tmp);

Definitely hard to come up with a more complex way of setting a bit in a
bitmap. Followed suit by the evaluation of the boot cpu APIC ID which
consists of more hacks than sensible code.

So I stopped working on the topology stuff and decided to do an overhaul of
the APIC code first. Cleaning up old gunk which dates back to the early SMP
days, making the CPU registration halfways understandable and then going
through all APIC callbacks to figure out what they actually do and whether
they are required at all. There is also quite some overhead through the
indirect calls and some of them are actually even pointlessly indirected
twice. At some point Peter yelled static_call() at me and that's what I
finally ended up implementing.

This builds and boots on 32bit and 64bit, but obviously needs a larger test
base especially on those old 32bit systems which are just museum pieces.

I have neither evaluated whether this has a measurable impact, but that's
something I leave to the perfomance teams. Definitely less indirect calls
in hotpaths are a win by definition.

Talking about those museums pieces and the related historic maze, I really
have to bring up the question again, whether we should finally kill support
for the museum CPUs and move on.

Ideally we remove 32bit support alltogether. I know the answer... :(

But what I really want to do is to make x86 SMP only. The amount of
#ifdeffery and hacks to keep the UP support alive is amazing. And we do this
just for the sake that it runs on some 25+ years old hardware for absolutely
zero value. It'd be not the first architecture to go SMP=y.

Yes, we "support" Alpha, PARISC, Itanic and other oddballs too, but that's
completely different. They are not getting new hardware every other day and
the main impact on the kernel as a whole is mostly static. They are
sometimes in the way of generalizing things in the core code. Other than
that their architecture code is self contained and they can tinker on it as
they see fit or let it slowly bitrot like Itanic.

But x86 is (still) alive and being extended and expanded. That means that
any refactoring of common infrastructure has to take the broken hardware
museum into account. It's doable, but it's not pretty and of really
questionable value. I wouldn't mind if there were a bunch of museum
attendants actively working on it with taste, but that's obviously wishful
thinking. We are even short of people with taste who work on contemporary
hardware support...

While I cursed myself at some point during this work for having merged
i386/x86_64 back then, I still think that it was the correct decision at
that point in time and saved us a lot of trouble. It admittedly added some
trouble which we would not have now, but it avoided the insanity of having
to maintain two trees with different bugs and "fixes" for the very same
problems. TBH quite some of the horrors which I just removed came out of
the x86/64 side. The oddballs of i386 early SMP support are a horror on
their own of course.

As we made that decision more than 15 years [!] ago, it's about time to make
new decisions.

Vented enough.

I'm sure that I broke things on the way, but we can't just continue with
the current mess and add duct tape over hay-wire over duct-tape over
hay-wire forever. At some point we need to bite the bullet and get rid
of the historical nonsense even if it's painful. That point is now.

So 58 patches and a lot of cursing later:

 58 files changed, 744 insertions(+), 1348 deletions(-)

Despite adding the new static call mechanics this endeavour deletes 600
lines of hilarities. There are more of those, but they need to be addressed
separately. Quite some of them in course of the topology evaluation rework,
which so far sports a negative diffstat too.

Now I need a break and a stiff drink to get rid of the bad taste and the
nightmares caused by this.

The series is also available from git:

  git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git x86/apic

Thanks,

	tglx
---
 hyperv/hv_apic.c             |   26 +-
 hyperv/hv_init.c             |    2 
 hyperv/hv_spinlock.c         |    2 
 hyperv/hv_vtl.c              |    2 
 include/asm/apic.h           |  250 +++++++++++++----------
 include/asm/io_apic.h        |    7 
 include/asm/mpspec.h         |   28 --
 include/asm/processor.h      |    1 
 include/asm/smp.h            |   11 -
 kernel/acpi/boot.c           |   12 -
 kernel/apic/Makefile         |    2 
 kernel/apic/apic.c           |  453 +++++++++++++------------------------------
 kernel/apic/apic_common.c    |   21 +
 kernel/apic/apic_flat_64.c   |   80 +------
 kernel/apic/apic_noop.c      |   91 +-------
 kernel/apic/apic_numachip.c  |   50 ----
 kernel/apic/bigsmp_32.c      |   89 +-------
 kernel/apic/hw_nmi.c         |    4 
 kernel/apic/init.c           |  101 +++++++++
 kernel/apic/io_apic.c        |   30 +-
 kernel/apic/ipi.c            |  176 +++++++---------
 kernel/apic/local.h          |   30 ++
 kernel/apic/msi.c            |    2 
 kernel/apic/probe_32.c       |  117 ++---------
 kernel/apic/probe_64.c       |   18 -
 kernel/apic/vector.c         |   16 -
 kernel/apic/x2apic_cluster.c |   23 --
 kernel/apic/x2apic_phys.c    |   74 ++-----
 kernel/apic/x2apic_uv_x.c    |   51 ----
 kernel/cpu/acrn.c            |    2 
 kernel/cpu/amd.c             |    2 
 kernel/cpu/common.c          |    2 
 kernel/cpu/hygon.c           |    3 
 kernel/cpu/mce/amd.c         |    2 
 kernel/cpu/mce/inject.c      |    3 
 kernel/cpu/mce/threshold.c   |    2 
 kernel/cpu/mshyperv.c        |    4 
 kernel/devicetree.c          |   21 -
 kernel/irq.c                 |   14 -
 kernel/irq_work.c            |    4 
 kernel/jailhouse.c           |    6 
 kernel/kvm.c                 |   12 -
 kernel/mpparse.c             |    6 
 kernel/nmi_selftest.c        |    2 
 kernel/setup.c               |    6 
 kernel/setup_percpu.c        |   10 
 kernel/sev.c                 |    2 
 kernel/smp.c                 |   10 
 kernel/smpboot.c             |  115 ----------
 kernel/vsmp_64.c             |    2 
 kvm/vmx/posted_intr.c        |    2 
 kvm/vmx/vmx.c                |    2 
 mm/srat.c                    |    5 
 pci/xen.c                    |    2 
 platform/uv/uv_nmi.c         |    2 
 xen/apic.c                   |   76 ++-----
 xen/enlighten_hvm.c          |    2 
 xen/smp_pv.c                 |    2 
 58 files changed, 744 insertions(+), 1348 deletions(-)


             reply	other threads:[~2023-07-17 23:14 UTC|newest]

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-17 23:14 Thomas Gleixner [this message]
2023-07-17 23:14 ` [patch 01/58] x86/cpu: Make identify_boot_cpu() static Thomas Gleixner
2023-07-17 23:14 ` [patch 02/58] x86/cpu: Remove unused physid_*() nonsense Thomas Gleixner
2023-07-17 23:14 ` [patch 03/58] x86/apic: Rename disable_apic Thomas Gleixner
2023-07-17 23:14 ` [patch 04/58] x86/apic/ioapic: Rename skip_ioapic_setup Thomas Gleixner
2023-07-17 23:14 ` [patch 05/58] x86/apic: Remove pointless x86_bios_cpu_apicid Thomas Gleixner
2023-07-17 23:14 ` [patch 06/58] x86/apic: Get rid of hard_smp_processor_id() Thomas Gleixner
2023-07-17 23:14 ` [patch 07/58] x86/apic: Remove unused max_physical_apicid Thomas Gleixner
2023-07-17 23:14 ` [patch 08/58] x86/apic: Nuke unused apic::inquire_remote_apic() Thomas Gleixner
2023-07-17 23:14 ` [patch 09/58] x86/apic: Get rid of boot_cpu_physical_apicid madness Thomas Gleixner
2023-07-17 23:14 ` [patch 10/58] x86/apic: Register boot CPU APIC early Thomas Gleixner
2023-07-17 23:14 ` [patch 11/58] x86/apic: Remove the pointless APIC version check Thomas Gleixner
2023-07-17 23:14 ` [patch 12/58] x86/of: Fix the APIC address registration Thomas Gleixner
2023-07-17 23:14 ` [patch 13/58] x86/apic: Make some APIC init functions bool Thomas Gleixner
2023-07-17 23:14 ` [patch 14/58] x86/apic: Split register_apic_address() Thomas Gleixner
2023-07-17 23:14 ` [patch 15/58] x86/apic: Sanitize APIC address setup Thomas Gleixner
2023-07-17 23:14 ` [patch 16/58] x86/apic: Sanitize num_processors handling Thomas Gleixner
2023-07-17 23:15 ` [patch 17/58] x86/apic: Nuke another processor check Thomas Gleixner
2023-07-17 23:15 ` [patch 18/58] x86/apic: Remove check_phys_apicid_present() Thomas Gleixner
2023-07-17 23:15 ` [patch 19/58] x86/apic: Get rid of apic_phys Thomas Gleixner
2023-07-18 13:11   ` Thomas Gleixner
2023-07-20  4:18     ` Michael Kelley (LINUX)
2023-07-20  8:03       ` Thomas Gleixner
2023-07-21  4:17         ` Michael Kelley (LINUX)
2023-07-17 23:15 ` [patch 20/58] x86/apic/32: Sanitize logical APIC ID handling Thomas Gleixner
2023-07-17 23:15 ` [patch 21/58] x86/apic/32: Remove x86_cpu_to_logical_apicid Thomas Gleixner
2023-07-17 23:15 ` [patch 22/58] x86/apic/ipi: Code cleanup Thomas Gleixner
2023-07-17 23:15 ` [patch 23/58] x86/apic: Mop up early_per_cpu() abuse Thomas Gleixner
2023-07-17 23:15 ` [patch 24/58] x86/apic/32: Remove pointless default_acpi_madt_oem_check() Thomas Gleixner
2023-07-18  4:28   ` Juergen Gross
2023-07-17 23:15 ` [patch 25/58] x86/apic/32: Decrapify the def_bigsmp mechanism Thomas Gleixner
2023-07-17 23:15 ` [patch 26/58] x86/apic/32: Remove bigsmp_cpu_present_to_apicid() Thomas Gleixner
2023-07-17 23:15 ` [patch 27/58] x86/apic: Nuke empty init_apic_ldr() callbacks Thomas Gleixner
2023-07-17 23:15 ` [patch 28/58] x86/apic: Nuke apic::apicid_to_cpu_present() Thomas Gleixner
2023-07-17 23:15 ` [patch 29/58] x86/ioapic/32: Decrapify phys_id_present_map operation Thomas Gleixner
2023-07-17 23:15 ` [patch 30/58] x86/apic: Mop up *setup_apic_routing() Thomas Gleixner
2023-07-17 23:15 ` [patch 31/58] x86/apic: Mop up apic::apic_id_registered() Thomas Gleixner
2023-07-17 23:15 ` [patch 32/58] x86/apic/ipi: Tidy up the code and fixup comments Thomas Gleixner
2023-07-17 23:15 ` [patch 33/58] x86/apic: Consolidate wait_icr_idle() implementations Thomas Gleixner
2023-07-17 23:15 ` [patch 34/58] x86/apic: Allow apic::wait_icr_idle() to be NULL Thomas Gleixner
2023-07-17 23:15 ` [patch 35/58] x86/apic: Allow apic::safe_wait_icr_idle() " Thomas Gleixner
2023-07-17 23:15 ` [patch 36/58] x86/apic: Move safe wait_icr_idle() next to apic_mem_wait_icr_idle() Thomas Gleixner
2023-07-17 23:15 ` [patch 37/58] x86/apic/uv: Get rid of wrapper callbacks Thomas Gleixner
2023-07-17 23:15 ` [patch 38/58] x86/apic/x2apic: Share all common IPI functions Thomas Gleixner
2023-07-17 23:15 ` [patch 39/58] x86/apic/64: Uncopypaste probing Thomas Gleixner
2023-07-17 23:15 ` [patch 40/58] x86/apic: Wrap APIC ID validation into an inline Thomas Gleixner
2023-07-17 23:15 ` [patch 41/58] x86/apic: Add max_apic_id member Thomas Gleixner
2023-07-18  0:19   ` Linus Torvalds
2023-07-18  7:47     ` Thomas Gleixner
2023-07-18 15:54       ` Thomas Gleixner
2023-07-18 16:06       ` Linus Torvalds
2023-07-18 18:59         ` Thomas Gleixner
2023-07-17 23:15 ` [patch 42/58] x86/apic: Simplify X2APIC ID validation Thomas Gleixner
2023-07-17 23:15 ` [patch 43/58] x86/apic: Prepare x2APIC for using apic::max_apic_id Thomas Gleixner
2023-07-17 23:15 ` [patch 44/58] x86/apic: Sanitize APID ID range validation Thomas Gleixner
2023-07-17 23:15 ` [patch 45/58] x86/apic: Remove pointless NULL initializations Thomas Gleixner
2023-07-17 23:15 ` [patch 46/58] x86/apic/noop: Tidy up the code Thomas Gleixner
2023-07-17 23:15 ` [patch 47/58] x86/apic: Remove pointless arguments from [native_]eoi_write() Thomas Gleixner
2023-07-23 22:45   ` Wei Liu
2023-07-17 23:15 ` [patch 48/58] x86/apic: Nuke ack_APIC_irq() Thomas Gleixner
2023-07-23 22:45   ` Wei Liu
2023-07-17 23:15 ` [patch 49/58] x86/apic: Wrap apic->native_eoi() into a helper Thomas Gleixner
2023-07-17 23:15 ` [patch 50/58] x86/apic: Provide common init infrastructure Thomas Gleixner
2023-07-18 21:29   ` Peter Keresztes Schmidt
2023-07-18 21:51     ` Thomas Gleixner
2023-07-17 23:15 ` [patch 51/58] x86/apic: Provide apic_update_callback() Thomas Gleixner
2023-07-21  4:27   ` Michael Kelley (LINUX)
2023-07-17 23:15 ` [patch 52/58] x86/apic: Replace acpi_wake_cpu_handler_update() and apic_set_eoi_cb() Thomas Gleixner
2023-07-23 22:55   ` Wei Liu
2023-07-17 23:15 ` [patch 53/58] x86/apic: Convert other overrides to apic_update_callback() Thomas Gleixner
2023-07-21 17:49   ` Michael Kelley (LINUX)
2023-07-23 12:32     ` Thomas Gleixner
2023-07-17 23:15 ` [patch 54/58] x86/xen/apic: Mark apic __ro_after_init Thomas Gleixner
2023-07-18 15:31   ` Juergen Gross
2023-07-18 15:56     ` Thomas Gleixner
2023-07-18 16:08   ` Juergen Gross
2023-07-17 23:16 ` [patch 55/58] x86/apic: Mark all hotpath APIC callback wrappers __always_inline Thomas Gleixner
2023-07-17 23:16 ` [patch 56/58] x86/apic: Wrap IPI calls into helper functions Thomas Gleixner
2023-07-17 23:16 ` [patch 57/58] x86/apic: Provide static call infrastructure for APIC callbacks Thomas Gleixner
2023-07-18  9:19   ` Peter Zijlstra
2023-07-17 23:16 ` [patch 58/58] x86/apic: Turn on static calls Thomas Gleixner
2023-07-18 13:35   ` Thomas Gleixner
2023-07-18  0:27 ` [patch 00/58] x86/apic: Decrapification and " Linus Torvalds
2023-07-18  9:23 ` Peter Zijlstra
2023-07-18 14:29 ` Linus Walleij
2023-07-24 16:59   ` William Breathitt Gray
2023-07-20 12:43 ` Marc Gonzalez
2023-07-20 13:13   ` Peter Zijlstra
2023-07-20 13:47     ` Thomas Gleixner
2023-07-20 15:16     ` Marc Gonzalez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230717223049.327865981@linutronix.de \
    --to=tglx@linutronix.de \
    --cc=andrew.cooper3@citrix.com \
    --cc=arjan@linux.intel.com \
    --cc=jgross@suse.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=thomas.lendacky@amd.com \
    --cc=torvalds@linux-foundation.org \
    --cc=wei.liu@kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).