From: Sourabh Jain <sourabhjain@linux.ibm.com>
To: linuxppc-dev@ozlabs.org
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>,
Sourabh Jain <sourabhjain@linux.ibm.com>,
Naveen N Rao <naveen@kernel.org>,
"Aneesh Kumar K . V" <aneesh.kumar@kernel.org>,
Aditya Gupta <adityag@linux.ibm.com>,
Hari Bathini <hbathini@linux.ibm.com>
Subject: [PATCH v8 0/3] powerpc: make fadump resilient with memory add/remove events
Date: Sat, 17 Feb 2024 12:50:01 +0530 [thread overview]
Message-ID: <20240217072004.148293-1-sourabhjain@linux.ibm.com> (raw)
Problem:
========
Due to changes in memory resources caused by either memory hotplug or
online/offline events, the elfcorehdr, which describes the cpus and
memory of the crashed kernel to the kernel that collects the dump (known
as second/fadump kernel), becomes outdated. Consequently, attempting
dump collection with an outdated elfcorehdr can lead to failed or
inaccurate dump collection.
Memory hotplug or online/offline events is referred as memory add/remove
events in reset of the patch series.
Existing solution:
==================
Monitor memory add/remove events in userspace using udev rules, and
re-register fadump whenever there are changes in memory resources. This
leads to the creation of a new elfcorehdr with updated system memory
information.
Challenges with existing solution:
==================================
1. Performing bulk memory add/remove with udev-based fadump
re-registration can lead to race conditions and, more importantly,
it creates a large wide window during which fadump is inactive until
all memory add/remove events are settled.
2. Re-registering fadump for every memory add/remove event is
inefficient.
3. Memory for elfcorehdr is allocated based on the memblock regions
available during first kernel early boot and it remains fixed
thereafter. However, if the elfcorehdr is later recreated with
additional memblock regions, its size will increase, potentially
leading to memory corruption.
Proposed solution:
==================
Address the aforementioned challenges by shifting the creation of
elfcorehdr from the first kernel (also referred as the crashed kernel),
where it was created and frequently recreated for every memory
add/remove event, to the fadump kernel. As a result, the elfcorehdr only
needs to be created once, thus eliminating the necessity to re-register
fadump during memory add/remove events.
To know more about elfcorehdr creation in the fadump kernel, refer to
the first patch in this series.
The second patch includes a new sysfs interface that tells userspace
that fadump re-registration isn't needed for memory add/remove events.
note that userspace changes do not need to be in sync with kernel
changes; they can roll out independently.
Since there are significant changes in the fadump implementation, the
third patch updates the fadump documentation to reflect the changes made
in this patch series.
Kernel tree rebased on 6.8.0-rc4 with patch series applied:
=========================================================
https://github.com/sourabhjains/linux/tree/fadump-mem-hotplug-v8
Userspace changes:
==================
To realize this feature, one must update the kdump udev rules to prevent
fadump re-registration during memory add/remove events.
On rhel apply the following changes to file
/usr/lib/udev/rules.d/98-kexec.rules
-run+="/bin/sh -c '/usr/bin/systemctl is-active kdump.service || exit 0; /usr/bin/systemd-run --quiet --no-block /usr/lib/udev/kdump-udev-throttler'"
+# don't re-register fadump if the value of the node
+# /sys/kernel/fadump/hotplug_ready is 1.
+
+run+="/bin/sh -c '/usr/bin/systemctl is-active kdump.service || exit 0; ! test -f /sys/kernel/fadump_enabled || cat /sys/kernel/fadump_enabled | grep 0 || ! test -f /sys/kernel/fadump/hotplug_ready || cat /sys/kernel/fadump/hotplug_ready | grep 0 || exit 0; /usr/bin/systemd-run --quiet --no-block /usr/lib/udev/kdump-udev-throttler'"
Changelog:
==========
v8: 16 Feb 2023
- Move `elfcorehdr_addr` and `elfcorehdr_size` struct attributes
from `struct fadump_crash_info_header` to `struct fw_dump`.
- Make minor changes in commit message 1/3.
- Rebase it to 6.8-rc4.
v7: 11 Jan 2024
https://lore.kernel.org/all/20240111140943.297501-1-sourabhjain@linux.ibm.com/
- Rebase it to 6.7
v6: 8 Dec 2023
https://lore.kernel.org/all/20231208115159.82236-1-sourabhjain@linux.ibm.com/
- Add size fields for `pt_regs` and `cpumask` in the fadump header
structure
- Don't process the dump if the size of `pt_regs` and `cpu_mask` is
not same in the crashed and fadump kernel
- Include an additional check for endianness mismatch when the magic
number doesn't match, to print the relevant error message
- Don't process the dump if the fadump header contains an old magic number
- Rebased it to 6.7.0-rc4
v5: 29 Oct 2023
https://lore.kernel.org/all/20231029124548.12198-1-sourabhjain@linux.ibm.com/
- Fix a comment on the first patch
v4: 21 Oct 2023
https://lore.kernel.org/all/20231021181733.204311-1-sourabhjain@linux.ibm.com/
- Fix a build warning about type casting
v3: 9 Oct 2023
https://lore.kernel.org/all/20231009041953.36139-1-sourabhjain@linux.ibm.com/
- Assign physical address of elfcorehdr to fdh->elfcorehdr_addr
- Rename a variable, boot_mem_dest_addr -> boot_mem_dest_offset
v2: 25 Sep 2023
https://lore.kernel.org/all/20230925051214.678957-1-sourabhjain@linux.ibm.com/
- Fixed a few indentation issues reported by the checkpatch script.
- Rebased it to 6.6.0-rc3
v1: 17 Sep 2023
https://lore.kernel.org/all/20230917080225.561627-1-sourabhjain@linux.ibm.com/
Cc: Aditya Gupta <adityag@linux.ibm.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Naveen N Rao <naveen@kernel.org>
Sourabh Jain (3):
powerpc: make fadump resilient with memory add/remove events
powerpc/fadump: add hotplug_ready sysfs interface
Documentation/powerpc: update fadump implementation details
Documentation/ABI/testing/sysfs-kernel-fadump | 11 +
.../arch/powerpc/firmware-assisted-dump.rst | 91 +++--
arch/powerpc/include/asm/fadump-internal.h | 31 +-
arch/powerpc/kernel/fadump.c | 353 +++++++++++-------
arch/powerpc/platforms/powernv/opal-fadump.c | 22 +-
arch/powerpc/platforms/pseries/rtas-fadump.c | 30 +-
6 files changed, 299 insertions(+), 239 deletions(-)
--
2.43.0
next reply other threads:[~2024-02-17 7:21 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-17 7:20 Sourabh Jain [this message]
2024-02-17 7:20 ` [PATCH v8 1/3] powerpc: make fadump resilient with memory add/remove events Sourabh Jain
2024-02-29 5:10 ` Sourabh Jain
2024-03-11 8:38 ` Hari Bathini
2024-03-12 4:41 ` Sourabh Jain
2024-02-17 7:20 ` [PATCH v8 2/3] powerpc/fadump: add hotplug_ready sysfs interface Sourabh Jain
2024-02-17 7:20 ` [PATCH v8 3/3] Documentation/powerpc: update fadump implementation details Sourabh Jain
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240217072004.148293-1-sourabhjain@linux.ibm.com \
--to=sourabhjain@linux.ibm.com \
--cc=adityag@linux.ibm.com \
--cc=aneesh.kumar@kernel.org \
--cc=hbathini@linux.ibm.com \
--cc=linuxppc-dev@ozlabs.org \
--cc=mahesh@linux.ibm.com \
--cc=naveen@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).