From: Haren Myneni <haren@linux.ibm.com>
To: mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com
Subject: [PATCH 00/10] powerpc/pseries/vas: NXGZIP support with DLPAR
Date: Mon, 29 Nov 2021 09:44:53 -0800 [thread overview]
Message-ID: <a2187018b4e030fe6c7e408b3a73c37c85472e10.camel@linux.ibm.com> (raw)
PowerPC provides HW compression with NX coprocessor. This feature
is available on both PowerNV and PowerVM and included in Linux.
Since each powerpc chip has one NX coprocessor, the VAS introduces
the concept of windows / credits to manage access to this hardware
resource. On powerVM, these limited resources should be available
across all LPARs. So the hypervisor assigns the specific credits
to each LPAR based on processor entitlement so that one LPAR does
not overload NX. The hypervisor can reject the window open request
to a partition if exceeds its credit limit (1 credit per window).
So the total number of target credits in a partition can be changed
if the core configuration is modified. The hypervisor expects the
partition to modify its window usage depends on new target
credits. For example, if the partition uses more credits than the
new target credits, it should close the excessive windows so that
the NX resource will be available to other partitions.
This patch series enables OS to support this dynamic credit
management with DLPAR core removal/add.
Core removal operation:
- Get new VAS capabilities from the hypervisor when the DLPAR
notifier is received. This capabilities provides the new target
credits based on new processor entitlement. In the case of QoS
credit changes, the notification will be issued by updating
the target_creds via sysfs.
- If the partition is already used more than the new target credits,
the kernel selects windows, unmap the current paste address and
close them in the hypervisor, It uses LIFO to identify these
windows - last windows that are opened are the first ones to be
closed.
- When the user space issue requests on these windows, NX generates
page fault on the unmap paste address. The kernel handles the
fault by returning the paste instruction failure if the window is
not active (means unmap paste). Then up to the library / user
space to fall back to SW compression or manage with the current
windows.
Core add operation:
- The kernel can see increased target credits from the new VAS
capabilities.
- Scans the window list for the closed windows in the hypervisor
due to lost credit before and selects windows based on same LIFO.
- Make these corresponding windows active and create remap with
the same VMA on the new paste address in the fault handler.
- Then the user space should expect paste successful later.
Patch 1: Define common names for sysfs target/used/avail_creds so
that same sysfs entries can be used even on PowerNV later.
Patch 2: Add VAS notifier for DLPAR core add / removal
Patch 3: Save LPID in the vas window struct during initial window
open and use it when reopen later.
Patch 4: When credits are available, reopen windows that are closed
before with core removal.
Patch 5: Close windows in the hypervisor when the partition exceeds
its usage than the new target credits.
Patch 6: If the window is closed in the hypervisor before the user
space issue the initial mmap(), return -EACCES failure.
Patch 7: Add new mmap fault handler which handles the page fault
from NX on paste address.
Patch 8: Return the paste instruction failure if the window is not
active.
Patch 9 & 10: The user space determines the credit usage with sysfs
target/avail/used_creds interfaces. drmgr uses target_creds
to notify OS for QoS credit changes.
Thanks to Nicholas Piggin and Aneesh Kumar for the valuable suggestions
on the NXGZIP design to support DLPAR operations.
Haren Myneni (10):
powerpc/pseries/vas: Use common names in VAS capability structure
powerpc/pseries/vas: Add notifier for DLPAR core removal/add
powerpc/pseries/vas: Save partition PID in pseries_vas_window struct
powerpc/pseries/vas: Reopen windows with DLPAR core add
powerpc/pseries/vas: Close windows with DLPAR core removal
powerpc/vas: Map paste address only if window is active
powerpc/vas: Add paste address mmap fault handler
powerpc/vas: Return paste instruction failure if window is not active
powerpc/pseries/vas: sysfs interface to export capabilities
powerpc/pseries/vas: Write 'target_creds' for QoS credits change
arch/powerpc/include/asm/ppc-opcode.h | 2 +
arch/powerpc/include/asm/vas.h | 16 ++
arch/powerpc/platforms/book3s/vas-api.c | 129 ++++++++-
arch/powerpc/platforms/pseries/Makefile | 2 +-
arch/powerpc/platforms/pseries/vas-sysfs.c | 246 +++++++++++++++++
arch/powerpc/platforms/pseries/vas.c | 293 ++++++++++++++++++++-
arch/powerpc/platforms/pseries/vas.h | 16 +-
7 files changed, 691 insertions(+), 13 deletions(-)
create mode 100644 arch/powerpc/platforms/pseries/vas-sysfs.c
--
2.27.0
next reply other threads:[~2021-11-29 17:45 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-29 17:44 Haren Myneni [this message]
2021-11-29 17:46 ` [PATCH 01/10] powerpc/pseries/vas: Use common names in VAS capability structure Haren Myneni
2021-11-29 17:47 ` [PATCH 02/10] powerpc/pseries/vas: Add notifier for DLPAR core removal/add Haren Myneni
2021-11-29 17:48 ` [PATCH 03/10] powerpc/pseries/vas: Save LPID in pseries_vas_window struct Haren Myneni
2021-11-29 17:49 ` [PATCH 04/10] powerpc/pseries/vas: Reopen windows with DLPAR core add Haren Myneni
2021-11-29 17:50 ` [PATCH 05/10] powerpc/pseries/vas: Close windows with DLPAR core removal Haren Myneni
2021-11-29 17:51 ` [PATCH 06/10] powerpc/vas: Map paste address only if window is active Haren Myneni
2021-11-29 17:52 ` [PATCH 07/10] powerpc/vas: Add paste address mmap fault handler Haren Myneni
2021-11-29 17:53 ` [PATCH 08/10] powerpc/vas: Return paste instruction failure if no active window Haren Myneni
2021-11-29 17:54 ` [PATCH 09/10] powerpc/pseries/vas: sysfs interface to export capabilities Haren Myneni
2021-11-29 17:55 ` [PATCH 10/10] powerpc/pseries/vas: Write 'target_creds' for QoS credits change Haren Myneni
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a2187018b4e030fe6c7e408b3a73c37c85472e10.camel@linux.ibm.com \
--to=haren@linux.ibm.com \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=npiggin@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).