Re: [PATCH 00/16] Enable VAS and NX-GZIP support on powerVM

[Christophe Leroy <christophe.leroy@csgroup.eu>]

Le 11/04/2021 à 02:27, Haren Myneni a écrit :
> 
> This patch series enables VAS / NX-GZIP on powerVM which allows
> the user space to do copy/paste with the same existing interface
> that is available on powerNV.

Can you explain (here and in patch 1 at least) what VAS and NX means ?
Is that Vector Addition System ? Is that Virtual Address Space ?
(https://en.wikipedia.org/wiki/VAS)

> 
> VAS Enablement:
> - Get all VAS capabilities using H_QUERY_VAS_CAPABILITIES that are
>    available in the hypervisor. These capabilities tells OS which
>    type of features (credit types such as Default and Quality of
>    Service (QoS)). Also gives specific capabilities for each credit
>    type: Maximum window credits, Maximum LPAR credits, Target credits
>    in that parition (varies from max LPAR credits based DLPAR
>    operation), whether supports user mode COPY/PASTE and etc.
> - Register LPAR VAS operations such as open window. get paste
>    address and close window with the current VAS user space API.
> - Open window operation - Use H_ALLOCATE_VAS_WINDOW HCALL to open
>    window and H_MODIFY_VAS_WINDOW HCALL to setup the window with LPAR
>    PID and etc.
> - mmap to paste address returned in H_ALLOCATE_VAS_WINDOW HCALL
> - To close window, H_DEALLOCATE_VAS_WINDOW HCALL is used to close in
>    the hypervisor.
> 
> NX Enablement:
> - Get NX capabilities from the the hypervisor which provides Maximum
>    buffer length in a single GZIP request, recommended minimum
>    compression / decompression lengths.
> - Register to VAS to enable user space VAS API
> 
> Main feature differences with powerNV implementation:
> - Each VAS window will be configured with a number of credits which
>    means that many requests can be issues simultaniously on that
>    window. On powerNV, 1K credits are configured per window.
>    Whereas on powerVM, the hypervisor allows 1 credit per window
>    at present.
> - The hypervisor introduced 2 different types of credits: Default -
>    Uses normal priority FIFO and Quality of Service (QoS) - Uses high
>    priority FIFO. On powerVM, VAS/NX HW resources are shared across
>    LPARs. The total number of credits available on a system depends
>    on cores configured. We may see more credits are assigned across
>    the system than the NX HW resources can handle. So to avoid NX HW
>    contention, pHyp introduced QoS credits which can be configured
>    by system administration with HMC API. Then the total number of
>    available default credits on LPAR varies based on QoS credits
>    configured.
> - On powerNV, windows are allocated on a specific VAS instance
>    and the user space can select VAS instance with the open window
>    ioctl. Since VAS instances can be shared across partitions on
>    powerVM, the hypervisor manages window allocations on different
>    VAS instances. So H_ALLOCATE_VAS_WINDOW allows to select by domain
>    indentifiers (H_HOME_NODE_ASSOCIATIVITY values by cpu). By default
>    the hypervisor selects VAS instance closer to CPU resources that the
>    parition uses. So vas_id in ioctl interface is ignored on powerVM
>    except vas_id=-1 which is used to allocate window based on CPU that
>    the process is executing. This option is needed for process affinity
>    to NUMA node.
> 
>    The existing applications that linked with libnxz should work as
>    long as the job request length is restricted to
>    req_max_processed_len.
> 
>    Tested the following patches on P10 successfully with test cases
>    given: https://github.com/libnxz/power-gzip
> 
>    Note: The hypervisor supports user mode NX from p10 onwards. Linux
> 	supports user mode VAS/NX on P10 only with radix page tables.
> 
> Patches 1- 4:   Make the code that is needed for both powerNV and
>                  powerVM to powerpc platform independent.
> Patch5:         Modify vas-window struct to support both and the
>                  related changes.
> Patch 6:        Define HCALL and the related VAS/NXGZIP specific
>                  structs.
> Patch 7:        Define QoS credit flag in window open ioctl
> Patch 8:        Implement Allocate, Modify and Deallocate HCALLs
> Patch 9:        Retrieve VAS capabilities from the hypervisor
> Patch 10;       Implement window operations and integrate with API
> Patch 11:       Setup IRQ and NX fault handling
> Patch 12;       Add sysfs interface to expose VAS capabilities
> Patch 13 - 14:  Make the code common to add NX-GZIP enablement
> Patch 15:       Get NX capabilities from the hypervisor
> patch 16;       Add sysfs interface to expose NX capabilities
> 
> Haren Myneni (16):
>    powerpc/powernv/vas: Rename register/unregister functions
>    powerpc/vas: Make VAS API powerpc platform independent
>    powerpc/vas: Create take/drop task reference functions
>    powerpc/vas: Move update_csb/dump_crb to platform independent
>    powerpc/vas:  Define and use common vas_window struct
>    powerpc/pseries/vas: Define VAS/NXGZIP HCALLs and structs
>    powerpc/vas: Define QoS credit flag to allocate window
>    powerpc/pseries/vas: Implement allocate/modify/deallocate HCALLS
>    powerpc/pseries/vas: Implement to get all capabilities
>    powerpc/pseries/vas: Integrate API with open/close windows
>    powerpc/pseries/vas: Setup IRQ and fault handling
>    powerpc/pseries/vas: sysfs interface to export capabilities
>    crypto/nx: Rename nx-842-pseries file name to nx-common-pseries
>    crypto/nx: Register and unregister VAS interface
>    crypto/nx: Get NX capabilities for GZIP coprocessor type
>    crypto/nx: sysfs interface to export NX capabilities
> 
>   arch/powerpc/Kconfig                          |  15 +
>   arch/powerpc/include/asm/hvcall.h             |   7 +
>   arch/powerpc/include/asm/vas.h                | 122 +++-
>   arch/powerpc/include/uapi/asm/vas-api.h       |   6 +-
>   arch/powerpc/kernel/Makefile                  |   1 +
>   arch/powerpc/kernel/vas-api.c                 | 485 +++++++++++++
>   arch/powerpc/platforms/powernv/Kconfig        |  14 -
>   arch/powerpc/platforms/powernv/Makefile       |   2 +-
>   arch/powerpc/platforms/powernv/vas-api.c      | 278 --------
>   arch/powerpc/platforms/powernv/vas-debug.c    |  12 +-
>   arch/powerpc/platforms/powernv/vas-fault.c    | 155 +---
>   arch/powerpc/platforms/powernv/vas-trace.h    |   6 +-
>   arch/powerpc/platforms/powernv/vas-window.c   | 250 ++++---
>   arch/powerpc/platforms/powernv/vas.h          |  42 +-
>   arch/powerpc/platforms/pseries/Makefile       |   1 +
>   arch/powerpc/platforms/pseries/vas-sysfs.c    | 173 +++++
>   arch/powerpc/platforms/pseries/vas.c          | 674 ++++++++++++++++++
>   arch/powerpc/platforms/pseries/vas.h          |  98 +++
>   drivers/crypto/nx/Makefile                    |   2 +-
>   drivers/crypto/nx/nx-common-powernv.c         |   6 +-
>   .../{nx-842-pseries.c => nx-common-pseries.c} | 135 ++++
>   21 files changed, 1885 insertions(+), 599 deletions(-)
>   create mode 100644 arch/powerpc/kernel/vas-api.c
>   delete mode 100644 arch/powerpc/platforms/powernv/vas-api.c
>   create mode 100644 arch/powerpc/platforms/pseries/vas-sysfs.c
>   create mode 100644 arch/powerpc/platforms/pseries/vas.c
>   create mode 100644 arch/powerpc/platforms/pseries/vas.h
>   rename drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} (90%)
>