linux-crypto.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Daniel Axtens <dja@axtens.net>
To: Haren Myneni <haren@linux.ibm.com>, herbert@gondor.apana.org.au
Cc: mikey@neuling.org, npiggin@gmail.com,
	linux-crypto@vger.kernel.org, sukadev@linux.vnet.ibm.com,
	linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v3 9/9] Documentation/powerpc: VAS API
Date: Fri, 20 Mar 2020 23:24:51 +1100	[thread overview]
Message-ID: <87blorw6wc.fsf@dja-thinkpad.axtens.net> (raw)
In-Reply-To: <1583541541.9256.50.camel@hbabu-laptop>

Hi Haren,

This is good documentation.

> Power9 introduced Virtual Accelerator Switchboard (VAS) which allows
> userspace to communicate with Nest Accelerator (NX) directly. But
> kernel has to establish channel to NX for userspace. This document
> describes user space API that application can use to establish
> communication channel.
>
> Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.ibm.com>
> Signed-off-by: Haren Myneni <haren@linux.ibm.com>
> ---
>  Documentation/powerpc/index.rst   |   1 +
>  Documentation/powerpc/vas-api.rst | 246 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 247 insertions(+)
>  create mode 100644 Documentation/powerpc/vas-api.rst
>
> diff --git a/Documentation/powerpc/index.rst b/Documentation/powerpc/index.rst
> index 0d45f0f..afe2d5e 100644
> --- a/Documentation/powerpc/index.rst
> +++ b/Documentation/powerpc/index.rst
> @@ -30,6 +30,7 @@ powerpc
>      syscall64-abi
>      transactional_memory
>      ultravisor
> +    vas-api
>  
>  .. only::  subproject and html
>  
> diff --git a/Documentation/powerpc/vas-api.rst b/Documentation/powerpc/vas-api.rst
> new file mode 100644
> index 0000000..13ce4e7
> --- /dev/null
> +++ b/Documentation/powerpc/vas-api.rst
> @@ -0,0 +1,246 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. _VAS-API:
> +
> +===================================================
> +Virtual Accelerator Switchboard (VAS) userspace API
> +===================================================
> +
> +Introduction
> +============
> +
> +Power9 processor introduced Virtual Accelerator Switchboard (VAS) which
> +allows both userspace and kernel communicate to co-processor
> +(hardware accelerator) referred to as the Nest Accelerator (NX). The NX
> +unit comprises of one or more hardware engines or co-processor types
> +such as 842 compression, GZIP compression and encryption. On power9,
> +userspace applications will have access to only GZIP Compression engine
> +which supports ZLIB and GZIP compression algorithms in the hardware.
> +
> +To communicate with NX, kernel has to establish a channel or window and
> +then requests can be submitted directly without kernel involvement.
> +Requests to the GZIP engine must be formatted as a co-processor Request
> +Block (CRB) and these CRBs must be submitted to the NX using COPY/PASTE
> +instructions to paste the CRB to hardware address that is associated with
> +the engine's request queue.
> +
> +The GZIP engine provides two priority levels of requests: Normal and
> +High. Only Normal requests are supported from userspace right now.
> +
> +This document explains userspace API that is used to interact with
> +kernel to setup channel / window which can be used to send compression
> +requests directly to NX accelerator.
> +
> +
> +Overview
> +========
> +
> +Application access to the GZIP engine is provided through
> +/dev/crypto/nx-gzip device node implemented by the VAS/NX device driver.
> +An application must open the /dev/crypto/nx-gzip device to obtain a file
> +descriptor (fd). Then should issue VAS_TX_WIN_OPEN ioctl with this fd to
> +establish connection to the engine. It means send window is opened on GZIP
> +engine for this process. Once a connection is established, the application
> +should use the mmap() system call to map the hardware address of engine's
> +request queue into the application's virtual address space.
> +
> +The application can then submit one or more requests to the the engine by
> +using copy/paste instructions and pasting the CRBs to the virtual address
> +(aka paste_address) returned by mmap(). User space can close the
> +established connection or send window by closing the file descriptior
> +(close(fd)) or upon the process exit.
> +
> +Note that applications can send several requests with the same window or
> +can establish multiple windows, but one window for each file descriptor.
> +
> +Following sections provide additional details and references about the
> +individual steps.
> +
> +NX-GZIP Device Node
> +===================
> +
> +There is one /dev/crypto/nx-gzip node in the system and it provides
> +access to all GZIP engines in the system. The only valid operations on
> +/dev/crypto/nx-gzip are:
> +
> +	* open() the device for read and write.
> +	* issue VAS_TX_WIN_OPEN ioctl
> +	* mmap() the engine's request queue into application's virtual
> +	  address space (i.e. get a paste_address for the co-processor
> +	  engine).
> +	* close the device node.
> +
> +Other file operations on this device node are undefined.
> +
> +Note that the copy and paste operations go directly to the hardware and
> +do not go through this device. Refer COPY/PASTE document for more
> +details.
> +
> +Although a system may have several instances of the NX co-processor
> +engines (typically, one per P9 chip) there is just one
> +/dev/crypto/nx-gzip device node in the system. When the nx-gzip device
> +node is opened, Kernel opens send window on a suitable instance of NX
> +accelerator. It finds CPU on which the user process is executing and
> +determine the NX instance for the corresponding chip on which this CPU
> +belongs.
> +
> +Applications may chose a specific instance of the NX co-processor using
> +the vas_id field in the VAS_TX_WIN_OPEN ioctl as detailed below.
> +
> +A userspace library libnxz is available here but still in development:
> +	 https://github.com/abalib/power-gzip
> +
> +Applications that use inflate / deflate calls can link with libnxz
> +instead of libz and use NX GZIP compression without any modification.
> +
> +Open /dev/crypto/nx-gzip
> +========================
> +
> +The nx-gzip device should be opened for read and write. No special
> +privileges are needed to open the device. Each window coreesponds to one

s/coreesponds/corresponds/

> +file descriptor. So if the userspace process needs multiple windows,
> +several open calls have to be issued.
> +
> +See open(2) system call man pages for other details such as return values,
> +error codes and restrictions.
> +codes and restrictions.

You have 'codes and restrictions' twice here.

> +
> +VAS_TX_WIN_OPEN ioctl
> +=====================
> +
> +Applications should use the VAS_TX_WIN_OPEN ioctl as follows to establish
> +a connection with NX co-processor engine:
> +
> +	::
> +		struct vas_tx_win_open_attr {
> +			__u32   version;
> +			__s16   vas_id; /* specific instance of vas or -1
> +						for default */
> +			__u16   reserved1;
> +			__u64   flags;	/* For future use */
> +			__u64   reserved2[6];
> +		};
> +
> +	version: The version field must be currently set to 1.
> +	vas_id: If '-1' is passed, kernel will make a best-effort attempt
> +		to assign an optimal instance of NX for the process. To
> +		select the specific VAS instance, refer
> +		"Discovery of available VAS engines" section below.
> +
> +	flags, reserved1 and reserved2[6] fields are for future extension
> +	and must be set to 0.
> +
> +	The attributes attr for the VAS_TX_WIN_OPEN ioctl are defined as
> +	follows:
> +		#define VAS_MAGIC 'v'
> +		#define VAS_TX_WIN_OPEN _IOW(VAS_MAGIC, 1,
> +						struct vas_tx_win_open_attr)
> +
> +		struct vas_tx_win_open_attr attr;
> +		rc = ioctl(fd, VAS_TX_WIN_OPEN, &attr);
> +
> +	The VAS_TX_WIN_OPEN ioctl returns 0 on success. On errors, it
> +	returns -1 and sets the errno variable to indicate the error.
> +
> +	Error conditions:
> +		EINVAL	fd does not refer to a valid VAS device.
> +		EINVAL	Invalid vas ID
> +		EINVAL	version is not set with proper value
> +		EEXIST	Window is already opened for the given fd
> +		ENOMEM	Memory is not available to allocate window
> +		ENOSPC	System has too many active windows (connections)
> +			opened
> +		EINVAL	reserved fields are not set to 0.
> +
> +	See the ioctl(2) man page for more details, error codes and
> +	restrictions.
> +
> +mmap() NX-GZIP device
> +=====================
> +
> +The mmap() system call for a NX-GZIP device fd returns a paste_address
> +that the application can use to copy/paste its CRB to the hardware engines.
> +	::
> +
> +		paste_addr = mmap(addr, size, prot, flags, fd, offset);
> +
> +	Only restrictions on mmap for a NX-GZIP device fd are:
> +		* size should be 4K page size

Patch 3 seems to allow a 64k page if the system is compiled for 64k
pages... Should it restrict it to 4K?

> +		* offset parameter should be 0ULL
> +
> +	Refer to mmap(2) man page for additional details/restrictions.
> +	In addition to the error conditions listed on the mmap(2) man
> +	page, can also fail with one of the following error codes:
> +
> +		EINVAL	fd is not associated with an open window
> +			(i.e mmap() does not follow a successful call
> +			to the VAS_TX_WIN_OPEN ioctl).
> +		EINVAL	offset field is not 0ULL.
> +
> +Discovery of available VAS engines
> +==================================
> +
> +Each available VAS instance in the system will have a device tree node
> +like /proc/device-tree/vas@* or /proc/device-tree/xscom@*/vas@*.
> +Determine the chip or VAS instance and use the corresponding ibm,vas-id
> +property value in this node to select specific VAS instance.
> +
> +Copy/Paste operations
> +=====================
> +
> +Applications should use the copy and paste instructions defined in the RFC
> +to copy/paste the CRB.

In which RFC?

> +
> +CRB Specification and use NX
> +============================
> +
> +Applications should format requests to the co-processor using the
> +co-processor Request Block (CRBs). Refer NX workbook for the format of
> +CRB and use NX from userspace such as sending requests and checking
> +request status.

Where would someone find the NX workbook?

Regards,
Daniel

> +
> +Simple example
> +==============
> +
> +	::
> +		int use_nx_gzip()
> +		{
> +			int rc, fd;
> +			void *addr;
> +			struct vas_setup_attr txattr;
> +
> +			fd = open("/dev/crypto/nx-gzip", O_RDWR);
> +			if (fd < 0) {
> +				fprintf(stderr, "open nx-gzip failed\n");
> +				return -1;
> +			}
> +			memset(&txattr, 0, sizeof(txattr));
> +			txattr.version = 1;
> +			txattr.vas_id = -1
> +			rc = ioctl(fd, VAS_TX_WIN_OPEN,
> +					(unsigned long)&txattr);
> +			if (rc < 0) {
> +				fprintf(stderr, "ioctl() n %d, error %d\n",
> +						rc, errno);
> +				return rc;
> +			}
> +			addr = mmap(NULL, 4096, PROT_READ|PROT_WRITE,
> +					MAP_SHARED, fd, 0ULL);
> +			if (addr == MAP_FAILED) {
> +				fprintf(stderr, "mmap() failed, errno %d\n",
> +						errno);
> +				return -errno;
> +			}
> +			do {
> +				//Format CRB request with compression or
> +				//uncompression
> +				// Refer tests for vas_copy/vas_paste
> +				vas_copy((&crb, 0, 1);
> +				vas_paste(addr, 0, 1);
> +				// Poll on csb.flags with timeout
> +				// csb address is listed in CRB
> +			} while (true)
> +			close(fd) or window can be closed upon process exit
> +		}
> +
> +	Refer https://github.com/abalib/power-gzip for tests or more
> +	use cases.
> -- 
> 1.8.3.1

  reply	other threads:[~2020-03-20 12:24 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-07  0:27 [PATCH v3 0/9] crypto/nx: Enable GZIP engine and provide userpace API Haren Myneni
2020-03-07  0:32 ` [PATCH v3 1/9] powerpc/vas: Initialize window attributes for GZIP coprocessor type Haren Myneni
2020-03-07  0:32 ` [PATCH v3 2/9] powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API Haren Myneni
2020-03-07  0:33 ` [PATCH v3 3/9] powerpc/vas: Add VAS user space API Haren Myneni
2020-03-18  7:12   ` Daniel Axtens
2020-03-19  1:16   ` Daniel Axtens
2020-03-19  1:53     ` Haren Myneni
2020-03-20 12:18   ` Daniel Axtens
2020-03-22 20:50     ` Haren Myneni
2020-03-07  0:34 ` [PATCH v3 4/9] crypto/nx: Initialize coproc entry with kzalloc Haren Myneni
2020-03-07  0:34 ` [PATCH v3 5/9] crypto/nx: Rename nx-842-powernv file name to nx-common-powernv Haren Myneni
2020-03-07  0:36 ` [PATCH v3 6/9] crypto/NX: Make enable code generic to add new GZIP compression type Haren Myneni
2020-03-07  0:37 ` [PATCH v3 7/9] crypto/nx: Enable and setup GZIP compresstion type Haren Myneni
2020-03-07  0:38 ` [PATCH v3 8/9] crypto/nx: Remove 'pid' in vas_tx_win_attr struct Haren Myneni
2020-03-07  0:39 ` [PATCH v3 9/9] Documentation/powerpc: VAS API Haren Myneni
2020-03-20 12:24   ` Daniel Axtens [this message]
2020-03-23  0:54     ` Haren Myneni
2020-03-23  8:32   ` Nicholas Piggin
2020-03-16 13:04 ` [PATCH v3 0/9] crypto/nx: Enable GZIP engine and provide userpace API Daniel Axtens
2020-03-16 23:31   ` Haren Myneni

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87blorw6wc.fsf@dja-thinkpad.axtens.net \
    --to=dja@axtens.net \
    --cc=haren@linux.ibm.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=mikey@neuling.org \
    --cc=npiggin@gmail.com \
    --cc=sukadev@linux.vnet.ibm.com \
    --subject='Re: [PATCH v3 9/9] Documentation/powerpc: VAS API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).