* [PATCH v2 01/15] misc: nnpi: Document NNP-I's driver overview
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 02/15] misc: nnpi: Initialize NNP-I framework and PCIe modules Guy Zadicario
` (14 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Introduce overview documentation for NNP-I card and driver
to let new readers of the driver understand better the
driver and the NNP-I device.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
Documentation/misc-devices/index.rst | 1 +
Documentation/misc-devices/intel-nnpi.rst | 237 ++++++++++++++++++++++++++++++
2 files changed, 238 insertions(+)
create mode 100644 Documentation/misc-devices/intel-nnpi.rst
diff --git a/Documentation/misc-devices/index.rst b/Documentation/misc-devices/index.rst
index 30ac58f..7f14fc4 100644
--- a/Documentation/misc-devices/index.rst
+++ b/Documentation/misc-devices/index.rst
@@ -22,6 +22,7 @@ fit into other categories.
dw-xdata-pcie
ibmvmc
ics932s401
+ intel-nnpi
isl29003
lis3lv02d
max6875
diff --git a/Documentation/misc-devices/intel-nnpi.rst b/Documentation/misc-devices/intel-nnpi.rst
new file mode 100644
index 0000000..cb67fb4
--- /dev/null
+++ b/Documentation/misc-devices/intel-nnpi.rst
@@ -0,0 +1,237 @@
+.. SPDX-License-Identifier: GPL-2.0-ONLY
+
+======================================================================
+Device driver for Intel NNP-I (Neural Network Processor for Inference)
+======================================================================
+
+Terminology
+===========
+To avoid confusion and for better understanding of the code for new reviewers,
+here is a list of few definitions used throughout this document, in commit
+messages and in the code:
+
+card:
+ The NNP-I card, including hardware and embedded software running inside it,
+ which is connected to the host through PCIe.
+
+host:
+ The computer the NNP-I card is attached to through PCIe, and which runs the
+ Linux kernel with NNP-I driver.
+
+IPC protocol:
+ Protocol of messages exchanged between the host and card.
+
+command:
+ A message sent from host to card, typically a command to execute on the
+ card.
+
+response:
+ A message sent from card to the host, typically as a response to a command
+ sent previously from the host. or, a message sent from card to indicate some
+ event, such as an error condition, or the completion of an asynchronous request.
+
+nnp_user:
+ An object created for each file descriptor opened for /dev/nnpi_host. It
+ manages host memory resources. Each nnp_user has its own set of host resources which
+ cannot be shared with other nnp_user objects.
+
+channel, cmd_chan:
+ An object which manages communication between user-space applications and
+ cards. It provides an interface to send commands to the card and receive responses
+ from the card. Each application can create one or more channels to the same
+ or different cards. One channel connects a single nnp_user to a single card.
+ Each channel is associated with one NNP-I card and one nnp_user.
+ The channel can only access host memory resources belonging to that
+ nnp_user.
+
+ELBI:
+ This acronym refers to the set of hardware registers of the NNP-I card which
+ are accessible through BAR0. It stands for: "External Local Bus Interface".
+
+model:
+ A neural network used to process input data and generate output data
+
+Description
+===========
+NNP-I is a PCIe card which accelerates deep-learning inference
+applications. The card is equipped with LPDDR4 memory, a DMA engine, x86 cores
+and specialized compute units called Inference Compute Engines (ICE). These
+compute engines are designed for power-efficient inference-related computations.
+
+The card DRAM as well as the ICEs and other hardware components on the card
+are not accessible from host CPU; they are programmed and controlled by
+software components running on the card's x86 cores. Communication between the
+SW components on the card and the host is done through three interfaces:
+
+- A small size "command queue" and "response queue", through which commands
+ and responses can be sent and received to/from the card.
+- Two doorbell registers, through which the host and card can communicate state.
+- A DMA engine on the card, which copies big memory chunks between host and card
+ DRAM.
+
+A typical flow of an inference application running on the host is the following:
+
+1) Allocate memory on host to hold the model, input and output data.
+2) Request card's software to allocate space on the card DRAM for the model,
+ input and output data.
+3) Load a model into host DRAM, and request the card's DMA engine to copy it
+ over to the card's DRAM.
+4) Load host memory with input data.
+5) Schedule commands to the card to copy the input data to the card
+ DRAM, execute the model on this input, and copy the output data back to
+ host memory.
+
+The card's software stack consists of a BIOS image which is flashed on
+the card and kept in a dedicated persistent memory, and a full embedded Linux image
+which is loaded to the card during boot/reset process.
+During power-on, the card BIOS and host driver communicate through
+the doorbell registers and the "command queue". The card boot flow consists of
+loading a "boot image" to host memory and communicating the location of this
+image to the card's BIOS, the card's BIOS copies that boot image to card
+DRAM using the DMA engine and starts booting the embedded Linux running on the
+card's x86 cores.
+
+Each NNP-I card can support multiple inference application contexts; each
+context has its own space for card and host resource IDs.
+There is no hardware level restriction on one context to access resources of
+another context, however this is prevented by the card software stack by having a
+separate resource ID space for each of the contexts.
+
+There may be multiple NNP-I cards connected to one host. An inference application can
+hold channels to multiple NNP-I cards and should be able to map and access
+the same host resource memory on all cards, however only if the host resource
+and all device contexts are created by the same application. Possible use cases
+for this capability is to run different models on different cards in parallel
+on the same data, or passing output of one model ran on one card as input to
+another model which is loaded on another card. The driver implements
+that requirement by exporting two char devices, one for host resource management
+and another for card access. The application must provide host resource file
+descriptor to the card's character device to allow access to host resources created from
+the same file descriptor.
+
+ABI
+===
+There are two character device classes created by the driver with IOCTL
+interface, provided by include/uapi/misc/intel_nnpi.h:
+
+/dev/nnpi_host:
+
+ A character device which is not related to physical NNP-I card.
+ It has 4 IOCTLs for creating, destroying,
+ locking and unlocking host resources. "host resource" is a
+ set of pinned memory pages on host which can be mapped
+ to PCI space and accessed by the card's DMA engine.
+ This character device is created on the first probed NNP-I card
+ so it will not be present on systems with no NNP-I cards.
+
+/dev/nnpi%d:
+
+ A character device with instance for each NNP-I card.
+ It support 5 IOCTLs for:
+
+ * Creating a channel - A "channel" gives user-space the ability to
+ send commands and receive responses from the card.
+ For each channel an anonymous file descriptor is created and
+ returned to the user. Commands and responses to the card
+ are sent and received using write and read operations on
+ the channel file descriptor. The driver validates each
+ command sent and will reject unsupported or invalid commands.
+ Commands written to a channel are added to a queue; each
+ channel has its own command queue. For each card in the system
+ there's one kernel thread (msg_scheduler) which drains the
+ command queues to this card hardware command queue.
+ The channel is destroyed by closing the returned channel file
+ descriptor.
+ When creating a channel an open file descriptor for
+ /dev/nnpi_host needs to be provided. The channel object holds
+ a reference to that file, and the channel can map/unmap only
+ host resources which were created through that same file
+ descriptor.
+ Each channel has a unique 10-bit ID allocated by the driver.
+ Channel IDs in a range [0, 255] are used for inference
+ contexts.
+ Channel with ID greater than 255 is used for non-inference
+ related communication with the card (mainly maintenance,
+ stats query, etc).
+ * Map/Unmap host resource - Maps a host resource to card PCI
+ space and sends to the card a pointer to a page table of
+ the physical addresses of the resource pages.
+ Each map has a unique 16-bit ID. Commands sent to the card
+ can include such ID in order to reference a host resource.
+ The ID space for host resources is private for each channel.
+ * Create/Delete "ringbuffer" - This is exactly the same as
+ map/unmap host resource, but for special host resources,
+ called ring buffers, used to transfer data along with
+ some commands. There may be up to two host-to-card ring
+ buffers and two card-to-host ring buffers.
+
+sysfs
+=====
+There are multiple sysfs attributes for NNP-I card allowing to display
+card information and status, and some for control operations like enable/disable
+the card.
+
+Attributes are documented in `Documentation/ABI/testing/sysfs-driver-intel_nnpi`.
+
+PCI BARs
+========
+The card exposes two 64-bit BARs:
+
+BAR0-1:
+ 4KB including card registers to control the command and response hardware
+ queues (FIFOs), doorbell registers and control/interrupt status registers.
+ The offsets and bitfields of those registers are defined in
+ if_include/nnpi_elbi.h
+
+BAR2-3:
+ Card Memory region of 64MB. The host has read/write access to this region.
+ The first 16KB of this region hold card crash dump in case the card
+ software stack has crashed (Obviously, this is an erroneous state that should
+ never happen, but when it does the crash dump helps debugging).
+ The layout of this 16KB is defined in if_include/nnp_inbound_mem.h.
+ This region will be filled by the card on event of crash and can be
+ read by the host for debugging purposes. When a crash is detected on card,
+ the card will send an event response message to indicate that event.
+ The Rest of this memory region (64MB - 16KB) is used by peer-to-peer
+ applications to transfer data between two NNP-I cards. A single application
+ can use multiple cards, there are commands to the card which allows such
+ application to allocate device resources in this BAR space as well as commands
+ to issue DMA copy request to copy data from one card DRAM memory to a second
+ card memory if the destination resource address is within BAR 2-3 space.
+
+Card's command protocol
+=======================
+Commands to the card include 1, 2 or 3 64-bit values. The lower 6 bits in the
+command specify the command opcode. The opcode also defines the command size as
+each command has constant size. Commands which are targeted to a specific channel
+include the channel ID in bits 15:6 of the command and must use opcode value
+above or equal to 32.
+
+The definition of other bits is specific to each command.
+Responses from the card have the same format.
+
+The opcodes and structure of the command and responses are defined in
+drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
+
+Driver architecture
+===================
+Since the NNP-I card has its own software stack and the way to program it is
+mainly by sending commands and receive responses, the driver is split into
+two modules.
+
+The PCIe device driver module, intel_nnpi_pcie, is responsible for detecting
+the NNP-I card and provide functionality for sending it commands, receive
+responses and perform few control operations. This module does not understand
+the IPC protocol, it only provides the mechanism to send and receive raw data.
+This module does not also provide any user space interfaces. The intel_nnpi_pcie
+module is a pci driver for the Intel NNP-I card and glue it into the NNP-I
+framework module.
+
+The NNP-I framework module, intel_nnpi, is the "logical" module that provides the
+user space interfaces, understands the IPC protocol, packs and unpacks commands
+and responses and control the card by calling to interfaces implemented by the
+device driver module. The main structure for an NNP-I card device in the framework
+is &struct nnp_device.
+
+These two modules serve similar roles to the "transport" and "application" layers
+in the `standard TCP/IP terminology <https://www.guru99.com/tcp-ip-model.html#3>`_.
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 02/15] misc: nnpi: Initialize NNP-I framework and PCIe modules
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 01/15] misc: nnpi: Document NNP-I's driver overview Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 03/15] misc: nnpi: Manage and schedule messages to device Guy Zadicario
` (13 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Initialize the NNP-I framework module and expose PCIe device driver.
The NNP-I driver is divided into two modules, the framework module
and the PCIe driver module.
The PCIe driver probes the device and provides, to the framework module,
functions for sending and receiving raw commands and responses to/from
the device. It does not have semantic understanding of these commands
and responses, and transfers them as abstract binary data.
The framework module unpacks command and response messages, and uses the
PCIe driver module to send and receive them. It uses the PCIe driver module
as an abstract communication layer, and is unaware of the PCIe HW used to
communicate with the NNP-I card.
Using TCP/IP jargon, the PCIe driver module is the "transport" layer
and the main framework module is the "application" layer.
This logical seperation was done in order to ease maintenance and
validation, and make adding support for other "transport" layers easier.
The header files under the ipc_include directory provide HW register
definitions and protocol structures used to communicate with the NNP-I
device. These header files are used also by the NNP-I card's SW stack.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
MAINTAINERS | 5 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/intel-nnpi/Kconfig | 18 ++
drivers/misc/intel-nnpi/Makefile | 12 +
drivers/misc/intel-nnpi/device.c | 84 +++++
drivers/misc/intel-nnpi/device.h | 39 +++
.../misc/intel-nnpi/ipc_include/nnp_boot_defs.h | 71 ++++
drivers/misc/intel-nnpi/ipc_include/nnp_elbi.h | 91 ++++++
drivers/misc/intel-nnpi/nnp_pcie.c | 358 +++++++++++++++++++++
10 files changed, 680 insertions(+)
create mode 100644 drivers/misc/intel-nnpi/Kconfig
create mode 100644 drivers/misc/intel-nnpi/Makefile
create mode 100644 drivers/misc/intel-nnpi/device.c
create mode 100644 drivers/misc/intel-nnpi/device.h
create mode 100644 drivers/misc/intel-nnpi/ipc_include/nnp_boot_defs.h
create mode 100644 drivers/misc/intel-nnpi/ipc_include/nnp_elbi.h
create mode 100644 drivers/misc/intel-nnpi/nnp_pcie.c
diff --git a/MAINTAINERS b/MAINTAINERS
index bd7aff0c..ff0c3d7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9316,6 +9316,11 @@ S: Supported
W: https://01.org/linux-acpi
F: drivers/platform/x86/intel_menlow.c
+INTEL NNP-I PCI DRIVER
+M: Guy Zadicario <guy.zadicario@intel.com>
+S: Supported
+F: drivers/misc/intel-nnpi/
+
INTEL P-Unit IPC DRIVER
M: Zha Qipeng <qipeng.zha@intel.com>
L: platform-driver-x86@vger.kernel.org
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index f4fb5c5..4c67fac 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -462,6 +462,7 @@ source "drivers/misc/ocxl/Kconfig"
source "drivers/misc/bcm-vk/Kconfig"
source "drivers/misc/cardreader/Kconfig"
source "drivers/misc/habanalabs/Kconfig"
+source "drivers/misc/intel-nnpi/Kconfig"
source "drivers/misc/uacce/Kconfig"
source "drivers/misc/pvpanic/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index e92a56d..592ac64 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -54,6 +54,7 @@ obj-$(CONFIG_BCM_VK) += bcm-vk/
obj-y += cardreader/
obj-$(CONFIG_PVPANIC) += pvpanic/
obj-$(CONFIG_HABANA_AI) += habanalabs/
+obj-$(CONFIG_INTEL_NNPI) += intel-nnpi/
obj-$(CONFIG_UACCE) += uacce/
obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
diff --git a/drivers/misc/intel-nnpi/Kconfig b/drivers/misc/intel-nnpi/Kconfig
new file mode 100644
index 0000000..8534554
--- /dev/null
+++ b/drivers/misc/intel-nnpi/Kconfig
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+#
+# Copyright (C) 2019-2021 Intel Corporation
+#
+#
+
+config INTEL_NNPI
+ tristate "Intel(R) PCIe NNP-I (AI accelerator for inference) device driver"
+ depends on PCI
+ select DMA_SHARED_BUFFER
+ help
+ Device driver for Intel NNP-I PCIe accelerator card for AI inference.
+
+ If unsure, say N.
+
+ To compile this driver as a module, choose M here. Two modules will
+ be generated: intel_nnpi and intel_nnpi_pcie.
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
new file mode 100644
index 0000000..84b7528
--- /dev/null
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2019-2021 Intel Corporation
+#
+
+obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
+
+intel_nnpi-y := device.o
+
+intel_nnpi_pcie-y := nnp_pcie.o
+
+ccflags-y += -I$(srctree)/$(src)/ipc_include
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
new file mode 100644
index 0000000..3d80e95
--- /dev/null
+++ b/drivers/misc/intel-nnpi/device.c
@@ -0,0 +1,84 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/device.h>
+#include <linux/idr.h>
+#include <linux/module.h>
+
+#include "device.h"
+
+static DEFINE_IDA(dev_ida);
+
+/**
+ * nnpdev_init() - initialize NNP-I device structure.
+ * @nnpdev: device to be initialized
+ * @dev: device structure representing the card device
+ * @ops: NNP-I device driver operations
+ *
+ * This function is called by the device driver module when a new NNP-I device
+ * is created. The function initialize NNP-I framework's device structure.
+ * The device driver must call nnpdev_destroy before the underlying device is
+ * removed and before the driver module get unloaded.
+ * The device driver must also make sure that when nnpdev_destroy is called the
+ * device is quiesced. Meaning, the physical device does no longer throw events
+ * and no operations on the nnpdev will be requested.
+ *
+ * Return: zero on success, error value otherwise.
+ */
+int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
+ const struct nnp_device_ops *ops)
+{
+ int ret;
+
+ ret = ida_simple_get(&dev_ida, 0, NNP_MAX_DEVS, GFP_KERNEL);
+ if (ret < 0)
+ return ret;
+
+ nnpdev->id = ret;
+ /*
+ * It is fine to keep pointers to the underlying device and driver
+ * ops since driver must call nnpdev_destroy before the device is
+ * removed or module gets unloaded.
+ */
+ nnpdev->dev = dev;
+ nnpdev->ops = ops;
+
+ return 0;
+}
+EXPORT_SYMBOL(nnpdev_init);
+
+/**
+ * nnpdev_card_doorbell_value_changed() - card doorbell changed notification
+ * @nnpdev: The nnp device
+ * @doorbell_val: The new value of the doorbell register
+ *
+ * This function is called from the NNP-I device driver when the card's doorbell
+ * register is changed.
+ */
+void nnpdev_card_doorbell_value_changed(struct nnp_device *nnpdev,
+ u32 doorbell_val)
+{
+ dev_dbg(nnpdev->dev, "Got card doorbell value 0x%x\n", doorbell_val);
+}
+EXPORT_SYMBOL(nnpdev_card_doorbell_value_changed);
+
+/**
+ * nnpdev_destroy() - destroy nnp device object
+ * @nnpdev: The nnp device to be destroyed.
+ *
+ * This function must be called by the device driver module when NNP-I device
+ * is removed or the device driver get unloaded.
+ */
+void nnpdev_destroy(struct nnp_device *nnpdev)
+{
+ dev_dbg(nnpdev->dev, "Destroying NNP-I device\n");
+
+ ida_simple_remove(&dev_ida, nnpdev->id);
+}
+EXPORT_SYMBOL(nnpdev_destroy);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Intel(R) NNPI Framework");
+MODULE_AUTHOR("Intel Corporation");
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
new file mode 100644
index 0000000..4ff7aa9
--- /dev/null
+++ b/drivers/misc/intel-nnpi/device.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_DEVICE_H
+#define _NNPDRV_DEVICE_H
+
+#define NNP_MAX_DEVS 256
+
+/**
+ * struct nnp_device - structure for NNP-I device info
+ * @ops: device operations implemented by the underlying device driver
+ * @dev: pointer to struct device representing the NNP-I card.
+ * @id: NNP-I device number
+ */
+struct nnp_device {
+ const struct nnp_device_ops *ops;
+ struct device *dev;
+ int id;
+};
+
+/**
+ * struct nnp_device_ops - operations implemented by underlying device driver
+ * @cmdq_flush: empties the device command queue, discarding all queued
+ * commands.
+ */
+struct nnp_device_ops {
+ int (*cmdq_flush)(struct nnp_device *hw_dev);
+};
+
+/*
+ * Functions exported by the device framework module which are
+ * called by the lower layer NNP-I device driver module
+ */
+int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
+ const struct nnp_device_ops *ops);
+void nnpdev_destroy(struct nnp_device *nnpdev);
+void nnpdev_card_doorbell_value_changed(struct nnp_device *nnpdev,
+ u32 doorbell_val);
+#endif
diff --git a/drivers/misc/intel-nnpi/ipc_include/nnp_boot_defs.h b/drivers/misc/intel-nnpi/ipc_include/nnp_boot_defs.h
new file mode 100644
index 0000000..c26dc38
--- /dev/null
+++ b/drivers/misc/intel-nnpi/ipc_include/nnp_boot_defs.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNP_BOOT_DEFS_H
+#define _NNP_BOOT_DEFS_H
+
+#include <linux/bits.h>
+
+/*
+ * Value fields of card->host doorbell status register HOST_PCI_DOORBELL_VALUE
+ */
+#define NNP_CARD_BOOT_STATE_MASK GENMASK(7, 0)
+#define NNP_CARD_BIOS_UPDATE_COUNTER_MASK GENMASK(11, 8)
+#define NNP_CARD_ERROR_MASK GENMASK(15, 12)
+#define NNP_CARD_KEEP_ALIVE_MASK GENMASK(23, 20)
+
+/* Possible values for card boot state */
+/* BIOS has not yet initialized */
+#define NNP_CARD_BOOT_STATE_NOT_READY 0
+/* BIOS initilaized and waiting for os boot image over PCIe */
+#define NNP_CARD_BOOT_STATE_BIOS_READY 1
+/* recovery BIOS initilaized and waiting for capsule update over PCIe */
+#define NNP_CARD_BOOT_STATE_RECOVERY_BIOS_READY 2
+/* BIOS copied boot image successfully, os boot has started */
+#define NNP_CARD_BOOT_STATE_BOOT_STARTED 3
+/* card has booted and card driver has loaded */
+#define NNP_CARD_BOOT_STATE_DRV_READY 4
+/* card driver finished initialization and user space daemon has started */
+#define NNP_CARD_BOOT_STATE_CARD_READY 8
+/* BIOS copied data into the system info structure */
+#define NNP_CARD_BOOT_STATE_BIOS_SYSINFO_READY 10
+/* BIOS capsule update has started flashing the BIOS image */
+#define NNP_CARD_BOOT_STATE_BIOS_FLASH_STARTED 32
+
+/* Possible card error values */
+#define NNP_CARD_ERROR_HOST_ERROR 1
+#define NNP_CARD_ERROR_BOOT_PARAMS 2
+#define NNP_CARD_ERROR_IMAGE_COPY 3
+#define NNP_CARD_ERROR_CORRUPTED_IMAGE 4
+#define NNP_CARD_ERROR_NOT_CAPSULE 8
+#define NNP_CARD_ERROR_CAPSULE_FAILED 9
+/*
+ * Value fields of host->card doorbell status register PCI_HOST_DOORBELL_VALUE
+ */
+#define NNP_HOST_BOOT_STATE_MASK GENMASK(3, 0)
+#define NNP_HOST_ERROR_MASK GENMASK(7, 4)
+#define NNP_HOST_DRV_STATE_MASK GENMASK(11, 8)
+#define NNP_HOST_DRV_REQUEST_SELF_RESET_MASK BIT(16)
+#define NNP_HOST_KEEP_ALIVE_MASK GENMASK(23, 20)
+#define NNP_HOSY_P2P_POKE_MASK GENMASK(31, 24)
+
+/* Possible values for host boot state */
+/* boot/capsule image is not loaded yet to memory */
+#define NNP_HOST_BOOT_STATE_NOT_READY 0
+/* host driver is up and ready */
+#define NNP_HOST_BOOT_STATE_DRV_READY (BIT(3) | BIT(0))
+/* debug os image is loaded and ready in memory */
+#define NNP_HOST_BOOT_STATE_DEBUG_OS_IMAGE_READY (BIT(3) | BIT(1))
+
+/* Possible values for host error */
+#define NNP_HOST_ERROR_CANNOT_LOAD_IMAGE 1
+
+/* Possible values for host driver state */
+/* driver did not detected the device yet */
+#define NNP_HOST_DRV_STATE_NOT_READY 0
+/* driver initialized and ready */
+#define NNP_HOST_DRV_STATE_READY 1
+/* host/card protocol version mismatch */
+#define NNP_HOST_DRV_STATE_VERSION_ERROR 2
+
+#endif // of _NNP_BOOT_DEFS_H
diff --git a/drivers/misc/intel-nnpi/ipc_include/nnp_elbi.h b/drivers/misc/intel-nnpi/ipc_include/nnp_elbi.h
new file mode 100644
index 0000000..fd9ba2a
--- /dev/null
+++ b/drivers/misc/intel-nnpi/ipc_include/nnp_elbi.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNP_ELBI_H
+#define _NNP_ELBI_H
+
+#include <linux/bits.h>
+
+#define ELBI_BASE 0 /* offset of ELBI registers */
+#define ELBI_LINE_BDF (ELBI_BASE + 0x004)
+
+/*
+ * COMMAND FIFO registers
+ */
+#define ELBI_COMMAND_WRITE_WO_MSI_LOW (ELBI_BASE + 0x050)
+#define ELBI_COMMAND_WRITE_WO_MSI_HIGH (ELBI_BASE + 0x054)
+#define ELBI_COMMAND_WRITE_W_MSI_LOW (ELBI_BASE + 0x058)
+#define ELBI_COMMAND_WRITE_W_MSI_HIGH (ELBI_BASE + 0x05C)
+
+#define ELBI_COMMAND_FIFO_0_LOW (ELBI_BASE + 0x080)
+#define ELBI_COMMAND_FIFO_LOW(i) (ELBI_COMMAND_FIFO_0_LOW + (i) * 8 + 0)
+#define ELBI_COMMAND_FIFO_HIGH(i) (ELBI_COMMAND_FIFO_0_LOW + (i) * 8 + 4)
+#define ELBI_COMMAND_FIFO_DEPTH 16
+
+#define ELBI_COMMAND_IOSF_CONTROL (ELBI_BASE + 0x044)
+#define CMDQ_READ_PTR_MASK GENMASK(3, 0)
+#define CMDQ_WRITE_PTR_MASK GENMASK(12, 8)
+
+#define ELBI_COMMAND_PCI_CONTROL (ELBI_BASE + 0x048)
+#define ELBI_COMMAND_PCI_CONTROL_ALMOST_EMPTY_TH_MASK GENMASK(3, 0)
+#define ELBI_COMMAND_PCI_CONTROL_FLUSH_MASK BIT(8)
+
+/*
+ * RESPONSE FIFO registers
+ */
+#define ELBI_RESPONSE_FIFO_0_LOW (ELBI_BASE + 0x100)
+#define ELBI_RESPONSE_FIFO_LOW(i) (ELBI_RESPONSE_FIFO_0_LOW + (i) * 8 + 0)
+#define ELBI_RESPONSE_FIFO_HIGH(i) (ELBI_RESPONSE_FIFO_0_LOW + (i) * 8 + 4)
+#define ELBI_RESPONSE_FIFO_DEPTH 16
+
+#define ELBI_RESPONSE_PCI_CONTROL (ELBI_BASE + 0x060)
+#define RESPQ_READ_PTR_MASK GENMASK(3, 0)
+#define RESPQ_WRITE_PTR_MASK GENMASK(12, 8)
+
+/*
+ * Host side interrupt status & mask register
+ */
+#define ELBI_PCI_STATUS (ELBI_BASE + 0x008)
+#define ELBI_PCI_MSI_MASK (ELBI_BASE + 0x00C)
+#define ELBI_PCI_STATUS_CMDQ_EMPTY BIT(0)
+#define ELBI_PCI_STATUS_CMDQ_ALMOST_EMPTY BIT(1)
+#define ELBI_PCI_STATUS_CMDQ_READ_UPDATE BIT(2)
+#define ELBI_PCI_STATUS_CMDQ_FLUSH BIT(3)
+#define ELBI_PCI_STATUS_CMDQ_WRITE_ERROR BIT(4)
+#define ELBI_PCI_STATUS_RESPQ_FULL BIT(5)
+#define ELBI_PCI_STATUS_RESPQ_ALMOST_FULL BIT(6)
+#define ELBI_PCI_STATUS_RESPQ_NEW_RESPONSE BIT(7)
+#define ELBI_PCI_STATUS_RESPQ_FLUSH BIT(8)
+#define ELBI_PCI_STATUS_RESPQ_READ_ERROR BIT(9)
+#define ELBI_PCI_STATUS_RESPQ_READ_POINTER_ERROR BIT(10)
+#define ELBI_PCI_STATUS_DOORBELL BIT(11)
+#define ELBI_PCI_STATUS_DOORBELL_READ BIT(12)
+#define ELBI_PCI_STATUS_FLR_REQUEST BIT(13)
+#define ELBI_PCI_STATUS_LOCAL_D3 BIT(14)
+#define ELBI_PCI_STATUS_LOCAL_FLR BIT(15)
+
+/* DOORBELL registers */
+#define ELBI_PCI_HOST_DOORBELL_VALUE (ELBI_BASE + 0x034)
+#define ELBI_HOST_PCI_DOORBELL_VALUE (ELBI_BASE + 0x038)
+#define CARD_DOORBELL_REG ELBI_HOST_PCI_DOORBELL_VALUE
+#define HOST_DOORBELL_REG ELBI_PCI_HOST_DOORBELL_VALUE
+
+/* CPU_STATUS registers */
+/* CPU_STATUS_0 - Updated by BIOS with postcode */
+#define ELBI_CPU_STATUS_0 (ELBI_BASE + 0x1B8)
+/* CPU_STATUS_1 - Updated by BIOS with BIOS flash progress */
+#define ELBI_CPU_STATUS_1 (ELBI_BASE + 0x1BC)
+/* CPU_STATUS_2 - Updated by card driver - bitfields below */
+#define ELBI_CPU_STATUS_2 (ELBI_BASE + 0x1C0)
+/* CPU_STATUS_3 - not used */
+#define ELBI_CPU_STATUS_3 (ELBI_BASE + 0x1C4)
+
+/* Bitfields updated in ELBI_CPU_STATUS_2 indicating card driver states */
+#define ELBI_CPU_STATUS_2_FLR_MODE_MASK GENMASK(1, 0)
+
+/* values for FLR_MODE */
+#define FLR_MODE_WARN_RESET 0
+#define FLR_MODE_COLD_RESET 1
+#define FLR_MODE_IGNORE 3
+
+#endif
diff --git a/drivers/misc/intel-nnpi/nnp_pcie.c b/drivers/misc/intel-nnpi/nnp_pcie.c
new file mode 100644
index 0000000..88280d1
--- /dev/null
+++ b/drivers/misc/intel-nnpi/nnp_pcie.c
@@ -0,0 +1,358 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitfield.h>
+#include <linux/dev_printk.h>
+#include <linux/interrupt.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/wait.h>
+
+#include "device.h"
+#include "nnp_elbi.h"
+#include "nnp_boot_defs.h"
+
+/*
+ * SpringHill PCI card identity settings
+ */
+#define NNP_PCI_DEVICE_ID 0x45c6
+
+/**
+ * struct nnp_pci - structure for NNP-I PCIe device info.
+ * @nnpdev: the NNP-I framework's structure for this NNP-I card device
+ * @pdev: pointer to the pcie device struct
+ * @mmio_va: device's BAR0 mapped virtual address
+ * @mem_bar_va: device's BAR2 mapped virtual address, this is the
+ * "inbound memory region". This device memory region is
+ * described in ipc_include/nnp_inbound_mem.h
+ * @lock: protects accesses to cmd_read_update_count members.
+ * @response_buf: buffer to hold response messages pulled of the device's
+ * response queue.
+ * @card_status_wait: waitq that get signaled when device PCI status has changed
+ * or device has updated its read pointer of the command
+ * queue.
+ * @card_doorbell_val: card's doorbell register value, updated when doorbell
+ * interrupt is received.
+ * @card_status: Last device interrupt status register, updated in interrupt
+ * handler.
+ * @cmd_read_update_count: number of times the device has updated its read
+ * pointer to the device command queue.
+ */
+struct nnp_pci {
+ struct nnp_device nnpdev;
+ struct pci_dev *pdev;
+
+ void __iomem *mmio_va;
+ void __iomem *mem_bar_va;
+
+ spinlock_t lock;
+ u64 response_buf[ELBI_RESPONSE_FIFO_DEPTH];
+ wait_queue_head_t card_status_wait;
+ u32 card_doorbell_val;
+
+ u32 card_status;
+ u32 cmd_read_update_count;
+};
+
+#define NNP_DRIVER_NAME "nnp_pcie"
+
+/* interrupt mask bits we enable and handle at interrupt level */
+static u32 card_status_int_mask = ELBI_PCI_STATUS_CMDQ_READ_UPDATE |
+ ELBI_PCI_STATUS_RESPQ_NEW_RESPONSE |
+ ELBI_PCI_STATUS_DOORBELL;
+
+static inline void nnp_mmio_write(struct nnp_pci *nnp_pci, u32 off, u32 val)
+{
+ iowrite32(val, nnp_pci->mmio_va + off);
+}
+
+static inline u32 nnp_mmio_read(struct nnp_pci *nnp_pci, u32 off)
+{
+ return ioread32(nnp_pci->mmio_va + off);
+}
+
+static inline void nnp_mmio_write_8b(struct nnp_pci *nnp_pci, u32 off, u64 val)
+{
+ lo_hi_writeq(val, nnp_pci->mmio_va + off);
+}
+
+static inline u64 nnp_mmio_read_8b(struct nnp_pci *nnp_pci, u32 off)
+{
+ return lo_hi_readq(nnp_pci->mmio_va + off);
+}
+
+static void nnp_process_commands(struct nnp_pci *nnp_pci)
+{
+ u32 response_pci_control;
+ u32 read_pointer;
+ u32 write_pointer;
+ u32 avail_slots;
+ int i;
+
+ response_pci_control = nnp_mmio_read(nnp_pci, ELBI_RESPONSE_PCI_CONTROL);
+ read_pointer = FIELD_GET(RESPQ_READ_PTR_MASK, response_pci_control);
+ write_pointer = FIELD_GET(RESPQ_WRITE_PTR_MASK, response_pci_control);
+ if (read_pointer > write_pointer) {
+ /* This should never happen on proper device hardware */
+ dev_err(&nnp_pci->pdev->dev, "Mismatched read and write pointers\n");
+ /*
+ * For now just ignore it. Implement handling for such fatal
+ * device errors on a later patch
+ */
+ return;
+ }
+
+ /* Commands to read */
+ avail_slots = write_pointer - read_pointer;
+
+ if (!avail_slots)
+ return;
+
+ for (i = 0; i < avail_slots; i++) {
+ read_pointer = (read_pointer + 1) % ELBI_RESPONSE_FIFO_DEPTH;
+
+ nnp_pci->response_buf[i] =
+ nnp_mmio_read_8b(nnp_pci,
+ ELBI_RESPONSE_FIFO_LOW(read_pointer));
+ }
+
+ /*
+ * HW restriction - we cannot update the read pointer with the same
+ * value it currently have. This will be the case if we need to advance
+ * it by FIFO_DEPTH locations. In this case we will update it in two
+ * steps, first advance by 1, then to the proper value.
+ */
+ if (avail_slots == ELBI_COMMAND_FIFO_DEPTH) {
+ u32 next_read_pointer =
+ (read_pointer + 1) % ELBI_RESPONSE_FIFO_DEPTH;
+
+ response_pci_control &= ~RESPQ_READ_PTR_MASK;
+ response_pci_control |= FIELD_PREP(RESPQ_READ_PTR_MASK,
+ next_read_pointer);
+ nnp_mmio_write(nnp_pci, ELBI_RESPONSE_PCI_CONTROL,
+ response_pci_control);
+ }
+
+ response_pci_control &= ~RESPQ_READ_PTR_MASK;
+ response_pci_control |= FIELD_PREP(RESPQ_READ_PTR_MASK, read_pointer);
+ nnp_mmio_write(nnp_pci, ELBI_RESPONSE_PCI_CONTROL,
+ response_pci_control);
+}
+
+static void mask_all_interrupts(struct nnp_pci *nnp_pci)
+{
+ nnp_mmio_write(nnp_pci, ELBI_PCI_MSI_MASK, GENMASK(31, 0));
+}
+
+static void unmask_interrupts(struct nnp_pci *nnp_pci)
+{
+ nnp_mmio_write(nnp_pci, ELBI_PCI_MSI_MASK, ~card_status_int_mask);
+}
+
+static void notify_card_doorbell_value(struct nnp_pci *nnp_pci)
+{
+ nnp_pci->card_doorbell_val = nnp_mmio_read(nnp_pci, CARD_DOORBELL_REG);
+ nnpdev_card_doorbell_value_changed(&nnp_pci->nnpdev,
+ nnp_pci->card_doorbell_val);
+}
+
+static irqreturn_t threaded_interrupt_handler(int irq, void *data)
+{
+ struct nnp_pci *nnp_pci = data;
+ bool should_wake = false;
+
+ mask_all_interrupts(nnp_pci);
+
+ nnp_pci->card_status = nnp_mmio_read(nnp_pci, ELBI_PCI_STATUS);
+
+ nnp_mmio_write(nnp_pci, ELBI_PCI_STATUS,
+ nnp_pci->card_status & card_status_int_mask);
+
+ if (nnp_pci->card_status & ELBI_PCI_STATUS_CMDQ_READ_UPDATE) {
+ spin_lock(&nnp_pci->lock);
+ should_wake = true;
+ nnp_pci->cmd_read_update_count++;
+ spin_unlock(&nnp_pci->lock);
+ }
+
+ if (nnp_pci->card_status &
+ ELBI_PCI_STATUS_DOORBELL)
+ notify_card_doorbell_value(nnp_pci);
+
+ if (nnp_pci->card_status & ELBI_PCI_STATUS_RESPQ_NEW_RESPONSE)
+ nnp_process_commands(nnp_pci);
+
+ unmask_interrupts(nnp_pci);
+
+ if (should_wake)
+ wake_up_all(&nnp_pci->card_status_wait);
+
+ return IRQ_HANDLED;
+}
+
+static int nnp_setup_interrupts(struct nnp_pci *nnp_pci, struct pci_dev *pdev)
+{
+ int rc;
+ int irq;
+
+ mask_all_interrupts(nnp_pci);
+
+ rc = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI);
+ if (rc < 1)
+ return rc;
+
+ irq = pci_irq_vector(pdev, 0);
+
+ rc = devm_request_threaded_irq(&pdev->dev, irq, NULL,
+ threaded_interrupt_handler, IRQF_ONESHOT,
+ "nnpi-msi", nnp_pci);
+ if (rc)
+ goto err_irq_req_fail;
+
+ return 0;
+
+err_irq_req_fail:
+ pci_free_irq_vectors(pdev);
+ return rc;
+}
+
+static void nnp_free_interrupts(struct nnp_pci *nnp_pci, struct pci_dev *pdev)
+{
+ mask_all_interrupts(nnp_pci);
+ devm_free_irq(&pdev->dev, pci_irq_vector(pdev, 0), nnp_pci);
+ pci_free_irq_vectors(pdev);
+}
+
+static int nnp_cmdq_flush(struct nnp_device *nnpdev)
+{
+ struct nnp_pci *nnp_pci = container_of(nnpdev, struct nnp_pci, nnpdev);
+
+ nnp_mmio_write(nnp_pci, ELBI_COMMAND_PCI_CONTROL,
+ ELBI_COMMAND_PCI_CONTROL_FLUSH_MASK);
+
+ return 0;
+}
+
+static struct nnp_device_ops nnp_device_ops = {
+ .cmdq_flush = nnp_cmdq_flush,
+};
+
+static void set_host_boot_state(struct nnp_pci *nnp_pci, int boot_state)
+{
+ u32 doorbell_val = 0;
+
+ if (boot_state != NNP_HOST_BOOT_STATE_NOT_READY) {
+ doorbell_val = nnp_mmio_read(nnp_pci, HOST_DOORBELL_REG);
+ doorbell_val &= ~NNP_HOST_BOOT_STATE_MASK;
+ doorbell_val |= FIELD_PREP(NNP_HOST_BOOT_STATE_MASK, boot_state);
+ }
+
+ nnp_mmio_write(nnp_pci, HOST_DOORBELL_REG, doorbell_val);
+}
+
+static int nnp_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ struct device *dev = &pdev->dev;
+ struct nnp_pci *nnp_pci;
+ u32 status;
+ int rc;
+
+ nnp_pci = devm_kzalloc(dev, sizeof(*nnp_pci), GFP_KERNEL);
+ if (!nnp_pci)
+ return -ENOMEM;
+
+ nnp_pci->pdev = pdev;
+ pci_set_drvdata(pdev, nnp_pci);
+
+ init_waitqueue_head(&nnp_pci->card_status_wait);
+ spin_lock_init(&nnp_pci->lock);
+
+ rc = pcim_enable_device(pdev);
+ if (rc)
+ return dev_err_probe(dev, rc, "enable_device\n");
+
+ pci_set_master(pdev);
+
+ rc = pcim_iomap_regions(pdev, BIT(0) | BIT(2), NNP_DRIVER_NAME);
+ if (rc)
+ return dev_err_probe(dev, rc, "iomap_regions\n");
+
+ nnp_pci->mmio_va = pcim_iomap_table(pdev)[0];
+ nnp_pci->mem_bar_va = pcim_iomap_table(pdev)[2];
+
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
+ if (rc)
+ return dev_err_probe(dev, rc, "dma_set_mask\n");
+
+ rc = nnp_setup_interrupts(nnp_pci, pdev);
+ if (rc)
+ return dev_err_probe(dev, rc, "nnp_setup_interrupts\n");
+
+ /*
+ * done setting up the new pci device,
+ * add it to the NNP-I framework.
+ */
+ rc = nnpdev_init(&nnp_pci->nnpdev, dev, &nnp_device_ops);
+ if (rc)
+ return dev_err_probe(dev, rc, "nnpdev_init\n");
+
+ /* notify bios that host driver is up */
+ nnp_cmdq_flush(&nnp_pci->nnpdev);
+ set_host_boot_state(nnp_pci, NNP_HOST_BOOT_STATE_DRV_READY);
+
+ /* Update NNP-I framework with current value of card doorbell value */
+ notify_card_doorbell_value(nnp_pci);
+ status = nnp_mmio_read(nnp_pci, ELBI_PCI_STATUS);
+ if (status & ELBI_PCI_STATUS_DOORBELL)
+ nnp_mmio_write(nnp_pci, ELBI_PCI_STATUS, ELBI_PCI_STATUS_DOORBELL);
+
+ /* process any existing command in the response queue */
+ nnp_process_commands(nnp_pci);
+
+ /* Enable desired interrupts */
+ unmask_interrupts(nnp_pci);
+
+ return 0;
+}
+
+static void nnp_remove(struct pci_dev *pdev)
+{
+ struct nnp_pci *nnp_pci = pci_get_drvdata(pdev);
+
+ /* stop service new interrupts */
+ nnp_free_interrupts(nnp_pci, nnp_pci->pdev);
+
+ /*
+ * Inform card that host driver is down.
+ * This will also clear any state on the card so that
+ * if card is inserted again, it will be in a good, clear
+ * state.
+ */
+ set_host_boot_state(nnp_pci, NNP_HOST_BOOT_STATE_NOT_READY);
+
+ nnpdev_destroy(&nnp_pci->nnpdev);
+}
+
+static const struct pci_device_id nnp_pci_tbl[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_INTEL, NNP_PCI_DEVICE_ID) },
+ { }
+};
+
+static struct pci_driver nnp_driver = {
+ .name = NNP_DRIVER_NAME,
+ .id_table = nnp_pci_tbl,
+ .probe = nnp_probe,
+ .remove = nnp_remove,
+};
+
+module_pci_driver(nnp_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Intel(R) NNP-I PCIe driver");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DEVICE_TABLE(pci, nnp_pci_tbl);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 03/15] misc: nnpi: Manage and schedule messages to device
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 01/15] misc: nnpi: Document NNP-I's driver overview Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 02/15] misc: nnpi: Initialize NNP-I framework and PCIe modules Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 04/15] misc: nnpi: Define host/card ipc protocol Guy Zadicario
` (12 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Allocate a msg_scheduler object for each NNP-I device. Each object
manages multiple command queues - one for driver initiated commands
(called "cmdq"), and the rest for commands coming from user-space. A
kernel thread in the msg_scheduler schedules sending the commands from
these queues to the NNP-I device by using the cmdq_write_mesg function of
the NNP-I device driver ops.
The msg_scheduler object allows multiple user-mode applications to put
messages into the queue without blocking or waiting on a lock. It's
created on device creation and destroyed on device removal.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/Makefile | 2 +-
drivers/misc/intel-nnpi/device.c | 20 ++
drivers/misc/intel-nnpi/device.h | 9 +
drivers/misc/intel-nnpi/msg_scheduler.c | 319 ++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/msg_scheduler.h | 153 +++++++++++++++
drivers/misc/intel-nnpi/nnp_pcie.c | 153 +++++++++++++++
6 files changed, 655 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/intel-nnpi/msg_scheduler.c
create mode 100644 drivers/misc/intel-nnpi/msg_scheduler.h
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index 84b7528..43f09c0 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -5,7 +5,7 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
-intel_nnpi-y := device.o
+intel_nnpi-y := device.o msg_scheduler.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 3d80e95..60c5a94 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include "device.h"
+#include "msg_scheduler.h"
static DEFINE_IDA(dev_ida);
@@ -45,7 +46,25 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
nnpdev->dev = dev;
nnpdev->ops = ops;
+ nnpdev->cmdq_sched = nnp_msched_create(nnpdev);
+ if (!nnpdev->cmdq_sched) {
+ ret = -ENOMEM;
+ goto err_ida;
+ }
+
+ nnpdev->cmdq = nnp_msched_queue_create(nnpdev->cmdq_sched);
+ if (!nnpdev->cmdq) {
+ ret = -ENOMEM;
+ goto err_msg_sched;
+ }
+
return 0;
+
+err_msg_sched:
+ nnp_msched_destroy(nnpdev->cmdq_sched);
+err_ida:
+ ida_simple_remove(&dev_ida, nnpdev->id);
+ return ret;
}
EXPORT_SYMBOL(nnpdev_init);
@@ -75,6 +94,7 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
{
dev_dbg(nnpdev->dev, "Destroying NNP-I device\n");
+ nnp_msched_destroy(nnpdev->cmdq_sched);
ida_simple_remove(&dev_ida, nnpdev->id);
}
EXPORT_SYMBOL(nnpdev_destroy);
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 4ff7aa9..7d7ef60 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -11,20 +11,29 @@
* @ops: device operations implemented by the underlying device driver
* @dev: pointer to struct device representing the NNP-I card.
* @id: NNP-I device number
+ * @cmdq_sched: message scheduler thread which schedules and serializes command
+ * submissions to the device's command queue.
+ * @cmdq: input queue to @cmdq_sched used to schedule driver internal commands
+ * to be sent to the device.
*/
struct nnp_device {
const struct nnp_device_ops *ops;
struct device *dev;
int id;
+
+ struct nnp_msched *cmdq_sched;
+ struct nnp_msched_queue *cmdq;
};
/**
* struct nnp_device_ops - operations implemented by underlying device driver
* @cmdq_flush: empties the device command queue, discarding all queued
* commands.
+ * @cmdq_write_mesg: inserts a command message to the card's command queue.
*/
struct nnp_device_ops {
int (*cmdq_flush)(struct nnp_device *hw_dev);
+ int (*cmdq_write_mesg)(struct nnp_device *nnpdev, u64 *msg, u32 size);
};
/*
diff --git a/drivers/misc/intel-nnpi/msg_scheduler.c b/drivers/misc/intel-nnpi/msg_scheduler.c
new file mode 100644
index 0000000..c018630
--- /dev/null
+++ b/drivers/misc/intel-nnpi/msg_scheduler.c
@@ -0,0 +1,319 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+/*
+ * message scheduler implementation.
+ *
+ * That implements a scheduler object which is used to serialize
+ * command submission to an NNP-I device.
+ * It manages a list of message queues which hold command messages
+ * to be submitted to the card.
+ * It also implements a kernel thread which schedules draining
+ * the message queues in round-robin fashion.
+ *
+ * An instance of this object is created for each NNP-I device.
+ * A message queue is created for each user created channel as well
+ * as one message queue which is used by the kernel driver itself.
+ */
+
+#include <linux/err.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/printk.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "device.h"
+#include "msg_scheduler.h"
+
+/**
+ * struct msg_entry - struct to hold a single command message
+ * @msg: command message payload
+ * @size: size in 64-bit words
+ * @node: node to be included in list of command messages.
+ */
+struct msg_entry {
+ u64 msg[MSG_SCHED_MAX_MSG_SIZE];
+ unsigned int size;
+ struct list_head node;
+};
+
+/**
+ * do_sched() - fetch and write a message from one message queue.
+ * @sched: the scheduler
+ * @q: the queue to handle
+ *
+ * This function is called from the main scheduler thread to handle single
+ * message queue. It fetches one message from the queue and send it to the
+ * NNP-I device.
+ *
+ * The function should be called when the scheduler mutex is held to prevent
+ * the queue from being destroyed.
+ */
+static void do_sched(struct nnp_msched *sched, struct nnp_msched_queue *q)
+{
+ struct nnp_device *nnpdev = sched->nnpdev;
+ struct msg_entry *msg;
+ unsigned int left_msgs;
+
+ lockdep_assert_held(&sched->mutex);
+
+ /* Fetch one message from the queue */
+ spin_lock(&q->list_lock);
+ if (list_empty(&q->msgs)) {
+ spin_unlock(&q->list_lock);
+ return;
+ }
+
+ msg = list_first_entry(&q->msgs, struct msg_entry, node);
+ list_del(&msg->node);
+ q->msgs_num--;
+ left_msgs = q->msgs_num;
+ spin_lock(&sched->total_msgs_lock);
+ sched->total_msgs--;
+ spin_unlock(&sched->total_msgs_lock);
+ spin_unlock(&q->list_lock);
+
+ /*
+ * Write the fetched message out.
+ * Note that cmdq_write_mesg function may sleep.
+ */
+ nnpdev->ops->cmdq_write_mesg(nnpdev, msg->msg, msg->size);
+
+ kmem_cache_free(sched->slab_cache_ptr, msg);
+
+ /*
+ * Wake any waiting sync thread if the queue just
+ * became empty
+ */
+ if (!left_msgs)
+ wake_up_all(&q->sync_waitq);
+}
+
+/**
+ * msg_sched_thread() - the main function of the scheduler thread.
+ * @data: pointer to the msg scheduler object.
+ *
+ * This is the main function of the scheduler kernel thread.
+ * It loops in round-robin fashion on all queues, pulls one message
+ * each time and send it to the NNP-I device.
+ * For each application created channel, a different queue of
+ * command messages is allocated. This thread schedules and serializes
+ * accesses to the NNP-I device's command queue.
+ *
+ * Return: 0 when thread is stopped
+ */
+static int msg_sched_thread(void *data)
+{
+ struct nnp_msched *dev_sched = data;
+ struct nnp_msched_queue *q;
+ bool need_sched;
+
+ while (!kthread_should_stop()) {
+ mutex_lock(&dev_sched->mutex);
+ list_for_each_entry(q, &dev_sched->queues, node)
+ do_sched(dev_sched, q);
+
+ /*
+ * Wait for new messages to be available in some queue
+ * if no messages are known to exist
+ */
+ spin_lock(&dev_sched->total_msgs_lock);
+ set_current_state(TASK_INTERRUPTIBLE);
+ need_sched = !dev_sched->total_msgs;
+ spin_unlock(&dev_sched->total_msgs_lock);
+ mutex_unlock(&dev_sched->mutex);
+ if (need_sched)
+ schedule();
+ set_current_state(TASK_RUNNING);
+ }
+
+ return 0;
+}
+
+struct nnp_msched_queue *nnp_msched_queue_create(struct nnp_msched *scheduler)
+{
+ struct nnp_msched_queue *queue;
+
+ queue = kzalloc(sizeof(*queue), GFP_KERNEL);
+ if (!queue)
+ return NULL;
+
+ INIT_LIST_HEAD(&queue->msgs);
+ spin_lock_init(&queue->list_lock);
+ queue->msgs_num = 0;
+ queue->scheduler = scheduler;
+ init_waitqueue_head(&queue->sync_waitq);
+
+ mutex_lock(&scheduler->mutex);
+ list_add_tail(&queue->node, &scheduler->queues);
+ mutex_unlock(&scheduler->mutex);
+
+ return queue;
+}
+
+int nnp_msched_queue_destroy(struct nnp_msched_queue *queue)
+{
+ struct msg_entry *msg;
+
+ /* detach the queue from list of scheduled queues */
+ mutex_lock(&queue->scheduler->mutex);
+ list_del(&queue->node);
+ mutex_unlock(&queue->scheduler->mutex);
+
+ /* destroy all the messages of the queue */
+ spin_lock(&queue->list_lock);
+ while (!list_empty(&queue->msgs)) {
+ msg = list_first_entry(&queue->msgs, struct msg_entry, node);
+ list_del(&msg->node);
+ kmem_cache_free(queue->scheduler->slab_cache_ptr, msg);
+ }
+ spin_unlock(&queue->list_lock);
+
+ kfree(queue);
+
+ return 0;
+}
+
+static inline bool is_queue_empty(struct nnp_msched_queue *queue)
+{
+ bool ret;
+
+ spin_lock(&queue->list_lock);
+ ret = list_empty(&queue->msgs);
+ spin_unlock(&queue->list_lock);
+
+ return ret;
+}
+
+int nnp_msched_queue_sync(struct nnp_msched_queue *queue)
+{
+ int ret;
+
+ /* Wait for the queue to be empty */
+ ret = wait_event_interruptible(queue->sync_waitq, is_queue_empty(queue));
+
+ return ret;
+}
+
+int nnp_msched_queue_add_msg(struct nnp_msched_queue *queue, u64 *msg,
+ unsigned int size)
+{
+ unsigned int i;
+ struct msg_entry *m;
+ bool throttled;
+
+ if (size > MSG_SCHED_MAX_MSG_SIZE)
+ return -EINVAL;
+
+ m = kmem_cache_alloc(queue->scheduler->slab_cache_ptr, GFP_KERNEL);
+ if (!m)
+ return -ENOMEM;
+
+ for (i = 0; i < size; i++)
+ m->msg[i] = msg[i];
+
+ m->size = size;
+
+ spin_lock(&queue->list_lock);
+ throttled = queue->throttled;
+ if (!throttled) {
+ list_add_tail(&m->node, &queue->msgs);
+ queue->msgs_num++;
+ spin_lock(&queue->scheduler->total_msgs_lock);
+ queue->scheduler->total_msgs++;
+ spin_unlock(&queue->scheduler->total_msgs_lock);
+ }
+ spin_unlock(&queue->list_lock);
+
+ /* if queue flagged as throttled - silently ignore the message */
+ if (throttled) {
+ kmem_cache_free(queue->scheduler->slab_cache_ptr, m);
+ return 0;
+ }
+
+ wake_up_process(queue->scheduler->thread);
+
+ return 0;
+}
+
+struct nnp_msched *nnp_msched_create(struct nnp_device *nnpdev)
+{
+ struct nnp_msched *dev_sched;
+
+ dev_sched = kzalloc(sizeof(*dev_sched), GFP_KERNEL);
+ if (!dev_sched)
+ return NULL;
+
+ dev_sched->slab_cache_ptr = kmem_cache_create("msg_sched_slab",
+ sizeof(struct msg_entry),
+ 0, 0, NULL);
+ if (!dev_sched->slab_cache_ptr) {
+ kfree(dev_sched);
+ return NULL;
+ }
+
+ INIT_LIST_HEAD(&dev_sched->queues);
+
+ spin_lock_init(&dev_sched->total_msgs_lock);
+ mutex_init(&dev_sched->mutex);
+ dev_sched->nnpdev = nnpdev;
+
+ dev_sched->thread = kthread_run(msg_sched_thread, dev_sched,
+ "msg_sched_thread");
+ if (!dev_sched->thread) {
+ kmem_cache_destroy(dev_sched->slab_cache_ptr);
+ kfree(dev_sched);
+ return NULL;
+ }
+
+ return dev_sched;
+}
+
+void nnp_msched_destroy(struct nnp_msched *sched)
+{
+ struct nnp_msched_queue *q, *tmp;
+
+ nnp_msched_throttle_all(sched);
+
+ kthread_stop(sched->thread);
+
+ mutex_lock(&sched->mutex);
+ list_for_each_entry_safe(q, tmp, &sched->queues, node) {
+ /* destroy the queue */
+ list_del(&q->node);
+ kfree(q);
+ }
+ mutex_unlock(&sched->mutex);
+
+ kmem_cache_destroy(sched->slab_cache_ptr);
+
+ kfree(sched);
+}
+
+void nnp_msched_throttle_all(struct nnp_msched *sched)
+{
+ struct nnp_msched_queue *q;
+ struct msg_entry *msg, *tmp;
+
+ /*
+ * For each queue:
+ * 1) throttle the queue, so that no more messages will be inserted
+ * 2) delete all existing messages
+ */
+ mutex_lock(&sched->mutex);
+ list_for_each_entry(q, &sched->queues, node) {
+ spin_lock(&q->list_lock);
+ q->throttled = true;
+ list_for_each_entry_safe(msg, tmp, &q->msgs, node) {
+ list_del(&msg->node);
+ kmem_cache_free(sched->slab_cache_ptr, msg);
+ }
+ q->msgs_num = 0;
+ spin_unlock(&q->list_lock);
+ wake_up_all(&q->sync_waitq);
+ }
+ mutex_unlock(&sched->mutex);
+}
diff --git a/drivers/misc/intel-nnpi/msg_scheduler.h b/drivers/misc/intel-nnpi/msg_scheduler.h
new file mode 100644
index 0000000..c8033f2
--- /dev/null
+++ b/drivers/misc/intel-nnpi/msg_scheduler.h
@@ -0,0 +1,153 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNP_MSGF_SCHEDULER_H
+#define _NNP_MSGF_SCHEDULER_H
+
+#include <linux/mutex.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+
+#define MSG_SCHED_MAX_MSG_SIZE 3 /* maximum command message size, i qwords */
+
+/**
+ * struct nnp_msched - structure for msg scheduler object
+ * @thread: kernel thread which schedules message writes to device
+ * @nnpdev: the device the scheduler writes to
+ * @queues: list of message queues to schedule from
+ * @total_msgs_lock: protects accesses to @total_msgs
+ * @mutex: protects modifications to @queues
+ * @total_msgs: total count of messages in all queues yet to be written.
+ * @slab_cache_ptr: used to allocate entries in msg queue list.
+ *
+ * We have one msg scheduler object allocated for each NNP-I device,
+ * It manages a list of command message queues and a kernel thread
+ * which schedules sending the command messages to the device in a
+ * round-robin fashion.
+ */
+struct nnp_msched {
+ struct task_struct *thread;
+ struct nnp_device *nnpdev;
+ struct list_head queues;
+ spinlock_t total_msgs_lock;
+ struct mutex mutex;
+ unsigned int total_msgs;
+ struct kmem_cache *slab_cache_ptr;
+};
+
+/**
+ * struct nnp_msched_queue - structure to hold one list of command messages
+ * @scheduler: the scheduler object this queue belongs to
+ * @node: node of this element in @queues in msg_sched
+ * @msgs: list of command messages
+ * @sync_waitq: waitq used for waiting until queue becomes empty
+ * @throttled: if true, all messages in the queue should be discarded and no new
+ * messages can be added to it until it will become un-throttled.
+ * @msgs_num: number of messages in the queue
+ * @list_lock: protects @msgs
+ *
+ * This structure holds a list of command messages to be queued for submission
+ * to the device. Each application holding a channel for command submissions
+ * has its own command message queue.
+ */
+struct nnp_msched_queue {
+ struct nnp_msched *scheduler;
+ struct list_head node;
+ struct list_head msgs;
+ wait_queue_head_t sync_waitq;
+ bool throttled;
+ unsigned int msgs_num;
+ spinlock_t list_lock;
+};
+
+/**
+ * nnp_msched_create() - creates msg scheduler object
+ * @nnpdev: the device this scheduler writes messages to.
+ *
+ * This function creates a message scheduler object which can hold
+ * multiple message queues and a scheduling thread which pop messages
+ * from the different queues and synchronously sends them down to the device
+ * for transmission.
+ *
+ * Return: pointer to allocated scheduler object or NULL on failure
+ */
+struct nnp_msched *nnp_msched_create(struct nnp_device *nnpdev);
+
+/**
+ * nnp_msched_destroy() - destroys a msg scheduler object
+ * @sched: pointer to msg scheduler object
+ *
+ * This function will wait for the scheduler thread to complete
+ * and destroys the scheduler object as well as all messages and message
+ * queues.
+ * NOTE: caller must make sure that no new queues and messages will be added
+ * to this scheduler object while this function is in progress! There is no
+ * mutex to protect this, should be handled by the caller.
+ */
+void nnp_msched_destroy(struct nnp_msched *sched);
+
+/**
+ * nnp_msched_throttle_all() - Remove all messages and throttle all queues
+ * @sched: pointer to msg scheduler object
+ *
+ * This function removes all messages from all queues and marks all queues
+ * as throttled. No new messages can be added to a throttled queue until it
+ * becomes unthrottled.
+ *
+ * This function is called before the device is reset in order to stop sending
+ * any more messages to the device. When the reset is complete, the message
+ * queues are unthrottled. This is done to make sure that no messages generated
+ * before the reset will be sent to the device, also after the reset completes.
+ */
+void nnp_msched_throttle_all(struct nnp_msched *sched);
+
+/**
+ * nnp_msched_queue_create() - create a queue of messages handled by scheduler
+ * @scheduler: the msg scheduler object
+ *
+ * Return: pointer to msg scheduler queue object, NULL on failure.
+ */
+struct nnp_msched_queue *nnp_msched_queue_create(struct nnp_msched *scheduler);
+
+/**
+ * nnp_msched_queue_destroy() - destroy a message queue object
+ * @queue: the message queue object to be destroyed.
+ *
+ * This function destroys a message queue object, if the queue is not empty
+ * and still contains messages, the messages will be discarded and not sent to
+ * the device.
+ *
+ * Return: 0 on success.
+ */
+int nnp_msched_queue_destroy(struct nnp_msched_queue *queue);
+
+/**
+ * nnp_msched_queue_sync() - wait for message queue to be empty
+ * @queue: the message queue object
+ *
+ * Return: 0 on success, error value otherwise.
+ */
+int nnp_msched_queue_sync(struct nnp_msched_queue *queue);
+
+/**
+ * nnp_msched_queue_add_msg() - adds a message packet to a message queue
+ * @queue: the message queue object
+ * @msg: pointer to message content
+ * @size: size of message in 64-bit units
+ *
+ * This function adds a message to the queue. The message will be sent
+ * once the scheduler thread drains it from the queue.
+ *
+ * Return: 0 on success, error value otherwise
+ */
+int nnp_msched_queue_add_msg(struct nnp_msched_queue *queue, u64 *msg,
+ unsigned int size);
+
+/*
+ * Utility macro for calling nnp_msched_queue_add_msg by passing u64 array
+ * object which forms the message.
+ */
+#define nnp_msched_queue_msg(q, m) \
+ nnp_msched_queue_add_msg((q), (u64 *)&(m), sizeof((m)) / sizeof(u64))
+
+#endif /* _NNP_MSGF_SCHEDULER_H */
diff --git a/drivers/misc/intel-nnpi/nnp_pcie.c b/drivers/misc/intel-nnpi/nnp_pcie.c
index 88280d1..7aa9074 100644
--- a/drivers/misc/intel-nnpi/nnp_pcie.c
+++ b/drivers/misc/intel-nnpi/nnp_pcie.c
@@ -38,10 +38,14 @@
* queue.
* @card_doorbell_val: card's doorbell register value, updated when doorbell
* interrupt is received.
+ * @cmdq_free_slots: number of slots in the device's command queue which is known
+ * to be available.
+ * @cmdq_lock: protects @cmdq_free_slots calculation.
* @card_status: Last device interrupt status register, updated in interrupt
* handler.
* @cmd_read_update_count: number of times the device has updated its read
* pointer to the device command queue.
+ * @removing: true if device remove is in progress.
*/
struct nnp_pci {
struct nnp_device nnpdev;
@@ -55,8 +59,12 @@ struct nnp_pci {
wait_queue_head_t card_status_wait;
u32 card_doorbell_val;
+ u32 cmdq_free_slots;
+ spinlock_t cmdq_lock;
+
u32 card_status;
u32 cmd_read_update_count;
+ bool removing;
};
#define NNP_DRIVER_NAME "nnp_pcie"
@@ -228,6 +236,135 @@ static void nnp_free_interrupts(struct nnp_pci *nnp_pci, struct pci_dev *pdev)
pci_free_irq_vectors(pdev);
}
+/**
+ * nnp_cmdq_write_mesg_nowait() - tries to write full message to command queue
+ * @nnp_pci: the device
+ * @msg: pointer to the command message
+ * @size: size of the command message in qwords
+ * @read_update_count: returns current cmd_read_update_count value,
+ * valid only if function returns -EAGAIN.
+ *
+ * Return:
+ * * 0: Success, command has been written
+ * * -EAGAIN: command queue does not have room for the entire command
+ * message.
+ * read_update_count returns the current value of
+ * cmd_read_update_count counter which increments when the device
+ * advance its command queue read pointer. The caller may wait
+ * for this counter to be advanced past this point before calling
+ * this function again to re-try the write.
+ * * -ENODEV: device remove is in progress.
+ */
+static int nnp_cmdq_write_mesg_nowait(struct nnp_pci *nnp_pci, u64 *msg,
+ u32 size, u32 *read_update_count)
+{
+ u32 cmd_iosf_control;
+ u32 read_pointer, write_pointer;
+ int i;
+
+ if (nnp_pci->removing)
+ return -ENODEV;
+
+ if (!size)
+ return 0;
+
+ spin_lock(&nnp_pci->cmdq_lock);
+
+ if (nnp_pci->cmdq_free_slots < size) {
+ /* read command fifo pointers and compute free slots in fifo */
+ spin_lock(&nnp_pci->lock);
+ cmd_iosf_control = nnp_mmio_read(nnp_pci, ELBI_COMMAND_IOSF_CONTROL);
+ read_pointer = FIELD_GET(CMDQ_READ_PTR_MASK, cmd_iosf_control);
+ write_pointer =
+ FIELD_GET(CMDQ_WRITE_PTR_MASK, cmd_iosf_control);
+
+ nnp_pci->cmdq_free_slots = ELBI_COMMAND_FIFO_DEPTH -
+ (write_pointer - read_pointer);
+
+ if (nnp_pci->cmdq_free_slots < size) {
+ *read_update_count = nnp_pci->cmd_read_update_count;
+ spin_unlock(&nnp_pci->lock);
+ spin_unlock(&nnp_pci->cmdq_lock);
+ return -EAGAIN;
+ }
+ spin_unlock(&nnp_pci->lock);
+ }
+
+ /* Write all but the last qword without generating msi on card */
+ for (i = 0; i < size - 1; i++)
+ nnp_mmio_write_8b(nnp_pci, ELBI_COMMAND_WRITE_WO_MSI_LOW, msg[i]);
+
+ /* Write last qword with generating interrupt on card */
+ nnp_mmio_write_8b(nnp_pci, ELBI_COMMAND_WRITE_W_MSI_LOW, msg[i]);
+
+ nnp_pci->cmdq_free_slots -= size;
+
+ spin_unlock(&nnp_pci->cmdq_lock);
+
+ return 0;
+}
+
+/**
+ * check_read_count() - check if device has read commands from command FIFO
+ * @nnp_pci: the device
+ * @count: last known 'cmd_read_update_count' value
+ *
+ * cmd_read_update_count is advanced on each interrupt received because the
+ * device has advanced its read pointer into the command FIFO.
+ * This function checks the current cmd_read_update_count against @count and
+ * returns true if it is different. This is used to check if the device has
+ * freed some entries in the command FIFO after it became full.
+ *
+ * Return: true if current device read update count has been advanced
+ */
+static bool check_read_count(struct nnp_pci *nnp_pci, u32 count)
+{
+ bool ret;
+
+ spin_lock(&nnp_pci->lock);
+ ret = (count != nnp_pci->cmd_read_update_count);
+ spin_unlock(&nnp_pci->lock);
+
+ return ret;
+}
+
+/**
+ * nnp_cmdq_write_mesg() - writes a command message to device's command queue
+ * @nnpdev: the device handle
+ * @msg: The command message to write
+ * @size: size of the command message in qwords
+ *
+ * Return:
+ * * 0: Success, command has been written
+ * * -ENODEV: device remove is in progress.
+ */
+static int nnp_cmdq_write_mesg(struct nnp_device *nnpdev, u64 *msg, u32 size)
+{
+ int rc;
+ u32 rcnt = 0;
+ struct nnp_pci *nnp_pci = container_of(nnpdev, struct nnp_pci, nnpdev);
+
+ do {
+ rc = nnp_cmdq_write_mesg_nowait(nnp_pci, msg, size, &rcnt);
+ if (rc != -EAGAIN)
+ break;
+
+ rc = wait_event_interruptible(nnp_pci->card_status_wait,
+ check_read_count(nnp_pci, rcnt) ||
+ nnp_pci->removing);
+ if (!rc && nnp_pci->removing) {
+ rc = -ENODEV;
+ break;
+ }
+ } while (!rc);
+
+ if (rc)
+ dev_dbg(&nnp_pci->pdev->dev,
+ "Failed to write message size %u rc=%d!\n", size, rc);
+
+ return rc;
+}
+
static int nnp_cmdq_flush(struct nnp_device *nnpdev)
{
struct nnp_pci *nnp_pci = container_of(nnpdev, struct nnp_pci, nnpdev);
@@ -240,6 +377,7 @@ static int nnp_cmdq_flush(struct nnp_device *nnpdev)
static struct nnp_device_ops nnp_device_ops = {
.cmdq_flush = nnp_cmdq_flush,
+ .cmdq_write_mesg = nnp_cmdq_write_mesg,
};
static void set_host_boot_state(struct nnp_pci *nnp_pci, int boot_state)
@@ -271,6 +409,7 @@ static int nnp_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
init_waitqueue_head(&nnp_pci->card_status_wait);
spin_lock_init(&nnp_pci->lock);
+ spin_lock_init(&nnp_pci->cmdq_lock);
rc = pcim_enable_device(pdev);
if (rc)
@@ -328,6 +467,20 @@ static void nnp_remove(struct pci_dev *pdev)
nnp_free_interrupts(nnp_pci, nnp_pci->pdev);
/*
+ * Flag that the device is being removed and wake any possible
+ * thread waiting on the card's command queue.
+ * During the remove flow, we want to immediately fail any thread
+ * that is using the device without waiting for pending device
+ * requests to complete. We rather give precedance to device
+ * removal over waiting for all pending requests to finish.
+ * When we set the host boot state to "NOT_READY" in the doorbell
+ * register, the card will cleanup any state, so this "hard remove"
+ * is not an issue for next time the device will get inserted.
+ */
+ nnp_pci->removing = true;
+ wake_up_all(&nnp_pci->card_status_wait);
+
+ /*
* Inform card that host driver is down.
* This will also clear any state on the card so that
* if card is inserted again, it will be in a good, clear
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 04/15] misc: nnpi: Define host/card ipc protocol
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (2 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 03/15] misc: nnpi: Manage and schedule messages to device Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 05/15] misc: nnpi: Manage host memory resources Guy Zadicario
` (11 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Define the commands and memory block structures which can be sent to
the NNP-I device from the "device" layer.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h | 337 +++++++++++++++++++++
1 file changed, 337 insertions(+)
create mode 100644 drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
diff --git a/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h b/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
new file mode 100644
index 0000000..59b4a79
--- /dev/null
+++ b/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
@@ -0,0 +1,337 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _IPC_PROTOCOL_H
+#define _IPC_PROTOCOL_H
+
+#include <linux/bits.h>
+#include <linux/bitfield.h>
+#include <linux/dma-mapping.h>
+#include <linux/types.h>
+
+#define IPC_OP_MAX BIT(6)
+#define NNP_IPC_OPCODE_MASK GENMASK(5, 0)
+
+#define NNP_MSG_SIZE(msg) (sizeof(msg) / sizeof(__le64))
+
+/*
+ * NNP_PAGE_SIZE is a size of a page in the protocol.
+ * We do not use just PAGE_SIZE since it may differ between
+ * card and host.
+ */
+#define NNP_PAGE_SHIFT 12
+#define NNP_PAGE_SIZE BIT(12)
+
+#define NNP_VERSION_DOT_MASK GENMASK(4, 0)
+#define NNP_VERSION_MINOR_MASK GENMASK(9, 5)
+#define NNP_VERSION_MAJOR_MASK GENMASK(14, 10)
+#define NNP_VERSION_MAJOR(ver) FIELD_GET(NNP_VERSION_MAJOR_MASK, (ver))
+#define NNP_VERSION_MINOR(ver) FIELD_GET(NNP_VERSION_MINOR_MASK, (ver))
+#define NNP_VERSION_DOT(ver) FIELD_GET(NNP_VERSION_DOT_MASK, (ver))
+#define NNP_MAKE_VERSION(major, minor, dot) \
+ (FIELD_PREP(NNP_VERSION_MAJOR_MASK, (major)) | \
+ FIELD_PREP(NNP_VERSION_MINOR_MASK, (minor)) | \
+ FIELD_PREP(NNP_VERSION_DOT_MASK, (dot)))
+
+#define NNP_IPC_PROTOCOL_VERSION NNP_MAKE_VERSION(4, 1, 0)
+
+#define NNP_IPC_DMA_PFN_BITS 45 /* size of physical address in protocol */
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+#define NNP_IPC_DMA_PFN_MASK GENMASK_ULL(56, 12)
+#define NNP_IPC_DMA_MAX_ADDR GENMASK_ULL(56, 0)
+#else
+#define NNP_IPC_DMA_PFN_MASK GENMASK_ULL(31, 12)
+#define NNP_IPC_DMA_MAX_ADDR GENMASK_ULL(31, 0)
+#endif
+#define NNP_IPC_DMA_ADDR_TO_PFN(dma_addr) \
+ FIELD_GET(NNP_IPC_DMA_PFN_MASK, (dma_addr))
+#define NNP_IPC_DMA_PFN_TO_ADDR(dma_pfn) \
+ FIELD_PREP(NNP_IPC_DMA_PFN_MASK, (dma_pfn))
+
+#define NNP_IPC_INF_CHANNEL_BITS 8
+#define NNP_IPC_CHANNEL_BITS 10
+#define NNP_IPC_MAX_CHANNEL_RB 2
+
+/*
+ * Structures used inside data packets transferred in the protocol
+ */
+struct nnp_dma_chain_header {
+ __le64 dma_next;
+ __le32 total_nents;
+ __le32 start_offset;
+ __le64 size;
+} __packed;
+
+#define DMA_CHAIN_ENTRY_SIZE sizeof(__le64)
+#define DMA_CHAIN_HEADER_SIZE sizeof(struct nnp_dma_chain_header)
+#define DMA_CHAIN_ENTRY_NPAGES_BITS \
+ (sizeof(__le64) * __CHAR_BIT__ - NNP_IPC_DMA_PFN_BITS)
+#define NNP_MAX_CHUNK_SIZE \
+ (((1lu << DMA_CHAIN_ENTRY_NPAGES_BITS) - 1) << NNP_PAGE_SHIFT)
+
+struct nnp_dma_chain_entry {
+ u64 dma_chunk_pfn : NNP_IPC_DMA_PFN_BITS;
+ u64 n_pages : DMA_CHAIN_ENTRY_NPAGES_BITS;
+} __packed;
+
+#define DMA_CHAIN_ENTRY_PFN_MASK GENMASK_ULL(NNP_IPC_DMA_PFN_BITS - 1, 0)
+#define DMA_CHAIN_ENTRY_NPAGES_MASK GENMASK_ULL(63, NNP_IPC_DMA_PFN_BITS)
+
+#define NENTS_PER_PAGE \
+ ((NNP_PAGE_SIZE - DMA_CHAIN_HEADER_SIZE) / DMA_CHAIN_ENTRY_SIZE)
+
+/*
+ * IPC messages layout definition
+ */
+
+#define NNP_C2H_OP_MASK GENMASK_ULL(5, 0)
+
+/* NNP_IPC_C2H_OP_QUERY_VERSION_REPLY3 - 3 qwords */
+/* qword 0: */
+#define NNP_C2H_VERSION_REPLY_QW0_OP_MASK NNP_C2H_OP_MASK
+#define NNP_C2H_VERSION_REPLY_QW0_PROT_VER_MASK GENMASK_ULL(21, 6)
+#define NNP_C2H_VERSION_REPLY_QW0_FW_VER_MASK GENMASK_ULL(37, 22)
+#define NNP_C2H_VERSION_REPLY_QW0_CHAN_VER_MASK GENMASK_ULL(53, 38)
+/* qword 1: two bits for each possible response opcode specifying its size */
+/* qword 2: two bits for each possible command opcode specifying its size */
+
+/* NNP_IPC_C2H_OP_EVENT_REPORT - 1 qword */
+#define NNP_C2H_EVENT_REPORT_OP_MASK NNP_C2H_OP_MASK
+#define NNP_C2H_EVENT_REPORT_CODE_MASK GENMASK_ULL(12, 6)
+#define NNP_C2H_EVENT_REPORT_CHAN_ID_MASK GENMASK_ULL(20, 13)
+#define NNP_C2H_EVENT_REPORT_OBJ_ID_MASK GENMASK_ULL(36, 21)
+#define NNP_C2H_EVENT_REPORT_OBJ_ID2_MASK GENMASK_ULL(52, 37)
+#define NNP_C2H_EVENT_REPORT_VAL_MASK GENMASK_ULL(60, 53)
+#define NNP_C2H_EVENT_REPORT_CHAN_VALID_MASK BIT_ULL(61)
+#define NNP_C2H_EVENT_REPORT_OBJ_VALID_MASK BIT_ULL(62)
+#define NNP_C2H_EVENT_REPORT_OBJ2_VALID_MASK BIT_ULL(63)
+
+/* NNP_IPC_C2H_OP_SYS_INFO - 1 qword */
+#define NNP_C2H_SYS_INFO_OP_MASK NNP_C2H_OP_MASK
+
+#define NNP_H2C_OP_MASK GENMASK_ULL(5, 0)
+
+/* NNP_IPC_H2C_OP_QUERY_VERSION - 1 qword */
+#define NNP_H2C_QUERY_VERSION_OP_MASK NNP_H2C_OP_MASK
+
+/* NNP_IPC_H2C_OP_SETUP_CRASH_DUMP - 2 qwords */
+/* qword 0: */
+#define NNP_H2C_SETUP_CRASH_DUMP_QW0_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_SETUP_CRASH_DUMP_QW0_DMA_ADDR_MASK GENMASK_ULL(63, 19)
+/* qword 1: physical address of BAR2 */
+
+/* NNP_IPC_H2C_OP_SETUP_SYS_INFO_PAGE - 1 qword */
+#define NNP_H2C_SETUP_SYS_INFO_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_SETUP_SYS_INFO_NPAGES GENMASK_ULL(15, 6)
+#define NNP_H2C_SETUP_SYS_INFO_DMA_ADDR_MASK GENMASK_ULL(63, 19)
+
+/* NNP_IPC_H2C_OP_CHANNEL_OP - 1 qword */
+#define NNP_H2C_CHANNEL_OP_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_CHANNEL_OP_CHAN_ID_MASK GENMASK_ULL(15, 6)
+#define NNP_H2C_CHANNEL_OP_DESTROY_MASK BIT_ULL(16)
+#define NNP_H2C_CHANNEL_OP_PRIV_MASK BIT_ULL(31)
+#define NNP_H2C_CHANNEL_OP_UID_MASK GENMASK_ULL(63, 32)
+
+/* NNP_IPC_H2C_OP_CHANNEL_RB_OP - 1 qword */
+#define NNP_H2C_CHANNEL_RB_OP_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_CHANNEL_RB_OP_CHAN_ID_MASK GENMASK_ULL(15, 6)
+#define NNP_H2C_CHANNEL_RB_OP_H2C_MASK BIT_ULL(16)
+#define NNP_H2C_CHANNEL_RB_OP_ID_MASK BIT_ULL(17)
+#define NNP_H2C_CHANNEL_RB_OP_DESTROY_MASK BIT_ULL(18)
+#define NNP_H2C_CHANNEL_RB_OP_HOST_PFN_MASK GENMASK_ULL(63, 19)
+
+/* NNP_IPC_H2C_OP_CHANNEL_HOSTRES_OP - 2 qwords */
+/* qword 0: */
+#define NNP_H2C_CHANNEL_HOSTRES_QW0_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_CHANNEL_HOSTRES_QW0_CHAN_ID_MASK GENMASK_ULL(15, 6)
+#define NNP_H2C_CHANNEL_HOSTRES_QW0_ID_MASK GENMASK_ULL(31, 16)
+#define NNP_H2C_CHANNEL_HOSTRES_QW0_UNMAP_MASK BIT_ULL(32)
+/* qword 1: */
+#define NNP_H2C_CHANNEL_HOSTRES_QW1_HOST_PFN_MASK GENMASK_ULL(44, 0)
+
+/* NNP_IPC_H2C_OP_CLOCK_STAMP - 2 qwords */
+/* qword 0: */
+#define NNP_H2C_CLOCK_STAMP_QW0_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_CLOCK_STAMP_QW0_TYPE_MASK GENMASK_ULL(63, 8)
+/* qword 1: clock stamp value */
+
+/*
+ * IPC messages opcodes and related utility macros
+ */
+#define H2C_OPCODE_NAME(name) NNP_IPC_H2C_OP_ ## name
+#define H2C_OPCODE_NAME_STR(name) #name
+#define C2H_OPCODE_NAME(name) NNP_IPC_C2H_OP_ ## name
+#define C2H_OPCODE_NAME_STR(name) #name
+
+/*
+ * Define Host-to-card opcodes (valid range is 0 - 31)
+ */
+enum nnp_h2c_opcodes {
+ H2C_OPCODE_NAME(QUERY_VERSION) = 0,
+ H2C_OPCODE_NAME(CLOCK_STAMP) = 2,
+ H2C_OPCODE_NAME(SETUP_CRASH_DUMP) = 6,
+ H2C_OPCODE_NAME(SETUP_SYS_INFO_PAGE) = 7,
+ H2C_OPCODE_NAME(CHANNEL_OP) = 22,
+ H2C_OPCODE_NAME(CHANNEL_RB_OP) = 23,
+ H2C_OPCODE_NAME(CHANNEL_HOSTRES_OP) = 24,
+
+ H2C_OPCODE_NAME(BIOS_PROTOCOL) = 31,
+ H2C_OPCODE_NAME(LAST) = NNP_IPC_H2C_OP_BIOS_PROTOCOL,
+};
+
+/*
+ * Define Card-to-host opcodes
+ */
+enum nnp_c2h_opcodes {
+ NNP_IPC_C2H_OP_QUERY_VERSION_REPLY = 0,
+ NNP_IPC_C2H_OP_QUERY_VERSION_REPLY2 = 1,
+ NNP_IPC_C2H_OP_QUERY_VERSION_REPLY3 = 2,
+ NNP_IPC_C2H_OP_EVENT_REPORT = 4,
+ NNP_IPC_C2H_OP_SYS_INFO = 11,
+
+ NNP_IPC_C2H_OP_BIOS_PROTOCOL = 31,
+ NNP_IPC_C2H_OPCODE_LAST = NNP_IPC_C2H_OP_BIOS_PROTOCOL,
+};
+
+/*
+ * IPC messages protocol between the host driver and card BIOS
+ */
+
+enum nnp_bios_c2h_msg_types {
+ NNP_IPC_C2H_TYPE_BIOS_VERSION = 0x1,
+};
+
+enum nnp_bios_h2c_msg_types {
+ NNP_IPC_H2C_TYPE_BOOT_IMAGE_READY = 0x10,
+ NNP_IPC_H2C_TYPE_SYSTEM_INFO_REQ = 0x11,
+};
+
+/* NNP_IPC_C2H_OP_BIOS_PROTOCOL - 1 qword */
+#define NNP_C2H_BIOS_PROTOCOL_OP_MASK NNP_C2H_OP_MASK
+/* bios message type */
+#define NNP_C2H_BIOS_PROTOCOL_TYPE_MASK GENMASK_ULL(15, 8)
+/* message size in bytes */
+#define NNP_C2H_BIOS_PROTOCOL_SIZE_MASK GENMASK_ULL(31, 16)
+
+/* BIOS Revision Identification Specification, Rev. 2.0, 01/30/2015 */
+struct nnp_c2h_bios_version {
+ __le16 board_id[7];
+ __le16 board_rev;
+ __le16 dot1;
+ __le16 board_ext[3];
+ __le16 dot2;
+ __le16 version_major[4];
+ __le16 dot3;
+ __le16 build_type;
+ __le16 version_minor[2];
+ __le16 dot4;
+ __le16 time_stamp[10];
+ __le16 null_terminator;
+} __packed;
+
+struct nnp_c2h_bios_fw_ver_ack_data {
+ __le16 code_minor;
+ __le16 code_major;
+ __le16 code_build_no;
+ __le16 code_hot_fix;
+ __le16 rcvyminor;
+ __le16 rcvymajor;
+ __le16 rcvybuildno;
+ __le16 rcvy_hot_fix;
+ __le16 fitc_minor;
+ __le16 fitc_major;
+ __le16 fitcbuildno;
+ __le16 fitc_hot_fix;
+} __packed;
+
+struct nnp_c2h_fw_version {
+ __le16 major;
+ __le16 minor;
+ __le16 hotfix;
+ __le16 build;
+} __packed;
+
+struct nnp_c2h_cpu_info {
+ __le32 cpu_family; /* CPU Family: 0x000906D0 */
+ __u8 cpu_stepping; /* CPU Stepping */
+ __u8 cpu_sku; /* CPU SKU */
+ __le16 cpu_did; /* range 0x4580-0x45FF */
+ __le16 cpu_core_count; /* Number of enabled cores */
+ __le16 cpu_thread_count; /* Number of threads */
+} __packed;
+
+struct nnp_c2h_ice_info {
+ __le16 ice_count;
+ __le32 ice_available_mask;
+} __packed;
+
+struct nnp_c2h_system_info {
+ __u8 version; /* NNP_SYSTEM_INFO structure version */
+ __le16 board_id; /* Board identification- for RVP = 0x25 */
+ __u8 fab_id; /* Board Revision identification */
+ __u8 bom_id; /* Board Bill Of Material identification */
+ __u8 platform_type; /* For RVP= 0x2, M.2 = 0x3 */
+ __u8 platform_flavor; /* For NNP = 0x5- Embedded */
+ struct nnp_c2h_cpu_info cpu_info; /* CPU Information */
+ struct nnp_c2h_ice_info ice_info; /* ICE Information */
+ struct nnp_c2h_bios_version bios_ver; /* BIOS version string */
+ struct nnp_c2h_bios_fw_ver_ack_data csme_version;
+ struct nnp_c2h_fw_version pmc_version;
+} __packed;
+
+/* NNP_H2C_BOOT_IMAGE_READY command - 3 qwords */
+/* qword 0: */
+/* op should be set to NNP_IPC_H2C_OP_BIOS_PROTOCOL */
+#define NNP_H2C_BOOT_IMAGE_READY_QW0_OP_MASK NNP_H2C_OP_MASK
+/* bios message type = NNP_IPC_H2C_TYPE_BOOT_IMAGE_READY */
+#define NNP_H2C_BOOT_IMAGE_READY_QW0_TYPE_MASK GENMASK_ULL(15, 8)
+/* message size in bytes = 2*sizeof(u64) */
+#define NNP_H2C_BOOT_IMAGE_READY_QW0_SIZE_MASK GENMASK_ULL(31, 16)
+/* qword 1: page table describtor address */
+/* qword 2: */
+#define NNP_H2C_BOOT_IMAGE_READY_QW2_DESC_SIZE_MASK GENMASK_ULL(31, 0)
+#define NNP_H2C_BOOT_IMAGE_READY_QW2_IMAGE_SIZE_MASK GENMASK_ULL(63, 32)
+
+/* NNP_H2C_BIOS_SYS_INFO_REQ command - 3 qwords */
+/* qword 0: */
+/* op should be set to NNP_IPC_H2C_OP_BIOS_PROTOCOL */
+#define NNP_H2C_BIOS_SYS_INFO_REQ_QW0_OP_MASK NNP_H2C_OP_MASK
+/* bios message type = NNP_IPC_H2C_TYPE_SYSTEM_INFO_REQ */
+#define NNP_H2C_BIOS_SYS_INFO_REQ_QW0_TYPE_MASK GENMASK_ULL(15, 8)
+/* message size in bytes = 2*sizeof(u64) */
+#define NNP_H2C_BIOS_SYS_INFO_REQ_QW0_SIZE_MASK GENMASK_ULL(31, 16)
+/* qword 1: allocate sys info page address */
+/* qword 2: */
+#define NNP_H2C_BIOS_SYS_INFO_REQ_QW2_SIZE_MASK GENMASK_ULL(31, 0)
+
+#define NNP_BIOS_VERSION_LEN \
+ (sizeof(struct nnp_c2h_bios_version) / sizeof(__le16))
+#define NNP_BOARD_NAME_LEN 72
+#define NNP_IMAGE_VERSION_LEN 128
+#define NNP_PRD_SERIAL_LEN 16
+#define NNP_PART_NUM_LEN 12
+
+struct nnp_sys_info {
+ __le32 ice_mask;
+ char bios_version[NNP_BIOS_VERSION_LEN];
+ char board_name[NNP_BOARD_NAME_LEN];
+ char image_version[NNP_IMAGE_VERSION_LEN];
+ char prd_serial[NNP_PRD_SERIAL_LEN];
+ char brd_part_no[NNP_PART_NUM_LEN];
+ __le16 fpga_rev;
+ __le64 total_unprotected_memory;
+ __le64 total_ecc_memory;
+ __u8 stepping;
+} __packed;
+
+/*
+ * Define header structure for all "channel" message protocols.
+ * This protocol defines communication between host UMD and card.
+ */
+#define NNP_H2C_CHAN_MSG_OP_MASK NNP_H2C_OP_MASK
+#define NNP_H2C_CHAN_MSG_CHAN_ID_MASK GENMASK_ULL(15, 6)
+
+#define NNP_C2H_CHAN_MSG_OP_MASK NNP_C2H_OP_MASK
+#define NNP_C2H_CHAN_MSG_CHAN_ID_MASK GENMASK_ULL(15, 6)
+
+#endif /* of _IPC_PROTOCOL_H */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 05/15] misc: nnpi: Manage host memory resources
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (3 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 04/15] misc: nnpi: Define host/card ipc protocol Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 06/15] misc: nnpi: Allow usermode to manage host resources Guy Zadicario
` (10 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Provide interface for creating a "host resource" - a memory object
which can be mapped to a dma address space of one or more NNP-I
devices. These host resource objects manage memory blocks from which
big chunks of data, such as inference tensors, the device's embedded OS
and telemtry data are DMA'ed to and from the NNP-I device.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
Documentation/ABI/testing/sysfs-driver-intel_nnpi | 5 +
drivers/misc/intel-nnpi/Makefile | 2 +-
drivers/misc/intel-nnpi/hostres.c | 627 ++++++++++++++++++++++
drivers/misc/intel-nnpi/hostres.h | 167 ++++++
4 files changed, 800 insertions(+), 1 deletion(-)
create mode 100644 Documentation/ABI/testing/sysfs-driver-intel_nnpi
create mode 100644 drivers/misc/intel-nnpi/hostres.c
create mode 100644 drivers/misc/intel-nnpi/hostres.h
diff --git a/Documentation/ABI/testing/sysfs-driver-intel_nnpi b/Documentation/ABI/testing/sysfs-driver-intel_nnpi
new file mode 100644
index 0000000..b09880f
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-intel_nnpi
@@ -0,0 +1,5 @@
+What: /sys/class/nnpi_host/nnpi_host/total_hostres_size
+Date: Mar 2021
+Kernelversion: 5.13
+Contact: guy.zadicario@intel.com
+Description: Total size in bytes of all allocated NNP-I host resources.
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index 43f09c0..3713d51 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -5,7 +5,7 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
-intel_nnpi-y := device.o msg_scheduler.o
+intel_nnpi-y := device.o msg_scheduler.o hostres.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/hostres.c b/drivers/misc/intel-nnpi/hostres.c
new file mode 100644
index 0000000..e0d71c0
--- /dev/null
+++ b/drivers/misc/intel-nnpi/hostres.c
@@ -0,0 +1,627 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitfield.h>
+#include <linux/err.h>
+#include <linux/mutex.h>
+#include <linux/pagemap.h>
+#include <linux/pfn.h>
+#include <linux/printk.h>
+#include <linux/sched/mm.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include "hostres.h"
+#include "ipc_protocol.h"
+
+/**
+ * struct host_resource - structure for host memory resource object
+ * @ref: kref for that host resource object
+ * @size: size of the memory resource, in bytes
+ * @devices: list of devices this resource is mapped to (list of nnpdev_mapping)
+ * @lock: protects @devices
+ * @dir: DMA direction mask possible for this resource, when mapped to device.
+ * @pinned_mm: mm object used to pin the user allocated resource memory. NULL
+ * if the resource was not allocated by user-space.
+ * @vptr: virtual pointer to the resource memory if allocated by
+ * nnp_hostres_alloc(). NULL otherwise.
+ * @start_offset: holds the offset within the first pinned page where resource
+ * memory starts (relevant only when @pinned_mm is not NULL).
+ * @pages: array of resource memory pages.
+ * @n_pages: size of pages array.
+ */
+struct host_resource {
+ struct kref ref;
+ size_t size;
+ struct list_head devices;
+ spinlock_t lock;
+ enum dma_data_direction dir;
+
+ struct mm_struct *pinned_mm;
+ void *vptr;
+ unsigned int start_offset;
+
+ struct page **pages;
+ unsigned int n_pages;
+};
+
+/**
+ * struct nnpdev_mapping - mapping information of host resource to one device
+ * @ref: kref for that mapping object
+ * @res: pointer to the host resource
+ * @dev: the device the resource is mapped to
+ * @sgt: scatter table of host resource pages in memory
+ * @dma_chain_sgt: sg_table of dma_chain blocks (see description below).
+ * @dma_chain_order: order used to allocate scatterlist of @dma_chain_sgt.
+ * @node: list head to attach this object to a list of mappings
+ *
+ * This structure holds mapping information of one host resource to one
+ * NNP-I device. @sgt is the sg_table describes the DMA addresses of the
+ * resource chunks.
+ *
+ * When mapping a host memory resource for NNP-I device access, we need to send
+ * the DMA page table of the resource to the device. The device uses this page
+ * table when programming its DMA engine to read/write the host resource.
+ *
+ * The format of that page table is a chain of continuous DMA buffers, each
+ * starts with a 24 bytes header (&struct dma_chain_header) followed by 8 bytes
+ * entries, each describe a continuous block of the resource
+ * (&struct nnp_dma_chain_entry).
+ *
+ * The header of the chain has a pointer to the next buffer in the chain for
+ * the case where multiple DMA blocks are required to describe the
+ * entire resource. The address of the first block in the chain is sent to
+ * the device, which then fetches the entire chain when the resource is
+ * mapped. @dma_chain_sgt is an sg_table of memory mapped to the device and
+ * initialized with the resource page table in the above described format.
+ */
+struct nnpdev_mapping {
+ struct kref ref;
+ struct host_resource *res;
+ struct device *dev;
+ struct sg_table *sgt;
+ struct sg_table dma_chain_sgt;
+ unsigned int dma_chain_order;
+ struct list_head node;
+};
+
+/*
+ * Since host resources are pinned for their entire lifetime, it
+ * is useful to monitor the total size of NNP-I host resources
+ * allocated in the system.
+ */
+static size_t total_hostres_size;
+static DEFINE_MUTEX(total_size_mutex);
+
+/* Destroys host resource, when all references to it are released */
+static void release_hostres(struct kref *kref)
+{
+ struct host_resource *r = container_of(kref, struct host_resource, ref);
+
+ if (r->pinned_mm) {
+ unpin_user_pages(r->pages, r->n_pages);
+ account_locked_vm(r->pinned_mm, r->n_pages, false);
+ mmdrop(r->pinned_mm);
+ } else {
+ vfree(r->vptr);
+ }
+
+ kvfree(r->pages);
+ mutex_lock(&total_size_mutex);
+ total_hostres_size -= r->size;
+ mutex_unlock(&total_size_mutex);
+ kfree(r);
+}
+
+void nnp_hostres_get(struct host_resource *res)
+{
+ kref_get(&res->ref);
+};
+
+void nnp_hostres_put(struct host_resource *res)
+{
+ kref_put(&res->ref, release_hostres);
+}
+
+/* Really destroys mapping to device, when refcount is zero */
+static void release_mapping(struct kref *kref)
+{
+ struct nnpdev_mapping *m = container_of(kref, struct nnpdev_mapping, ref);
+
+ spin_lock(&m->res->lock);
+ list_del(&m->node);
+ spin_unlock(&m->res->lock);
+
+ dma_unmap_sgtable(m->dev, &m->dma_chain_sgt, DMA_TO_DEVICE, 0);
+ sgl_free_order(m->dma_chain_sgt.sgl, m->dma_chain_order);
+
+ dma_unmap_sg(m->dev, m->sgt->sgl, m->sgt->orig_nents, m->res->dir);
+ sg_free_table(m->sgt);
+ kfree(m->sgt);
+
+ nnp_hostres_put(m->res);
+
+ kfree(m);
+}
+
+static struct host_resource *alloc_hostres(size_t size, enum dma_data_direction dir)
+{
+ struct host_resource *r;
+
+ r = kzalloc(sizeof(*r), GFP_KERNEL);
+ if (!r)
+ return r;
+
+ kref_init(&r->ref);
+ spin_lock_init(&r->lock);
+ r->dir = dir;
+ r->size = size;
+ INIT_LIST_HEAD(&r->devices);
+
+ return r;
+}
+
+struct host_resource *nnp_hostres_alloc(size_t size, enum dma_data_direction dir)
+{
+ struct host_resource *r;
+ unsigned int i;
+ char *p;
+
+ if (size == 0 || dir == DMA_NONE)
+ return ERR_PTR(-EINVAL);
+
+ r = alloc_hostres(size, dir);
+ if (!r)
+ return ERR_PTR(-ENOMEM);
+
+ r->n_pages = PFN_UP(size);
+ r->vptr = vmalloc(r->n_pages * PAGE_SIZE);
+ if (!r->vptr)
+ goto free_res;
+
+ r->pages = kvmalloc_array(r->n_pages, sizeof(struct page *), GFP_KERNEL);
+ if (!r->pages)
+ goto free_vptr;
+
+ for (i = 0, p = r->vptr; i < r->n_pages; i++, p += PAGE_SIZE) {
+ r->pages[i] = vmalloc_to_page(p);
+ if (!r->pages[i])
+ goto free_pages;
+ }
+
+ mutex_lock(&total_size_mutex);
+ total_hostres_size += size;
+ mutex_unlock(&total_size_mutex);
+
+ return r;
+
+free_pages:
+ kvfree(r->pages);
+free_vptr:
+ vfree(r->vptr);
+free_res:
+ kfree(r);
+ return ERR_PTR(-ENOMEM);
+}
+
+struct host_resource *nnp_hostres_from_usermem(void __user *user_ptr, size_t size,
+ enum dma_data_direction dir)
+{
+ /*
+ * user_ptr is not being accessed, it is only used as parameter to
+ * pin_user_pages(), so it is OK to remove the __user annotation.
+ */
+ uintptr_t user_addr = (__force uintptr_t)user_ptr;
+ struct host_resource *r;
+ int err;
+ int gup_flags = 0;
+ int n, pinned;
+
+ if (size == 0 || dir == DMA_NONE)
+ return ERR_PTR(-EINVAL);
+
+ /* Restrict for 4 byte alignment */
+ if (user_addr & 0x3)
+ return ERR_PTR(-EINVAL);
+
+ if (!access_ok(user_ptr, size))
+ return ERR_PTR(-EFAULT);
+
+ r = alloc_hostres(size, dir);
+ if (!r)
+ return ERR_PTR(-ENOMEM);
+
+ r->start_offset = offset_in_page(user_addr);
+ user_addr &= PAGE_MASK;
+
+ r->n_pages = PFN_UP(size + r->start_offset);
+ r->pages = kvmalloc_array(r->n_pages, sizeof(struct page *), GFP_KERNEL);
+ if (!r->pages) {
+ err = -ENOMEM;
+ goto free_res;
+ }
+
+ err = account_locked_vm(current->mm, r->n_pages, true);
+ if (err)
+ goto free_pages;
+
+ if (nnp_hostres_is_input(r))
+ gup_flags = FOLL_WRITE;
+
+ /*
+ * The host resource is being re-used for multiple DMA
+ * transfers for streaming data into the device.
+ * In most situations will live long term.
+ */
+ gup_flags |= FOLL_LONGTERM;
+
+ for (pinned = 0; pinned < r->n_pages; pinned += n) {
+ n = pin_user_pages(user_addr + pinned * PAGE_SIZE,
+ r->n_pages - pinned, gup_flags,
+ &r->pages[pinned], NULL);
+ if (n < 0) {
+ err = -ENOMEM;
+ goto unaccount;
+ }
+ }
+
+ r->pinned_mm = current->mm;
+ mmgrab(r->pinned_mm);
+
+ mutex_lock(&total_size_mutex);
+ total_hostres_size += size;
+ mutex_unlock(&total_size_mutex);
+
+ return r;
+
+unaccount:
+ account_locked_vm(current->mm, r->n_pages, false);
+ unpin_user_pages(r->pages, pinned);
+free_pages:
+ kvfree(r->pages);
+free_res:
+ kfree(r);
+ return ERR_PTR(err);
+}
+
+/* Finds mapping by device and increase its refcount. NULL if not found */
+static struct nnpdev_mapping *get_mapping_for_dev(struct host_resource *res,
+ struct device *dev)
+{
+ struct nnpdev_mapping *m;
+
+ spin_lock(&res->lock);
+
+ list_for_each_entry(m, &res->devices, node) {
+ if (m->dev == dev) {
+ kref_get(&m->ref);
+ goto out;
+ }
+ }
+
+ m = NULL;
+out:
+ spin_unlock(&res->lock);
+ return m;
+}
+
+static bool entry_valid(struct scatterlist *sgl, u64 ipc_entry)
+{
+ unsigned long long dma_pfn;
+ unsigned long n_pages;
+
+ dma_pfn = FIELD_GET(DMA_CHAIN_ENTRY_PFN_MASK, ipc_entry);
+ if (NNP_IPC_DMA_PFN_TO_ADDR(dma_pfn) != sg_dma_address(sgl))
+ return false;
+
+ n_pages = FIELD_GET(DMA_CHAIN_ENTRY_NPAGES_MASK, ipc_entry);
+ if (n_pages != DIV_ROUND_UP(sg_dma_len(sgl), NNP_PAGE_SIZE))
+ return false;
+
+ return true;
+}
+
+/**
+ * build_ipc_dma_chain_array() - builds page list of the resource for IPC usage
+ * @m: pointer to device mapping info struct
+ * @use_one_entry: if true will generate all page table in one continuous
+ * DMA chunk. otherwise a chain of blocks will be used
+ * each of one page size.
+ * @start_offset: offset in first mapped page where resource memory starts,
+ *
+ * This function allocates scatterlist, map it to device and populate it with
+ * page table of the device mapped resource in format suitable to be used
+ * in the IPC protocol for sending the resource page table to the card.
+ * The format of the page table is described in the documentation of &struct
+ * nnpdev_mapping.
+ *
+ * Return: 0 on success, error value otherwise
+ */
+static int build_ipc_dma_chain_array(struct nnpdev_mapping *m, bool use_one_entry,
+ unsigned int start_offset)
+{
+ unsigned int i, k = 0;
+ u64 *p = NULL;
+ u64 e;
+ unsigned long long dma_addr, dma_pfn, size;
+ struct nnp_dma_chain_header *h;
+ struct scatterlist *sg, *map_sg;
+ struct scatterlist *chain_sg;
+ unsigned long n_pages;
+ unsigned int chain_size;
+ unsigned int chain_order;
+ unsigned int chain_nents;
+ unsigned int nents_per_entry;
+ unsigned int start_off = start_offset;
+ int rc;
+
+ if (use_one_entry) {
+ /*
+ * Allocate enough pages in one chunk that will fit
+ * the header and nnp_dma_chain_entry for all the sg_table
+ * entries.
+ */
+ nents_per_entry = m->sgt->nents;
+ chain_size = sizeof(struct nnp_dma_chain_header) +
+ m->sgt->nents * DMA_CHAIN_ENTRY_SIZE;
+ chain_order = get_order(chain_size);
+ } else {
+ /*
+ * Calc number of one page DMA buffers needed to hold the
+ * entire page table.
+ * NENTS_PER_PAGE is how much DMA chain entries fits
+ * in a single page following the chain header, must be at
+ * positive.
+ */
+ nents_per_entry = NENTS_PER_PAGE;
+ chain_size = DIV_ROUND_UP(m->sgt->nents, nents_per_entry) *
+ NNP_PAGE_SIZE;
+ chain_order = 0;
+ }
+
+ chain_sg = sgl_alloc_order(chain_size, chain_order, false, GFP_KERNEL,
+ &chain_nents);
+ if (!chain_sg)
+ return -ENOMEM;
+
+ m->dma_chain_sgt.sgl = chain_sg;
+ m->dma_chain_sgt.nents = chain_nents;
+ m->dma_chain_sgt.orig_nents = chain_nents;
+ m->dma_chain_order = chain_order;
+ rc = dma_map_sgtable(m->dev, &m->dma_chain_sgt, DMA_TO_DEVICE, 0);
+ if (rc)
+ goto free_chain_sg;
+
+ /* Initialize chain entry blocks */
+ map_sg = m->sgt->sgl;
+ for_each_sg(chain_sg, sg, chain_nents, i) {
+ /*
+ * Check that the allocated DMA address fits in IPC protocol.
+ * In the protocol, DMA addresses are sent as 4K page numbers
+ * and must fit in 45 bits.
+ * Meaning, if the DMA address is larger than 57 bits it will
+ * not fit.
+ */
+ if (sg_dma_address(sg) > NNP_IPC_DMA_MAX_ADDR)
+ goto unmap_chain_sg;
+
+ /* h: points to the header of current block */
+ h = sg_virt(sg);
+
+ /* p: points to current chunk entry in block */
+ p = (u64 *)(h + 1);
+
+ size = 0;
+ for (k = 0; k < nents_per_entry && map_sg; ++k) {
+ /*
+ * Build entry with DMA address as page number and
+ * size in pages
+ */
+ dma_addr = sg_dma_address(map_sg);
+ dma_pfn = NNP_IPC_DMA_ADDR_TO_PFN(dma_addr);
+ n_pages = DIV_ROUND_UP(sg_dma_len(map_sg), NNP_PAGE_SIZE);
+
+ e = FIELD_PREP(DMA_CHAIN_ENTRY_PFN_MASK, dma_pfn);
+ e |= FIELD_PREP(DMA_CHAIN_ENTRY_NPAGES_MASK, n_pages);
+
+ /*
+ * Check that packed entry matches the DMA chunk.
+ * (Will fail if either dma_pfn or n_pages fields overflows)
+ */
+ if (!entry_valid(map_sg, e))
+ goto unmap_chain_sg;
+
+ /* Fill entry value (should be 64-bit little-endian) */
+ p[k] = cpu_to_le64(e);
+
+ size += sg_dma_len(map_sg);
+
+ map_sg = sg_next(map_sg);
+ }
+
+ /* Initialize block header and link to next block */
+ h->total_nents = cpu_to_le32(m->sgt->nents);
+ h->start_offset = cpu_to_le32(start_off);
+ h->size = cpu_to_le64(size);
+ if (sg_next(sg))
+ h->dma_next = cpu_to_le64(sg_dma_address(sg_next(sg)));
+ else
+ h->dma_next = 0;
+ start_off = 0;
+ }
+
+ return 0;
+
+unmap_chain_sg:
+ dma_unmap_sgtable(m->dev, &m->dma_chain_sgt, DMA_TO_DEVICE, 0);
+free_chain_sg:
+ sgl_free_order(chain_sg, chain_order);
+ memset(&m->dma_chain_sgt, 0, sizeof(m->dma_chain_sgt));
+ return -ENOMEM;
+}
+
+struct nnpdev_mapping *nnp_hostres_map_device(struct host_resource *res,
+ struct nnp_device *nnpdev,
+ bool use_one_entry,
+ dma_addr_t *page_list,
+ u32 *total_chunks)
+{
+ int ret;
+ struct nnpdev_mapping *m;
+ struct scatterlist *sge;
+
+ if (!res || !nnpdev || !page_list)
+ return ERR_PTR(-EINVAL);
+
+ /* Check if already mapped for the device */
+ m = get_mapping_for_dev(res, nnpdev->dev);
+ if (m)
+ goto done;
+
+ nnp_hostres_get(res);
+
+ m = kmalloc(sizeof(*m), GFP_KERNEL);
+ if (!m) {
+ ret = -ENOMEM;
+ goto put_resource;
+ }
+
+ kref_init(&m->ref);
+
+ m->dev = nnpdev->dev;
+ m->res = res;
+
+ m->sgt = kmalloc(sizeof(*m->sgt), GFP_KERNEL);
+ if (!m->sgt) {
+ ret = -ENOMEM;
+ goto free_mapping;
+ }
+
+ sge = __sg_alloc_table_from_pages(m->sgt, res->pages, res->n_pages, 0,
+ res->size + res->start_offset,
+ NNP_MAX_CHUNK_SIZE, NULL, 0, GFP_KERNEL);
+ if (IS_ERR(sge)) {
+ ret = PTR_ERR(sge);
+ goto free_sgt_struct;
+ }
+
+ ret = dma_map_sg(m->dev, m->sgt->sgl, m->sgt->orig_nents, res->dir);
+ if (ret <= 0) {
+ /* dma_map_sg returns 0 on error with no error value */
+ ret = -ENOMEM;
+ goto free_sgt;
+ }
+
+ m->sgt->nents = ret;
+
+ ret = build_ipc_dma_chain_array(m, use_one_entry, res->start_offset);
+ if (ret < 0)
+ goto unmap;
+
+ spin_lock(&res->lock);
+ list_add(&m->node, &res->devices);
+ spin_unlock(&res->lock);
+
+done:
+ *page_list = sg_dma_address(m->dma_chain_sgt.sgl);
+ if (total_chunks)
+ *total_chunks = m->sgt->nents;
+
+ return m;
+
+unmap:
+ dma_unmap_sg(m->dev, m->sgt->sgl, m->sgt->orig_nents, res->dir);
+free_sgt:
+ sg_free_table(m->sgt);
+free_sgt_struct:
+ kfree(m->sgt);
+free_mapping:
+ kfree(m);
+put_resource:
+ nnp_hostres_put(res);
+ return ERR_PTR(ret);
+}
+
+void nnp_hostres_unmap_device(struct nnpdev_mapping *mapping)
+{
+ kref_put(&mapping->ref, release_mapping);
+}
+
+int nnp_hostres_user_lock(struct host_resource *res)
+{
+ struct nnpdev_mapping *m;
+
+ spin_lock(&res->lock);
+ list_for_each_entry(m, &res->devices, node)
+ dma_sync_sg_for_cpu(m->dev, m->sgt->sgl, m->sgt->orig_nents, res->dir);
+ spin_unlock(&res->lock);
+
+ return 0;
+}
+
+int nnp_hostres_user_unlock(struct host_resource *res)
+{
+ struct nnpdev_mapping *m;
+
+ spin_lock(&res->lock);
+ list_for_each_entry(m, &res->devices, node)
+ dma_sync_sg_for_device(m->dev, m->sgt->sgl, m->sgt->orig_nents,
+ res->dir);
+ spin_unlock(&res->lock);
+
+ return 0;
+}
+
+bool nnp_hostres_is_input(struct host_resource *res)
+{
+ return res->dir == DMA_TO_DEVICE || res->dir == DMA_BIDIRECTIONAL;
+}
+
+bool nnp_hostres_is_output(struct host_resource *res)
+{
+ return res->dir == DMA_FROM_DEVICE || res->dir == DMA_BIDIRECTIONAL;
+}
+
+size_t nnp_hostres_size(struct host_resource *res)
+{
+ return res->size;
+}
+
+void *nnp_hostres_vptr(struct host_resource *res)
+{
+ return res->vptr;
+}
+
+static ssize_t total_hostres_size_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ ssize_t ret;
+
+ mutex_lock(&total_size_mutex);
+ ret = sysfs_emit(buf, "%zu\n", total_hostres_size);
+ mutex_unlock(&total_size_mutex);
+
+ return ret;
+}
+static DEVICE_ATTR_RO(total_hostres_size);
+
+static struct attribute *nnp_host_attrs[] = {
+ &dev_attr_total_hostres_size.attr,
+ NULL
+};
+
+static struct attribute_group nnp_host_attrs_grp = {
+ .attrs = nnp_host_attrs,
+};
+
+int nnp_hostres_init_sysfs(struct device *dev)
+{
+ return sysfs_create_group(&dev->kobj, &nnp_host_attrs_grp);
+}
+
+void nnp_hostres_fini_sysfs(struct device *dev)
+{
+ sysfs_remove_group(&dev->kobj, &nnp_host_attrs_grp);
+}
diff --git a/drivers/misc/intel-nnpi/hostres.h b/drivers/misc/intel-nnpi/hostres.h
new file mode 100644
index 0000000..6397c73
--- /dev/null
+++ b/drivers/misc/intel-nnpi/hostres.h
@@ -0,0 +1,167 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_HOSTRES_H
+#define _NNPDRV_HOSTRES_H
+
+#include <linux/dma-mapping.h>
+#include "device.h"
+
+/**
+ * nnp_hostres_alloc() - allocate memory and create host resource
+ * @size: Size of the host resource to be created
+ * @dir: Resource direction (read or write or both)
+ *
+ * This function allocates memory pages and provides host resource handle.
+ * The memory is mapped to kernel virtual address.
+ * The resource can be Input(read by device), Output(write by device) and both.
+ *
+ * The return handle can be used as argument to one of the other nnpi_hostres*
+ * functions for:
+ * - mapping/unmapping the resource for NNP-I device.
+ * - pointer to the allocated memory can be retrieved by nnp_hostres_vptr()
+ *
+ * The handle should be released when no longer needed by a call to
+ * nnp_hostres_put.
+ *
+ * Return: pointer to created resource or error value
+ */
+struct host_resource *nnp_hostres_alloc(size_t size, enum dma_data_direction dir);
+
+/**
+ * nnp_hostres_from_usermem() - Creates host resource from user-space memory
+ * @user_ptr: user virtual memory to pin
+ * @size: size of user buffer to pin
+ * @dir: Resource direction (read or write or both)
+ *
+ * This function pins the provided user memory and create a host resource
+ * handle managing this memory.
+ * The provided handle can be used the same as the handle created by
+ * nnp_hostres_alloc.
+ * The resource can be Input(read by device), Output(write by device) and both.
+ *
+ * The handle should be released when no longer needed by a call to
+ * nnp_hostres_put.
+ *
+ * Return: pointer to created resource or error value
+ */
+struct host_resource *nnp_hostres_from_usermem(void __user *user_ptr, size_t size,
+ enum dma_data_direction dir);
+
+/**
+ * nnp_hostres_map_device() - Maps the host resource to NNP-I device
+ * @res: handle to host resource
+ * @nnpdev: handle to nnp device struct
+ * @use_one_entry: when true will produce ipc dma chain page table descriptor
+ * of the mapping in a single concurrent dma block.
+ * otherwise a chain of multiple blocks might be generated.
+ * @page_list: returns the dma address of the ipc dma chain page table
+ * descriptor.
+ * @total_chunks: returns the total number of elements in the mapping's
+ * sg_table. Can be NULL if this info is not required.
+ *
+ * This function maps the host resource to be accessible from device
+ * and returns the dma page list of DMA addresses packed in format
+ * suitable to be used in IPC protocol to be sent to the card.
+ *
+ * The resource can be mapped to multiple devices.
+ *
+ * Return: pointer to the mapping object or error on failure.
+ */
+struct nnpdev_mapping *nnp_hostres_map_device(struct host_resource *res,
+ struct nnp_device *nnpdev,
+ bool use_one_entry,
+ dma_addr_t *page_list,
+ u32 *total_chunks);
+
+/**
+ * nnp_hostres_unmap_device() - Unmaps the host resource from NNP-I device
+ * @mapping: mapping pointer, returned from nnp_hostres_map_device
+ *
+ * This function unmaps previously mapped host resource from device.
+ */
+void nnp_hostres_unmap_device(struct nnpdev_mapping *mapping);
+
+/**
+ * nnp_hostres_user_lock() - Lock the host resource to access from userspace
+ * @res: handle to host resource
+ *
+ * This function should be called before user-space application is accessing
+ * the host resource content (either for read or write). The function
+ * invalidates or flashes the cpu caches when necessary.
+ * The function does *not* impose any synchronization between application and
+ * device accesses to the resource memory. Such synchronization is handled
+ * in user-space.
+ *
+ * Return: error on failure.
+ */
+int nnp_hostres_user_lock(struct host_resource *res);
+
+/**
+ * nnp_hostres_user_unlock() - Unlocks the host resource from userspace access
+ * @res: handle to host resource
+ *
+ * This function should be called after user-space application is finished
+ * accessing the host resource content (either for read or write). The function
+ * invalidates or flashes the cpu caches when necessary.
+ *
+ * Return: error on failure.
+ */
+int nnp_hostres_user_unlock(struct host_resource *res);
+
+/**
+ * nnp_hostres_get() - Increases refcount of the hostres
+ * @res: handle to host resource
+ *
+ * This function increases refcount of the host resource.
+ */
+void nnp_hostres_get(struct host_resource *res);
+
+/**
+ * nnp_hostres_put() - Decreases refcount of the hostres
+ * @res: handle to host resource
+ *
+ * This function decreases refcount of the host resource and destroys it
+ * when it reaches 0.
+ */
+void nnp_hostres_put(struct host_resource *res);
+
+/**
+ * nnp_hostres_is_input() - Returns if the host resource is input resource
+ * @res: handle to host resource
+ *
+ * This function returns true if the host resource can be read by device.
+ * The "input" terminology is used here since such resources are usually
+ * used as inputs to device inference network.
+ *
+ * Return: true if the reasource is readable.
+ */
+bool nnp_hostres_is_input(struct host_resource *res);
+
+/**
+ * nnp_hostres_is_output() - Returns if the host resource is output resource
+ * @res: handle to host resource
+ *
+ * This function returns true if the host resource can be modified by device.
+ * The term "output" is used here since usually such resources are used for
+ * outputs of device inference network.
+ *
+ * Return: true if the reasource is writable.
+ */
+bool nnp_hostres_is_output(struct host_resource *res);
+
+size_t nnp_hostres_size(struct host_resource *res);
+
+/**
+ * nnp_hostres_vptr() - returns the virtual pointer to the resource buffer
+ * @res: handle to host resource
+ *
+ * Return: pointer to resource data or NULL if was not allocated by
+ * nnp_hostres_alloc()
+ */
+void *nnp_hostres_vptr(struct host_resource *res);
+
+int nnp_hostres_init_sysfs(struct device *dev);
+void nnp_hostres_fini_sysfs(struct device *dev);
+
+#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 06/15] misc: nnpi: Allow usermode to manage host resources
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (4 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 05/15] misc: nnpi: Manage host memory resources Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-17 7:02 ` Dave Airlie
2021-05-13 8:57 ` [PATCH v2 07/15] misc: nnpi: Disallow host memory resource access if no NNP-I devices exist Guy Zadicario
` (9 subsequent siblings)
15 siblings, 1 reply; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Provide an IOCTL interface for creating and destroying host memory
resources through a character device (/dev/nnpi_host).
There is a single instance of this character device in the system
regardless of the number of NNP-I devices attached because it
controls host resources which may be shared between different devices.
A nnp_user object, created when an application opens this character
device, identifies the user (client) of the driver and holds a list of
all host resources allocated by that user through the opened file
descriptor.
Host memory resources created through this character device can be mapped
to device access through IOCTLs made to a different, per-device, chardev
(will be introduced on next commits).
All resources will be destroyed when the application closes the connection
or exits.
The IOCTL interface is defined in: include/uapi/misc/intel_nnpi.h
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
MAINTAINERS | 1 +
drivers/misc/intel-nnpi/Makefile | 2 +-
drivers/misc/intel-nnpi/device.c | 14 ++
drivers/misc/intel-nnpi/host_chardev.c | 346 +++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/host_chardev.h | 12 ++
drivers/misc/intel-nnpi/nnp_user.c | 131 +++++++++++++
drivers/misc/intel-nnpi/nnp_user.h | 79 ++++++++
include/uapi/misc/intel_nnpi.h | 150 ++++++++++++++
8 files changed, 734 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/intel-nnpi/host_chardev.c
create mode 100644 drivers/misc/intel-nnpi/host_chardev.h
create mode 100644 drivers/misc/intel-nnpi/nnp_user.c
create mode 100644 drivers/misc/intel-nnpi/nnp_user.h
create mode 100644 include/uapi/misc/intel_nnpi.h
diff --git a/MAINTAINERS b/MAINTAINERS
index ff0c3d7..42f3e54 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9320,6 +9320,7 @@ INTEL NNP-I PCI DRIVER
M: Guy Zadicario <guy.zadicario@intel.com>
S: Supported
F: drivers/misc/intel-nnpi/
+F: include/uapi/misc/intel_nnpi.h
INTEL P-Unit IPC DRIVER
M: Zha Qipeng <qipeng.zha@intel.com>
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index 3713d51..da2863b 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -5,7 +5,7 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
-intel_nnpi-y := device.o msg_scheduler.o hostres.o
+intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 60c5a94..0f98398 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include "device.h"
+#include "host_chardev.h"
#include "msg_scheduler.h"
static DEFINE_IDA(dev_ida);
@@ -99,6 +100,19 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
}
EXPORT_SYMBOL(nnpdev_destroy);
+static int __init nnp_init(void)
+{
+ return nnp_init_host_interface();
+}
+subsys_initcall(nnp_init);
+
+static void __exit nnp_cleanup(void)
+{
+ nnp_release_host_interface();
+ /* dev_ida is already empty here - no point calling ida_destroy */
+}
+module_exit(nnp_cleanup);
+
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Intel(R) NNPI Framework");
MODULE_AUTHOR("Intel Corporation");
diff --git a/drivers/misc/intel-nnpi/host_chardev.c b/drivers/misc/intel-nnpi/host_chardev.c
new file mode 100644
index 0000000..6048fda
--- /dev/null
+++ b/drivers/misc/intel-nnpi/host_chardev.c
@@ -0,0 +1,346 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/cdev.h>
+#include <linux/device.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/hashtable.h>
+#include <linux/idr.h>
+#include <linux/printk.h>
+#include <linux/slab.h>
+#include <uapi/misc/intel_nnpi.h>
+
+#include "device.h"
+#include "host_chardev.h"
+#include "ipc_protocol.h"
+#include "nnp_user.h"
+
+static struct cdev cdev;
+static dev_t devnum;
+static struct class *class;
+static struct device *dev;
+
+static inline int is_host_file(struct file *f);
+
+static enum dma_data_direction to_dma_dir(unsigned int nnp_dir)
+{
+ /* Ignore IOCTL_INF_RES_NETWORK */
+ switch (nnp_dir & (IOCTL_INF_RES_INPUT | IOCTL_INF_RES_OUTPUT)) {
+ case (IOCTL_INF_RES_INPUT | IOCTL_INF_RES_OUTPUT):
+ return DMA_BIDIRECTIONAL;
+ case IOCTL_INF_RES_INPUT:
+ return DMA_TO_DEVICE;
+ case IOCTL_INF_RES_OUTPUT:
+ return DMA_FROM_DEVICE;
+ default:
+ break;
+ }
+
+ return DMA_NONE;
+}
+
+static long create_hostres(struct nnp_user_info *user_info, void __user *arg,
+ unsigned int size)
+{
+ int ret;
+ struct nnpdrv_ioctl_create_hostres req;
+ struct host_resource *hostres;
+ struct user_hostres *user_hostres_entry;
+ void __user *uptr;
+ unsigned int io_size = sizeof(req);
+
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ if (req.usage_flags & ~IOCTL_RES_USAGE_VALID_MASK)
+ return -EINVAL;
+
+ uptr = u64_to_user_ptr(req.user_ptr);
+ hostres = nnp_hostres_from_usermem(uptr, req.size,
+ to_dma_dir(req.usage_flags));
+
+ if (IS_ERR(hostres))
+ return PTR_ERR(hostres);
+
+ ret = nnp_user_add_hostres(user_info, hostres, &user_hostres_entry);
+ if (ret < 0) {
+ nnp_hostres_put(hostres);
+ return ret;
+ }
+
+ req.size = nnp_hostres_size(hostres);
+
+ /*
+ * The created user_hostres_entry holds refcount to the resource,
+ * no need to keep another one here.
+ */
+ nnp_hostres_put(hostres);
+
+ req.user_handle = user_hostres_entry->user_handle;
+ if (copy_to_user(arg, &req, io_size)) {
+ ret = -EFAULT;
+ goto destroy_hostres_entry;
+ }
+
+ return 0;
+
+destroy_hostres_entry:
+ nnp_user_remove_hostres(user_hostres_entry);
+
+ return ret;
+}
+
+static long destroy_hostres(struct nnp_user_info *user_info, void __user *arg,
+ unsigned int size)
+{
+ struct nnpdrv_ioctl_destroy_hostres destroy_args;
+ struct user_hostres *user_hostres_entry;
+ unsigned int io_size = sizeof(destroy_args);
+ int ret = 0;
+
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&destroy_args, arg, io_size))
+ return -EFAULT;
+
+ /* errno must be cleared on entry */
+ if (destroy_args.o_errno)
+ return -EINVAL;
+
+ mutex_lock(&user_info->mutex);
+ user_hostres_entry = idr_find(&user_info->idr, destroy_args.user_handle);
+ if (user_hostres_entry) {
+ nnp_user_remove_hostres_locked(user_hostres_entry);
+ } else {
+ destroy_args.o_errno = NNPER_NO_SUCH_RESOURCE;
+ if (copy_to_user(arg, &destroy_args, io_size))
+ ret = -EFAULT;
+ }
+
+ mutex_unlock(&user_info->mutex);
+ return ret;
+}
+
+static long lock_hostres(struct nnp_user_info *user_info, void __user *arg,
+ unsigned int size)
+{
+ int ret = 0;
+ struct nnpdrv_ioctl_lock_hostres lock_args;
+ struct user_hostres *user_hostres_entry;
+ unsigned int io_size = sizeof(lock_args);
+
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&lock_args, arg, io_size))
+ return -EFAULT;
+
+ /* errno must be cleared on entry */
+ if (lock_args.o_errno)
+ return -EINVAL;
+
+ mutex_lock(&user_info->mutex);
+ user_hostres_entry = idr_find(&user_info->idr, lock_args.user_handle);
+ if (user_hostres_entry) {
+ ret = nnp_hostres_user_lock(user_hostres_entry->hostres);
+ } else {
+ lock_args.o_errno = NNPER_NO_SUCH_RESOURCE;
+ if (copy_to_user(arg, &lock_args, io_size))
+ ret = -EFAULT;
+ }
+
+ mutex_unlock(&user_info->mutex);
+ return ret;
+}
+
+static long unlock_hostres(struct nnp_user_info *user_info, void __user *arg,
+ unsigned int size)
+{
+ int ret = 0;
+ struct user_hostres *user_hostres_entry;
+ struct nnpdrv_ioctl_lock_hostres lock_args;
+ unsigned int io_size = sizeof(lock_args);
+
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&lock_args, arg, io_size))
+ return -EFAULT;
+
+ /* errno must be cleared on entry */
+ if (lock_args.o_errno)
+ return -EINVAL;
+
+ mutex_lock(&user_info->mutex);
+ user_hostres_entry = idr_find(&user_info->idr, lock_args.user_handle);
+ if (user_hostres_entry) {
+ ret = nnp_hostres_user_unlock(user_hostres_entry->hostres);
+ } else {
+ lock_args.o_errno = NNPER_NO_SUCH_RESOURCE;
+ if (copy_to_user(arg, &lock_args, sizeof(lock_args)))
+ ret = -EFAULT;
+ }
+
+ mutex_unlock(&user_info->mutex);
+ return ret;
+}
+
+struct file *nnp_host_file_get(int host_fd)
+{
+ struct file *host_file;
+
+ host_file = fget(host_fd);
+ if (is_host_file(host_file))
+ return host_file;
+
+ if (host_file)
+ fput(host_file);
+
+ return NULL;
+}
+
+/*
+ * Inference host cdev (/dev/nnpi_host) file operation functions
+ */
+
+static int host_open(struct inode *inode, struct file *f)
+{
+ struct nnp_user_info *user_info;
+
+ if (!is_host_file(f))
+ return -EINVAL;
+
+ user_info = kzalloc(sizeof(*user_info), GFP_KERNEL);
+ if (!user_info)
+ return -ENOMEM;
+
+ nnp_user_init(user_info);
+
+ f->private_data = user_info;
+
+ return 0;
+}
+
+static int host_release(struct inode *inode, struct file *f)
+{
+ struct nnp_user_info *user_info;
+
+ if (!is_host_file(f))
+ return -EINVAL;
+
+ user_info = f->private_data;
+
+ nnp_user_destroy_all(user_info);
+ f->private_data = NULL;
+
+ return 0;
+}
+
+static long host_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
+{
+ long ret = 0;
+ struct nnp_user_info *user_info = f->private_data;
+ unsigned int ioc_nr, size;
+
+ if (!is_host_file(f))
+ return -ENOTTY;
+
+ if (_IOC_TYPE(cmd) != 'h')
+ return -EINVAL;
+
+ ioc_nr = _IOC_NR(cmd);
+ size = _IOC_SIZE(cmd);
+
+ switch (ioc_nr) {
+ case _IOC_NR(IOCTL_INF_CREATE_HOST_RESOURCE):
+ ret = create_hostres(user_info, (void __user *)arg, size);
+ break;
+ case _IOC_NR(IOCTL_INF_DESTROY_HOST_RESOURCE):
+ ret = destroy_hostres(user_info, (void __user *)arg, size);
+ break;
+ case _IOC_NR(IOCTL_INF_UNLOCK_HOST_RESOURCE):
+ ret = unlock_hostres(user_info, (void __user *)arg, size);
+ break;
+ case _IOC_NR(IOCTL_INF_LOCK_HOST_RESOURCE):
+ ret = lock_hostres(user_info, (void __user *)arg, size);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static const struct file_operations host_fops = {
+ .owner = THIS_MODULE,
+ .open = host_open,
+ .release = host_release,
+ .unlocked_ioctl = host_ioctl,
+ .compat_ioctl = host_ioctl,
+};
+
+static inline int is_host_file(struct file *f)
+{
+ return f && f->f_op == &host_fops;
+}
+
+int nnp_init_host_interface(void)
+{
+ int ret;
+
+ ret = alloc_chrdev_region(&devnum, 0, 1, NNPDRV_INF_HOST_DEV_NAME);
+ if (ret < 0)
+ return ret;
+
+ cdev_init(&cdev, &host_fops);
+ cdev.owner = THIS_MODULE;
+
+ ret = cdev_add(&cdev, devnum, 1);
+ if (ret < 0)
+ goto err_region;
+
+ class = class_create(THIS_MODULE, NNPDRV_INF_HOST_DEV_NAME);
+ if (IS_ERR(class)) {
+ ret = PTR_ERR(class);
+ goto err_cdev;
+ }
+
+ dev = device_create(class, NULL, devnum, NULL, NNPDRV_INF_HOST_DEV_NAME);
+ if (IS_ERR(dev)) {
+ ret = PTR_ERR(dev);
+ goto err_class;
+ }
+
+ ret = nnp_hostres_init_sysfs(dev);
+ if (ret < 0)
+ goto err_device;
+
+ return 0;
+
+err_device:
+ device_destroy(class, devnum);
+err_class:
+ class_destroy(class);
+err_cdev:
+ cdev_del(&cdev);
+err_region:
+ unregister_chrdev_region(devnum, 1);
+
+ return ret;
+}
+
+void nnp_release_host_interface(void)
+{
+ nnp_hostres_fini_sysfs(dev);
+ device_destroy(class, devnum);
+ class_destroy(class);
+ cdev_del(&cdev);
+ unregister_chrdev_region(devnum, 1);
+}
diff --git a/drivers/misc/intel-nnpi/host_chardev.h b/drivers/misc/intel-nnpi/host_chardev.h
new file mode 100644
index 0000000..5812e0f
--- /dev/null
+++ b/drivers/misc/intel-nnpi/host_chardev.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_INFERENCE_H
+#define _NNPDRV_INFERENCE_H
+
+int nnp_init_host_interface(void);
+void nnp_release_host_interface(void);
+
+struct file *nnp_host_file_get(int host_fd);
+
+#endif
diff --git a/drivers/misc/intel-nnpi/nnp_user.c b/drivers/misc/intel-nnpi/nnp_user.c
new file mode 100644
index 0000000..51d7418
--- /dev/null
+++ b/drivers/misc/intel-nnpi/nnp_user.c
@@ -0,0 +1,131 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#include <linux/slab.h>
+
+#include "nnp_user.h"
+
+void nnp_user_init(struct nnp_user_info *user_info)
+{
+ INIT_LIST_HEAD(&user_info->hostres_list);
+ mutex_init(&user_info->mutex);
+ kref_init(&user_info->ref);
+ idr_init(&user_info->idr);
+}
+
+void nnp_user_get(struct nnp_user_info *user_info)
+{
+ kref_get(&user_info->ref);
+}
+
+static void nnp_user_release(struct kref *kref)
+{
+ struct nnp_user_info *user_info =
+ container_of(kref, struct nnp_user_info, ref);
+ struct completion *completion = user_info->close_completion;
+
+ idr_destroy(&user_info->idr);
+ kfree(user_info);
+ complete(completion);
+}
+
+void nnp_user_put(struct nnp_user_info *user_info)
+{
+ kref_put(&user_info->ref, nnp_user_release);
+}
+
+int nnp_user_add_hostres(struct nnp_user_info *user_info,
+ struct host_resource *hostres,
+ struct user_hostres **user_hostres_entry)
+{
+ struct user_hostres *hr_entry;
+ int id;
+
+ hr_entry = kmalloc(sizeof(*hr_entry), GFP_KERNEL);
+ if (!hr_entry)
+ return -ENOMEM;
+
+ /*
+ * Increment refcount to hostres for the entry reference.
+ * (caller holds reference to it, so we know it exist).
+ */
+ nnp_hostres_get(hostres);
+ hr_entry->hostres = hostres;
+
+ /*
+ * We are called from ioctl of file that own this user_info,
+ * So it safe to assume it exist.
+ */
+ nnp_user_get(user_info);
+ hr_entry->user_info = user_info;
+
+ mutex_lock(&user_info->mutex);
+ /*
+ * We allocate handle starting from 1 and not 0 to allow
+ * user-space treat zero as invalid handle
+ */
+ id = idr_alloc(&user_info->idr, hr_entry, 1, -1, GFP_KERNEL);
+ if (id < 0) {
+ nnp_user_put(user_info);
+ nnp_hostres_put(hostres);
+ kfree(hr_entry);
+ mutex_unlock(&user_info->mutex);
+ return -ENOSPC;
+ }
+ hr_entry->user_handle = id;
+ list_add(&hr_entry->node, &user_info->hostres_list);
+ mutex_unlock(&user_info->mutex);
+
+ *user_hostres_entry = hr_entry;
+
+ return 0;
+}
+
+void nnp_user_remove_hostres_locked(struct user_hostres *hr_entry)
+{
+ struct nnp_user_info *user_info = hr_entry->user_info;
+
+ idr_remove(&user_info->idr, hr_entry->user_handle);
+ list_del(&hr_entry->node);
+
+ nnp_hostres_put(hr_entry->hostres);
+
+ kfree(hr_entry);
+ nnp_user_put(user_info);
+}
+
+void nnp_user_remove_hostres(struct user_hostres *hr_entry)
+{
+ struct nnp_user_info *user_info = hr_entry->user_info;
+
+ mutex_lock(&user_info->mutex);
+ nnp_user_remove_hostres_locked(hr_entry);
+ mutex_unlock(&user_info->mutex);
+}
+
+void nnp_user_destroy_all(struct nnp_user_info *user_info)
+{
+ struct user_hostres *user_hostres_entry;
+ DECLARE_COMPLETION_ONSTACK(completion);
+
+ mutex_lock(&user_info->mutex);
+
+ /* destroy all hostreses owned by the "user" */
+ while (!list_empty(&user_info->hostres_list)) {
+ user_hostres_entry = list_first_entry(&user_info->hostres_list,
+ struct user_hostres, node);
+ /*
+ * We can safely destroy this object without checking
+ * its refcount since we get here only after the host char-dev
+ * as well as all cmd_chan char-devs that may hold temporary
+ * reference to this object are already released.
+ */
+ nnp_user_remove_hostres_locked(user_hostres_entry);
+ }
+ mutex_unlock(&user_info->mutex);
+
+ /* wait for all channels and hostreses to be destroyed */
+ user_info->close_completion = &completion;
+ nnp_user_put(user_info);
+ wait_for_completion(&completion);
+}
diff --git a/drivers/misc/intel-nnpi/nnp_user.h b/drivers/misc/intel-nnpi/nnp_user.h
new file mode 100644
index 0000000..429ac13
--- /dev/null
+++ b/drivers/misc/intel-nnpi/nnp_user.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_INF_PROC_H
+#define _NNPDRV_INF_PROC_H
+
+#include <linux/kref.h>
+#include <linux/types.h>
+
+#include "hostres.h"
+
+/**
+ * struct nnp_user_info - structure for per-user info
+ * @ref: refcount to this "user" object
+ * @hostres_list: list of host resources
+ * @close_completion: used to wait for all channels of this user to be
+ * destroyed before closing the user.
+ * @mutex: protects hostres_list and idr modifications
+ * @idr: used to generate user handles to created host resources
+ * @user_list_node: list node to attach this struct in "list of users".
+ *
+ * structure to hold per-user info,
+ * a "user" is created for each open made to the host char dev (/dev/nnpi_host).
+ * It holds a list of all host resources created through requests from
+ * the same client ("user").
+ * device communication "channels", created by device char dev (/dev/nnpi%d)
+ * must be correlated with a "user" object which is supplied from user-space
+ * by the opened file descriptor to /dev/nnpi_host. Such "channel" may access
+ * only host resources created by the same "user".
+ * The lifetime of this object last at least for the duration of the host char
+ * device file struct but can last longer if some channel objects still hold
+ * a reference to it (this is why @ref is needed).
+ */
+struct nnp_user_info {
+ struct kref ref;
+ struct list_head hostres_list;
+ struct completion *close_completion;
+ struct mutex mutex;
+ struct idr idr;
+ struct list_head user_list_node;
+};
+
+/**
+ * struct user_hostres - structure for host resource created by user
+ * @node: list node to attach this struct to nnp_user_info::hostres_list
+ * @hostres: the actual host resource object
+ * @user_handle: handle allocated from idr object, used as handle to this
+ * object in ioctl ABI.
+ * @user_info: pointer to "user" which created this resource.
+ * it is used only during destruction of the object.
+ *
+ * structure for a host resource object which created through host char dev
+ * request. The lifetime of this structure ends when the user request to
+ * destroy it through ioctl call. The underlying @hostres may still continue
+ * to exist if command channel (cmd_chan) objects has mapped the resource to
+ * device access.
+ */
+struct user_hostres {
+ struct list_head node;
+ struct host_resource *hostres;
+ int user_handle;
+ struct nnp_user_info *user_info;
+};
+
+void nnp_user_init(struct nnp_user_info *user_info);
+
+void nnp_user_get(struct nnp_user_info *user_info);
+void nnp_user_put(struct nnp_user_info *user_info);
+
+int nnp_user_add_hostres(struct nnp_user_info *user_info,
+ struct host_resource *hostres,
+ struct user_hostres **user_hostres_entry);
+
+void nnp_user_remove_hostres(struct user_hostres *hr_entry);
+void nnp_user_remove_hostres_locked(struct user_hostres *hr_entry);
+
+void nnp_user_destroy_all(struct nnp_user_info *user_info);
+
+#endif
diff --git a/include/uapi/misc/intel_nnpi.h b/include/uapi/misc/intel_nnpi.h
new file mode 100644
index 0000000..5114aea
--- /dev/null
+++ b/include/uapi/misc/intel_nnpi.h
@@ -0,0 +1,150 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNP_UAPI_H
+#define _NNP_UAPI_H
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+#include <stdbool.h>
+
+#define NNPDRV_INF_HOST_DEV_NAME "nnpi_host"
+
+/*
+ * ioctls for /dev/nnpi_host device
+ */
+
+/*
+ * IOCTL_INF_CREATE_HOST_RESOURCE:
+ *
+ * A request to create a host memory resource object that can then be mapped
+ * and accessed by the NNP-I device's DMA engine.
+ * The created host resource is pinned in memory for its entire lifecycle.
+ * The memory of the resource is backed by user allocated memory which
+ * get pinned by the IOCTL.
+ *
+ * See description of nnpdrv_ioctl_create_hostres structure for more details.
+ *
+ * The ioctl returns a handle to the created host resource.
+ */
+#define IOCTL_INF_CREATE_HOST_RESOURCE \
+ _IOWR('h', 0, struct nnpdrv_ioctl_create_hostres)
+
+/*
+ * IOCTL_INF_DESTROY_HOST_RESOURCE:
+ *
+ * A request to destoy a host resource object.
+ */
+#define IOCTL_INF_DESTROY_HOST_RESOURCE \
+ _IOWR('h', 2, struct nnpdrv_ioctl_destroy_hostres)
+
+/*
+ * IOCTL_INF_LOCK_HOST_RESOURCE:
+ *
+ * A request to lock a host resource for cpu access for either
+ * read or write.
+ *
+ * This IOCTL does *not* synchronize accessed to host memory between host
+ * cpu and the device's DMA engine. It is used only for either flush or
+ * invalidate cpu caches to let the device see the last writes made from
+ * host cpu and let cpu read up-to-date content of the resource after the
+ * device changed it.
+ *
+ * This synchronization is not required on all platforms, when mapping
+ * the resource for device access, using IOCTL_NNPI_DEVICE_CHANNEL_MAP_HOSTRES,
+ * the application receive an indication if such synchronization is needed
+ * or not with that device.
+ *
+ * When such synchronization is needed:
+ * When application wants to change host resource content to be read by the
+ * device, it should first lock it for write, change its content by accessing
+ * it's mapped virtual address and then call this ioctl again to unlock it
+ * before sending a command to the device which may read the resource.
+ * When the application received indication that the device has changed the
+ * resource content, it should first lock the resource for reading before
+ * accessing its memory.
+ */
+#define IOCTL_INF_LOCK_HOST_RESOURCE \
+ _IOWR('h', 3, struct nnpdrv_ioctl_lock_hostres)
+
+/*
+ * IOCTL_INF_UNLOCK_HOST_RESOURCE:
+ *
+ * A request to unlock a host resource that was previously locked for cpu access.
+ */
+#define IOCTL_INF_UNLOCK_HOST_RESOURCE \
+ _IOWR('h', 4, struct nnpdrv_ioctl_lock_hostres)
+
+/*
+ * The below are possible bit masks that can be specified in
+ * usage_flags field of struct nnpdrv_ioctl_create_hostres.
+ * It specify attribute and usage flags for a host resource.
+ */
+#define IOCTL_INF_RES_INPUT (1u << 0) /* being read by the NNP-I device */
+#define IOCTL_INF_RES_OUTPUT (1u << 1) /* being written by the device */
+#define IOCTL_RES_USAGE_VALID_MASK (IOCTL_INF_RES_INPUT | IOCTL_INF_RES_OUTPUT)
+
+/**
+ * struct nnpdrv_ioctl_create_hostres - IOCTL_INF_CREATE_HOST_RESOURCE payload
+ * @user_ptr: User virtual address.
+ * @size: User memory size on input. Host resource size on output.
+ * @usage_flags: resource usage flag bits, IOCTL_INF_RES_*
+ * @user_handle: resource handle on output.
+ *
+ * argument structure for IOCTL_INF_CREATE_HOST_RESOURCE ioctl
+ *
+ * @user_ptr should be initialized to a user virtual address and @size
+ * should be initialized with it's size, the user memory will be pinned and will
+ * hold the host resource content.
+ *
+ * On output, @user_handle is a handle to the created host resource that can be
+ * used later with other IOCTLs and @size is the size of the host resource.
+ */
+struct nnpdrv_ioctl_create_hostres {
+ __u64 user_ptr;
+ __u64 size;
+ __u32 usage_flags;
+ __s32 user_handle;
+};
+
+/**
+ * struct nnpdrv_ioctl_lock_hostres - IOCTL_INF_LOCK_HOST_RESOURCE payload
+ * @user_handle: handle to host resource object
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * argument structure for IOCTL_INF_LOCK_HOST_RESOURCE and
+ * IOCTL_INF_LOCK_HOST_RESOURCE ioctl calls.
+ */
+struct nnpdrv_ioctl_lock_hostres {
+ __s32 user_handle;
+ __u32 o_errno;
+};
+
+/**
+ * struct nnpdrv_ioctl_destroy_hostres - IOCTL_INF_DESTROY_HOST_RESOURCE payload
+ * @user_handle: handle to host resource object
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * argument structure for IOCTL_INF_DESTROY_HOST_RESOURCE ioctl
+ */
+struct nnpdrv_ioctl_destroy_hostres {
+ __s32 user_handle;
+ __u32 o_errno;
+};
+
+/****************************************************************
+ * Error code values - errors returned in o_errno fields of
+ * above structures.
+ ****************************************************************/
+#define NNP_ERRNO_BASE 200
+#define NNPER_DEVICE_NOT_READY (NNP_ERRNO_BASE + 1)
+#define NNPER_NO_SUCH_RESOURCE (NNP_ERRNO_BASE + 2)
+#define NNPER_INCOMPATIBLE_RESOURCES (NNP_ERRNO_BASE + 3)
+#define NNPER_DEVICE_ERROR (NNP_ERRNO_BASE + 4)
+#define NNPER_NO_SUCH_CHANNEL (NNP_ERRNO_BASE + 5)
+#define NNPER_NO_SUCH_HOSTRES_MAP (NNP_ERRNO_BASE + 6)
+#define NNPER_VERSIONS_MISMATCH (NNP_ERRNO_BASE + 7)
+
+#endif /* of _NNP_UAPI_H */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v2 06/15] misc: nnpi: Allow usermode to manage host resources
2021-05-13 8:57 ` [PATCH v2 06/15] misc: nnpi: Allow usermode to manage host resources Guy Zadicario
@ 2021-05-17 7:02 ` Dave Airlie
0 siblings, 0 replies; 20+ messages in thread
From: Dave Airlie @ 2021-05-17 7:02 UTC (permalink / raw)
To: Guy Zadicario
Cc: Greg Kroah-Hartman, LKML, Olof Johansson, alexander.shishkin,
andriy.shevchenko, yochai.shefi-simchon
On Thu, 13 May 2021 at 19:55, Guy Zadicario <guy.zadicario@intel.com> wrote:
>
> Provide an IOCTL interface for creating and destroying host memory
> resources through a character device (/dev/nnpi_host).
>
> There is a single instance of this character device in the system
> regardless of the number of NNP-I devices attached because it
> controls host resources which may be shared between different devices.
>
> A nnp_user object, created when an application opens this character
> device, identifies the user (client) of the driver and holds a list of
> all host resources allocated by that user through the opened file
> descriptor.
>
> Host memory resources created through this character device can be mapped
> to device access through IOCTLs made to a different, per-device, chardev
> (will be introduced on next commits).
>
> All resources will be destroyed when the application closes the connection
> or exits.
>
> The IOCTL interface is defined in: include/uapi/misc/intel_nnpi.h
>
> Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
> Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> ---
> MAINTAINERS | 1 +
> drivers/misc/intel-nnpi/Makefile | 2 +-
> drivers/misc/intel-nnpi/device.c | 14 ++
> drivers/misc/intel-nnpi/host_chardev.c | 346 +++++++++++++++++++++++++++++++++
> drivers/misc/intel-nnpi/host_chardev.h | 12 ++
> drivers/misc/intel-nnpi/nnp_user.c | 131 +++++++++++++
> drivers/misc/intel-nnpi/nnp_user.h | 79 ++++++++
> include/uapi/misc/intel_nnpi.h | 150 ++++++++++++++
> 8 files changed, 734 insertions(+), 1 deletion(-)
> create mode 100644 drivers/misc/intel-nnpi/host_chardev.c
> create mode 100644 drivers/misc/intel-nnpi/host_chardev.h
> create mode 100644 drivers/misc/intel-nnpi/nnp_user.c
> create mode 100644 drivers/misc/intel-nnpi/nnp_user.h
> create mode 100644 include/uapi/misc/intel_nnpi.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index ff0c3d7..42f3e54 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -9320,6 +9320,7 @@ INTEL NNP-I PCI DRIVER
> M: Guy Zadicario <guy.zadicario@intel.com>
> S: Supported
> F: drivers/misc/intel-nnpi/
> +F: include/uapi/misc/intel_nnpi.h
>
> INTEL P-Unit IPC DRIVER
> M: Zha Qipeng <qipeng.zha@intel.com>
> diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
> index 3713d51..da2863b 100644
> --- a/drivers/misc/intel-nnpi/Makefile
> +++ b/drivers/misc/intel-nnpi/Makefile
> @@ -5,7 +5,7 @@
>
> obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
>
> -intel_nnpi-y := device.o msg_scheduler.o hostres.o
> +intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o
>
> intel_nnpi_pcie-y := nnp_pcie.o
>
> diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
> index 60c5a94..0f98398 100644
> --- a/drivers/misc/intel-nnpi/device.c
> +++ b/drivers/misc/intel-nnpi/device.c
> @@ -8,6 +8,7 @@
> #include <linux/module.h>
>
> #include "device.h"
> +#include "host_chardev.h"
> #include "msg_scheduler.h"
>
> static DEFINE_IDA(dev_ida);
> @@ -99,6 +100,19 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
> }
> EXPORT_SYMBOL(nnpdev_destroy);
>
> +static int __init nnp_init(void)
> +{
> + return nnp_init_host_interface();
> +}
> +subsys_initcall(nnp_init);
> +
> +static void __exit nnp_cleanup(void)
> +{
> + nnp_release_host_interface();
> + /* dev_ida is already empty here - no point calling ida_destroy */
> +}
> +module_exit(nnp_cleanup);
> +
> MODULE_LICENSE("GPL");
> MODULE_DESCRIPTION("Intel(R) NNPI Framework");
> MODULE_AUTHOR("Intel Corporation");
> diff --git a/drivers/misc/intel-nnpi/host_chardev.c b/drivers/misc/intel-nnpi/host_chardev.c
> new file mode 100644
> index 0000000..6048fda
> --- /dev/null
> +++ b/drivers/misc/intel-nnpi/host_chardev.c
> @@ -0,0 +1,346 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright (C) 2019-2021 Intel Corporation */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/cdev.h>
> +#include <linux/device.h>
> +#include <linux/file.h>
> +#include <linux/fs.h>
> +#include <linux/hashtable.h>
> +#include <linux/idr.h>
> +#include <linux/printk.h>
> +#include <linux/slab.h>
> +#include <uapi/misc/intel_nnpi.h>
> +
> +#include "device.h"
> +#include "host_chardev.h"
> +#include "ipc_protocol.h"
> +#include "nnp_user.h"
> +
> +static struct cdev cdev;
> +static dev_t devnum;
> +static struct class *class;
> +static struct device *dev;
> +
> +static inline int is_host_file(struct file *f);
> +
> +static enum dma_data_direction to_dma_dir(unsigned int nnp_dir)
> +{
> + /* Ignore IOCTL_INF_RES_NETWORK */
> + switch (nnp_dir & (IOCTL_INF_RES_INPUT | IOCTL_INF_RES_OUTPUT)) {
> + case (IOCTL_INF_RES_INPUT | IOCTL_INF_RES_OUTPUT):
> + return DMA_BIDIRECTIONAL;
> + case IOCTL_INF_RES_INPUT:
> + return DMA_TO_DEVICE;
> + case IOCTL_INF_RES_OUTPUT:
> + return DMA_FROM_DEVICE;
> + default:
> + break;
> + }
> +
> + return DMA_NONE;
> +}
> +
> +static long create_hostres(struct nnp_user_info *user_info, void __user *arg,
> + unsigned int size)
> +{
> + int ret;
> + struct nnpdrv_ioctl_create_hostres req;
> + struct host_resource *hostres;
> + struct user_hostres *user_hostres_entry;
> + void __user *uptr;
> + unsigned int io_size = sizeof(req);
> +
> + if (size != io_size)
> + return -EINVAL;
> +
> + if (copy_from_user(&req, arg, io_size))
> + return -EFAULT;
> +
> + if (req.usage_flags & ~IOCTL_RES_USAGE_VALID_MASK)
> + return -EINVAL;
> +
> + uptr = u64_to_user_ptr(req.user_ptr);
> + hostres = nnp_hostres_from_usermem(uptr, req.size,
> + to_dma_dir(req.usage_flags));
> +
> + if (IS_ERR(hostres))
> + return PTR_ERR(hostres);
> +
> + ret = nnp_user_add_hostres(user_info, hostres, &user_hostres_entry);
> + if (ret < 0) {
> + nnp_hostres_put(hostres);
> + return ret;
> + }
> +
> + req.size = nnp_hostres_size(hostres);
> +
> + /*
> + * The created user_hostres_entry holds refcount to the resource,
> + * no need to keep another one here.
> + */
> + nnp_hostres_put(hostres);
> +
> + req.user_handle = user_hostres_entry->user_handle;
> + if (copy_to_user(arg, &req, io_size)) {
> + ret = -EFAULT;
> + goto destroy_hostres_entry;
> + }
> +
> + return 0;
> +
> +destroy_hostres_entry:
> + nnp_user_remove_hostres(user_hostres_entry);
> +
> + return ret;
> +}
> +
> +static long destroy_hostres(struct nnp_user_info *user_info, void __user *arg,
> + unsigned int size)
> +{
> + struct nnpdrv_ioctl_destroy_hostres destroy_args;
> + struct user_hostres *user_hostres_entry;
> + unsigned int io_size = sizeof(destroy_args);
> + int ret = 0;
> +
> + if (size != io_size)
> + return -EINVAL;
> +
> + if (copy_from_user(&destroy_args, arg, io_size))
> + return -EFAULT;
> +
> + /* errno must be cleared on entry */
> + if (destroy_args.o_errno)
> + return -EINVAL;
> +
> + mutex_lock(&user_info->mutex);
> + user_hostres_entry = idr_find(&user_info->idr, destroy_args.user_handle);
> + if (user_hostres_entry) {
> + nnp_user_remove_hostres_locked(user_hostres_entry);
> + } else {
> + destroy_args.o_errno = NNPER_NO_SUCH_RESOURCE;
> + if (copy_to_user(arg, &destroy_args, io_size))
> + ret = -EFAULT;
> + }
> +
> + mutex_unlock(&user_info->mutex);
> + return ret;
> +}
> +
> +static long lock_hostres(struct nnp_user_info *user_info, void __user *arg,
> + unsigned int size)
> +{
> + int ret = 0;
> + struct nnpdrv_ioctl_lock_hostres lock_args;
> + struct user_hostres *user_hostres_entry;
> + unsigned int io_size = sizeof(lock_args);
> +
> + if (size != io_size)
> + return -EINVAL;
> +
> + if (copy_from_user(&lock_args, arg, io_size))
> + return -EFAULT;
> +
> + /* errno must be cleared on entry */
> + if (lock_args.o_errno)
> + return -EINVAL;
> +
> + mutex_lock(&user_info->mutex);
> + user_hostres_entry = idr_find(&user_info->idr, lock_args.user_handle);
> + if (user_hostres_entry) {
> + ret = nnp_hostres_user_lock(user_hostres_entry->hostres);
> + } else {
> + lock_args.o_errno = NNPER_NO_SUCH_RESOURCE;
> + if (copy_to_user(arg, &lock_args, io_size))
> + ret = -EFAULT;
> + }
> +
> + mutex_unlock(&user_info->mutex);
> + return ret;
> +}
> +
> +static long unlock_hostres(struct nnp_user_info *user_info, void __user *arg,
> + unsigned int size)
> +{
> + int ret = 0;
> + struct user_hostres *user_hostres_entry;
> + struct nnpdrv_ioctl_lock_hostres lock_args;
> + unsigned int io_size = sizeof(lock_args);
> +
> + if (size != io_size)
> + return -EINVAL;
> +
> + if (copy_from_user(&lock_args, arg, io_size))
> + return -EFAULT;
> +
> + /* errno must be cleared on entry */
> + if (lock_args.o_errno)
> + return -EINVAL;
> +
> + mutex_lock(&user_info->mutex);
> + user_hostres_entry = idr_find(&user_info->idr, lock_args.user_handle);
> + if (user_hostres_entry) {
> + ret = nnp_hostres_user_unlock(user_hostres_entry->hostres);
> + } else {
> + lock_args.o_errno = NNPER_NO_SUCH_RESOURCE;
> + if (copy_to_user(arg, &lock_args, sizeof(lock_args)))
> + ret = -EFAULT;
> + }
> +
> + mutex_unlock(&user_info->mutex);
> + return ret;
> +}
> +
> +struct file *nnp_host_file_get(int host_fd)
> +{
> + struct file *host_file;
> +
> + host_file = fget(host_fd);
> + if (is_host_file(host_file))
> + return host_file;
> +
> + if (host_file)
> + fput(host_file);
> +
> + return NULL;
> +}
> +
> +/*
> + * Inference host cdev (/dev/nnpi_host) file operation functions
> + */
> +
> +static int host_open(struct inode *inode, struct file *f)
> +{
> + struct nnp_user_info *user_info;
> +
> + if (!is_host_file(f))
> + return -EINVAL;
> +
> + user_info = kzalloc(sizeof(*user_info), GFP_KERNEL);
> + if (!user_info)
> + return -ENOMEM;
> +
> + nnp_user_init(user_info);
> +
> + f->private_data = user_info;
> +
> + return 0;
> +}
> +
> +static int host_release(struct inode *inode, struct file *f)
> +{
> + struct nnp_user_info *user_info;
> +
> + if (!is_host_file(f))
> + return -EINVAL;
> +
> + user_info = f->private_data;
> +
> + nnp_user_destroy_all(user_info);
> + f->private_data = NULL;
> +
> + return 0;
> +}
> +
> +static long host_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
> +{
> + long ret = 0;
> + struct nnp_user_info *user_info = f->private_data;
> + unsigned int ioc_nr, size;
> +
> + if (!is_host_file(f))
> + return -ENOTTY;
> +
> + if (_IOC_TYPE(cmd) != 'h')
> + return -EINVAL;
> +
> + ioc_nr = _IOC_NR(cmd);
> + size = _IOC_SIZE(cmd);
> +
> + switch (ioc_nr) {
> + case _IOC_NR(IOCTL_INF_CREATE_HOST_RESOURCE):
> + ret = create_hostres(user_info, (void __user *)arg, size);
> + break;
> + case _IOC_NR(IOCTL_INF_DESTROY_HOST_RESOURCE):
> + ret = destroy_hostres(user_info, (void __user *)arg, size);
> + break;
> + case _IOC_NR(IOCTL_INF_UNLOCK_HOST_RESOURCE):
> + ret = unlock_hostres(user_info, (void __user *)arg, size);
> + break;
> + case _IOC_NR(IOCTL_INF_LOCK_HOST_RESOURCE):
> + ret = lock_hostres(user_info, (void __user *)arg, size);
> + break;
> + default:
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static const struct file_operations host_fops = {
> + .owner = THIS_MODULE,
> + .open = host_open,
> + .release = host_release,
> + .unlocked_ioctl = host_ioctl,
> + .compat_ioctl = host_ioctl,
> +};
> +
> +static inline int is_host_file(struct file *f)
> +{
> + return f && f->f_op == &host_fops;
> +}
> +
> +int nnp_init_host_interface(void)
> +{
> + int ret;
> +
> + ret = alloc_chrdev_region(&devnum, 0, 1, NNPDRV_INF_HOST_DEV_NAME);
> + if (ret < 0)
> + return ret;
> +
> + cdev_init(&cdev, &host_fops);
> + cdev.owner = THIS_MODULE;
> +
> + ret = cdev_add(&cdev, devnum, 1);
> + if (ret < 0)
> + goto err_region;
> +
> + class = class_create(THIS_MODULE, NNPDRV_INF_HOST_DEV_NAME);
> + if (IS_ERR(class)) {
> + ret = PTR_ERR(class);
> + goto err_cdev;
> + }
> +
> + dev = device_create(class, NULL, devnum, NULL, NNPDRV_INF_HOST_DEV_NAME);
> + if (IS_ERR(dev)) {
> + ret = PTR_ERR(dev);
> + goto err_class;
> + }
> +
> + ret = nnp_hostres_init_sysfs(dev);
> + if (ret < 0)
> + goto err_device;
> +
> + return 0;
> +
> +err_device:
> + device_destroy(class, devnum);
> +err_class:
> + class_destroy(class);
> +err_cdev:
> + cdev_del(&cdev);
> +err_region:
> + unregister_chrdev_region(devnum, 1);
> +
> + return ret;
> +}
> +
> +void nnp_release_host_interface(void)
> +{
> + nnp_hostres_fini_sysfs(dev);
> + device_destroy(class, devnum);
> + class_destroy(class);
> + cdev_del(&cdev);
> + unregister_chrdev_region(devnum, 1);
> +}
> diff --git a/drivers/misc/intel-nnpi/host_chardev.h b/drivers/misc/intel-nnpi/host_chardev.h
> new file mode 100644
> index 0000000..5812e0f
> --- /dev/null
> +++ b/drivers/misc/intel-nnpi/host_chardev.h
> @@ -0,0 +1,12 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright (C) 2019-2021 Intel Corporation */
> +
> +#ifndef _NNPDRV_INFERENCE_H
> +#define _NNPDRV_INFERENCE_H
> +
> +int nnp_init_host_interface(void);
> +void nnp_release_host_interface(void);
> +
> +struct file *nnp_host_file_get(int host_fd);
> +
> +#endif
> diff --git a/drivers/misc/intel-nnpi/nnp_user.c b/drivers/misc/intel-nnpi/nnp_user.c
> new file mode 100644
> index 0000000..51d7418
> --- /dev/null
> +++ b/drivers/misc/intel-nnpi/nnp_user.c
> @@ -0,0 +1,131 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright (C) 2019-2021 Intel Corporation */
> +
> +#include <linux/slab.h>
> +
> +#include "nnp_user.h"
> +
> +void nnp_user_init(struct nnp_user_info *user_info)
> +{
> + INIT_LIST_HEAD(&user_info->hostres_list);
> + mutex_init(&user_info->mutex);
> + kref_init(&user_info->ref);
> + idr_init(&user_info->idr);
> +}
> +
> +void nnp_user_get(struct nnp_user_info *user_info)
> +{
> + kref_get(&user_info->ref);
> +}
> +
> +static void nnp_user_release(struct kref *kref)
> +{
> + struct nnp_user_info *user_info =
> + container_of(kref, struct nnp_user_info, ref);
> + struct completion *completion = user_info->close_completion;
> +
> + idr_destroy(&user_info->idr);
> + kfree(user_info);
> + complete(completion);
> +}
> +
> +void nnp_user_put(struct nnp_user_info *user_info)
> +{
> + kref_put(&user_info->ref, nnp_user_release);
> +}
> +
> +int nnp_user_add_hostres(struct nnp_user_info *user_info,
> + struct host_resource *hostres,
> + struct user_hostres **user_hostres_entry)
> +{
> + struct user_hostres *hr_entry;
> + int id;
> +
> + hr_entry = kmalloc(sizeof(*hr_entry), GFP_KERNEL);
> + if (!hr_entry)
> + return -ENOMEM;
> +
> + /*
> + * Increment refcount to hostres for the entry reference.
> + * (caller holds reference to it, so we know it exist).
> + */
> + nnp_hostres_get(hostres);
> + hr_entry->hostres = hostres;
> +
> + /*
> + * We are called from ioctl of file that own this user_info,
> + * So it safe to assume it exist.
> + */
> + nnp_user_get(user_info);
> + hr_entry->user_info = user_info;
> +
> + mutex_lock(&user_info->mutex);
> + /*
> + * We allocate handle starting from 1 and not 0 to allow
> + * user-space treat zero as invalid handle
> + */
> + id = idr_alloc(&user_info->idr, hr_entry, 1, -1, GFP_KERNEL);
> + if (id < 0) {
> + nnp_user_put(user_info);
> + nnp_hostres_put(hostres);
> + kfree(hr_entry);
> + mutex_unlock(&user_info->mutex);
> + return -ENOSPC;
> + }
> + hr_entry->user_handle = id;
> + list_add(&hr_entry->node, &user_info->hostres_list);
> + mutex_unlock(&user_info->mutex);
> +
> + *user_hostres_entry = hr_entry;
> +
> + return 0;
> +}
> +
> +void nnp_user_remove_hostres_locked(struct user_hostres *hr_entry)
> +{
> + struct nnp_user_info *user_info = hr_entry->user_info;
> +
> + idr_remove(&user_info->idr, hr_entry->user_handle);
> + list_del(&hr_entry->node);
> +
> + nnp_hostres_put(hr_entry->hostres);
> +
> + kfree(hr_entry);
> + nnp_user_put(user_info);
> +}
> +
> +void nnp_user_remove_hostres(struct user_hostres *hr_entry)
> +{
> + struct nnp_user_info *user_info = hr_entry->user_info;
> +
> + mutex_lock(&user_info->mutex);
> + nnp_user_remove_hostres_locked(hr_entry);
> + mutex_unlock(&user_info->mutex);
> +}
> +
> +void nnp_user_destroy_all(struct nnp_user_info *user_info)
> +{
> + struct user_hostres *user_hostres_entry;
> + DECLARE_COMPLETION_ONSTACK(completion);
> +
> + mutex_lock(&user_info->mutex);
> +
> + /* destroy all hostreses owned by the "user" */
> + while (!list_empty(&user_info->hostres_list)) {
> + user_hostres_entry = list_first_entry(&user_info->hostres_list,
> + struct user_hostres, node);
> + /*
> + * We can safely destroy this object without checking
> + * its refcount since we get here only after the host char-dev
> + * as well as all cmd_chan char-devs that may hold temporary
> + * reference to this object are already released.
> + */
> + nnp_user_remove_hostres_locked(user_hostres_entry);
> + }
> + mutex_unlock(&user_info->mutex);
> +
> + /* wait for all channels and hostreses to be destroyed */
> + user_info->close_completion = &completion;
> + nnp_user_put(user_info);
> + wait_for_completion(&completion);
> +}
> diff --git a/drivers/misc/intel-nnpi/nnp_user.h b/drivers/misc/intel-nnpi/nnp_user.h
> new file mode 100644
> index 0000000..429ac13
> --- /dev/null
> +++ b/drivers/misc/intel-nnpi/nnp_user.h
> @@ -0,0 +1,79 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright (C) 2019-2021 Intel Corporation */
> +
> +#ifndef _NNPDRV_INF_PROC_H
> +#define _NNPDRV_INF_PROC_H
> +
> +#include <linux/kref.h>
> +#include <linux/types.h>
> +
> +#include "hostres.h"
> +
> +/**
> + * struct nnp_user_info - structure for per-user info
> + * @ref: refcount to this "user" object
> + * @hostres_list: list of host resources
> + * @close_completion: used to wait for all channels of this user to be
> + * destroyed before closing the user.
> + * @mutex: protects hostres_list and idr modifications
> + * @idr: used to generate user handles to created host resources
> + * @user_list_node: list node to attach this struct in "list of users".
> + *
> + * structure to hold per-user info,
> + * a "user" is created for each open made to the host char dev (/dev/nnpi_host).
> + * It holds a list of all host resources created through requests from
> + * the same client ("user").
> + * device communication "channels", created by device char dev (/dev/nnpi%d)
> + * must be correlated with a "user" object which is supplied from user-space
> + * by the opened file descriptor to /dev/nnpi_host. Such "channel" may access
> + * only host resources created by the same "user".
> + * The lifetime of this object last at least for the duration of the host char
> + * device file struct but can last longer if some channel objects still hold
> + * a reference to it (this is why @ref is needed).
> + */
> +struct nnp_user_info {
> + struct kref ref;
> + struct list_head hostres_list;
> + struct completion *close_completion;
> + struct mutex mutex;
> + struct idr idr;
> + struct list_head user_list_node;
> +};
> +
> +/**
> + * struct user_hostres - structure for host resource created by user
> + * @node: list node to attach this struct to nnp_user_info::hostres_list
> + * @hostres: the actual host resource object
> + * @user_handle: handle allocated from idr object, used as handle to this
> + * object in ioctl ABI.
> + * @user_info: pointer to "user" which created this resource.
> + * it is used only during destruction of the object.
> + *
> + * structure for a host resource object which created through host char dev
> + * request. The lifetime of this structure ends when the user request to
> + * destroy it through ioctl call. The underlying @hostres may still continue
> + * to exist if command channel (cmd_chan) objects has mapped the resource to
> + * device access.
> + */
> +struct user_hostres {
> + struct list_head node;
> + struct host_resource *hostres;
> + int user_handle;
> + struct nnp_user_info *user_info;
> +};
> +
> +void nnp_user_init(struct nnp_user_info *user_info);
> +
> +void nnp_user_get(struct nnp_user_info *user_info);
> +void nnp_user_put(struct nnp_user_info *user_info);
> +
> +int nnp_user_add_hostres(struct nnp_user_info *user_info,
> + struct host_resource *hostres,
> + struct user_hostres **user_hostres_entry);
> +
> +void nnp_user_remove_hostres(struct user_hostres *hr_entry);
> +void nnp_user_remove_hostres_locked(struct user_hostres *hr_entry);
> +
> +void nnp_user_destroy_all(struct nnp_user_info *user_info);
> +
> +#endif
> diff --git a/include/uapi/misc/intel_nnpi.h b/include/uapi/misc/intel_nnpi.h
> new file mode 100644
> index 0000000..5114aea
> --- /dev/null
> +++ b/include/uapi/misc/intel_nnpi.h
> @@ -0,0 +1,150 @@
> +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
> +/* Copyright (C) 2019-2021 Intel Corporation */
> +
> +#ifndef _NNP_UAPI_H
> +#define _NNP_UAPI_H
> +
> +#include <linux/types.h>
> +#include <linux/ioctl.h>
> +#include <stdbool.h>
> +
> +#define NNPDRV_INF_HOST_DEV_NAME "nnpi_host"
> +
> +/*
> + * ioctls for /dev/nnpi_host device
> + */
> +
> +/*
> + * IOCTL_INF_CREATE_HOST_RESOURCE:
> + *
> + * A request to create a host memory resource object that can then be mapped
> + * and accessed by the NNP-I device's DMA engine.
> + * The created host resource is pinned in memory for its entire lifecycle.
Drive-by comments, but who has access to this interface? non-root?
How much memory can they pin with it?
Pinning is not a nice way to do things, you likely need some sort of
memory manager and fencing system.
Dave.
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 07/15] misc: nnpi: Disallow host memory resource access if no NNP-I devices exist
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (5 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 06/15] misc: nnpi: Allow usermode to manage host resources Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 08/15] misc: nnpi: Boot NNP-I device Guy Zadicario
` (8 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
There is no point allowing a user application to create host resources
on a system that does not equipped with any NNP-I devices.
Fail openning the nnpi_host character device when no NNP-I devices are
attached.
It is OK to do that check without any synchronization as a race would not
be an issue.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/device.c | 5 +++++
drivers/misc/intel-nnpi/device.h | 2 ++
drivers/misc/intel-nnpi/host_chardev.c | 7 +++++++
3 files changed, 14 insertions(+)
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 0f98398..a3c6a1d 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -13,6 +13,11 @@
static DEFINE_IDA(dev_ida);
+bool nnpdev_no_devices(void)
+{
+ return ida_is_empty(&dev_ida);
+}
+
/**
* nnpdev_init() - initialize NNP-I device structure.
* @nnpdev: device to be initialized
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 7d7ef60..562bbc4 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -36,6 +36,8 @@ struct nnp_device_ops {
int (*cmdq_write_mesg)(struct nnp_device *nnpdev, u64 *msg, u32 size);
};
+bool nnpdev_no_devices(void);
+
/*
* Functions exported by the device framework module which are
* called by the lower layer NNP-I device driver module
diff --git a/drivers/misc/intel-nnpi/host_chardev.c b/drivers/misc/intel-nnpi/host_chardev.c
index 6048fda..fad5954 100644
--- a/drivers/misc/intel-nnpi/host_chardev.c
+++ b/drivers/misc/intel-nnpi/host_chardev.c
@@ -217,6 +217,13 @@ static int host_open(struct inode *inode, struct file *f)
if (!is_host_file(f))
return -EINVAL;
+ /*
+ * No point to serve host resource creation while no
+ * NNP-I devices exist in the system.
+ */
+ if (nnpdev_no_devices())
+ return -ENODEV;
+
user_info = kzalloc(sizeof(*user_info), GFP_KERNEL);
if (!user_info)
return -ENOMEM;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 08/15] misc: nnpi: Boot NNP-I device
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (6 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 07/15] misc: nnpi: Disallow host memory resource access if no NNP-I devices exist Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 09/15] misc: nnpi: Process device response messages Guy Zadicario
` (7 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Boot the NNP-I device after the card is powered-on or reset. When the
NNP-I card comes up, it's flashed BIOS starts running.
Since the persistent storage on the device is rather small, it needs to
get its OS boot image from the host driver. So once the card's BIOS is
running, the host driver loads and provides a boot image to the card BIOS,
which uses it to run the Embedded Linux image and SW stack on the card.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/Makefile | 3 +-
drivers/misc/intel-nnpi/bootimage.c | 246 ++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/bootimage.h | 43 ++++++
drivers/misc/intel-nnpi/device.c | 291 ++++++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/device.h | 77 ++++++++++
drivers/misc/intel-nnpi/nnp_pcie.c | 16 ++
6 files changed, 675 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/intel-nnpi/bootimage.c
create mode 100644 drivers/misc/intel-nnpi/bootimage.h
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index da2863b..e46c89f 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -5,7 +5,8 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
-intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o
+intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o \
+ bootimage.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/bootimage.c b/drivers/misc/intel-nnpi/bootimage.c
new file mode 100644
index 0000000..38b5156
--- /dev/null
+++ b/drivers/misc/intel-nnpi/bootimage.c
@@ -0,0 +1,246 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitfield.h>
+#include <linux/device.h>
+#include <linux/firmware.h>
+#include <linux/fs.h>
+#include <linux/namei.h>
+#include <linux/printk.h>
+
+#include "bootimage.h"
+#include "device.h"
+#include "hostres.h"
+#include "ipc_protocol.h"
+#include "nnp_boot_defs.h"
+
+#define MAX_IMAGE_NAME_LEN (NAME_MAX + 1)
+
+void nnpdev_boot_image_init(struct image_info *boot_image)
+{
+ boot_image->state = IMAGE_NONE;
+ boot_image->hostres = NULL;
+ mutex_init(&boot_image->mutex);
+}
+
+static int load_firmware(struct image_info *image_info)
+{
+ const struct firmware *fw;
+ struct nnp_device *nnpdev = container_of(image_info, struct nnp_device,
+ boot_image);
+ struct device *dev = nnpdev->dev;
+ struct kstat stat;
+ struct path path;
+ static const char *fname = "/lib/firmware/" NNP_FIRMWARE_NAME;
+ void *vptr;
+ int ret;
+
+ lockdep_assert_held(&image_info->mutex);
+
+ /*
+ * find image file size
+ *
+ * NOTE: we look for the file under a constant path "/lib/firmware"
+ * since it works and accepted on all platforms that NNP-I device
+ * can be installed.
+ * A better solution would be to look at the same paths that the
+ * firmware API will search however the firmware API does not
+ * export any function to do the search and there is no point
+ * duplicating it here.
+ */
+ ret = kern_path(fname, LOOKUP_FOLLOW, &path);
+ if (ret) {
+ pr_err("Could not find image under /lib/firmware\n");
+ return ret;
+ }
+
+ ret = vfs_getattr(&path, &stat, STATX_SIZE, 0);
+ path_put(&path);
+ if (ret) {
+ pr_err("Failed to get file size for %s error=%d\n", fname, ret);
+ return ret;
+ }
+
+ /* create host resource to hold the boot image content */
+ image_info->hostres = nnp_hostres_alloc(stat.size, DMA_TO_DEVICE);
+ if (IS_ERR(image_info->hostres))
+ return PTR_ERR(image_info->hostres);
+
+ vptr = nnp_hostres_vptr(image_info->hostres);
+
+ /*
+ * load the image into the host resource.
+ * We load directly to pre-allocated host resource memory
+ * in order to prevent caching of the boot image inside
+ * firmware API
+ */
+ ret = request_firmware_into_buf(&fw, NNP_FIRMWARE_NAME, dev, vptr,
+ stat.size);
+ if (ret) {
+ pr_err("failed to load firmware %s ret=%d\n", fname, ret);
+ nnp_hostres_put(image_info->hostres);
+ image_info->hostres = NULL;
+ return ret;
+ }
+
+ release_firmware(fw);
+ image_info->state = IMAGE_AVAILABLE;
+
+ return 0;
+}
+
+static void load_image_handler(struct work_struct *work)
+{
+ struct image_info *image_info = container_of(work, struct image_info,
+ work);
+ struct nnp_device *nnpdev = container_of(image_info, struct nnp_device,
+ boot_image);
+ dma_addr_t page_list_addr;
+ unsigned int total_chunks;
+ unsigned int image_size;
+ u64 cmd[3];
+ u32 val;
+ int ret;
+
+ mutex_lock(&image_info->mutex);
+
+ /* do not load if image load request has canceled */
+ if (image_info->state != IMAGE_REQUESTED) {
+ mutex_unlock(&image_info->mutex);
+ return;
+ }
+
+ /* load boot image from disk */
+ ret = load_firmware(image_info);
+ if (ret) {
+ image_info->state = IMAGE_LOAD_FAILED;
+ goto fail;
+ }
+
+ /* map image to the device */
+ image_info->hostres_map = nnp_hostres_map_device(image_info->hostres,
+ nnpdev, true,
+ &page_list_addr,
+ &total_chunks);
+ if (IS_ERR(image_info->hostres_map)) {
+ nnp_hostres_put(image_info->hostres);
+ image_info->hostres = NULL;
+ image_info->state = IMAGE_NONE;
+ goto fail;
+ }
+
+ mutex_unlock(&image_info->mutex);
+
+ image_size = (unsigned int)nnp_hostres_size(image_info->hostres);
+
+ /* image successfully mapped - send it to the device to boot */
+ dev_dbg(nnpdev->dev,
+ "Mapped boot image num_chunks=%u total_size=%u\n", total_chunks,
+ image_size);
+
+ /* write image address directly to the command Q */
+ cmd[0] = FIELD_PREP(NNP_H2C_BOOT_IMAGE_READY_QW0_OP_MASK,
+ NNP_IPC_H2C_OP_BIOS_PROTOCOL);
+ cmd[0] |= FIELD_PREP(NNP_H2C_BOOT_IMAGE_READY_QW0_TYPE_MASK,
+ NNP_IPC_H2C_TYPE_BOOT_IMAGE_READY);
+ cmd[0] |= FIELD_PREP(NNP_H2C_BOOT_IMAGE_READY_QW0_SIZE_MASK,
+ 2 * sizeof(u64));
+
+ cmd[1] = (u64)page_list_addr + sizeof(struct nnp_dma_chain_header);
+
+ cmd[2] = FIELD_PREP(NNP_H2C_BOOT_IMAGE_READY_QW2_DESC_SIZE_MASK,
+ total_chunks * sizeof(struct nnp_dma_chain_entry));
+ cmd[2] |= FIELD_PREP(NNP_H2C_BOOT_IMAGE_READY_QW2_IMAGE_SIZE_MASK,
+ image_size);
+
+ nnpdev->ops->cmdq_write_mesg(nnpdev, cmd, 3);
+ return;
+
+fail:
+ /* notify card that boot image cannot be loaded */
+ val = FIELD_PREP(NNP_HOST_ERROR_MASK,
+ NNP_HOST_ERROR_CANNOT_LOAD_IMAGE);
+ nnpdev->ops->set_host_doorbell_value(nnpdev, val);
+ mutex_unlock(&image_info->mutex);
+}
+
+/**
+ * nnpdev_load_boot_image() - load boot image and send it to device
+ * @nnpdev: the device requested the image
+ *
+ * This function starts the flow of loading a boot image and map it to the
+ * requesting device. It will launch a work to load the boot image.
+ * It is an error to call this function if boot image load for the same
+ * device is already in progress.
+ *
+ * Return:
+ * * 0 - boot image was successfully loaded, mapped and sent to the device.
+ * * -EINVAL - image load is already in progress
+ */
+int nnpdev_load_boot_image(struct nnp_device *nnpdev)
+{
+ struct image_info *image_info = &nnpdev->boot_image;
+
+ /* check if the image is already loaded or in progress */
+ mutex_lock(&image_info->mutex);
+ if (image_info->state != IMAGE_NONE) {
+ mutex_unlock(&image_info->mutex);
+ return -EINVAL;
+ }
+
+ /* initialize image load request */
+ image_info->state = IMAGE_REQUESTED;
+ mutex_unlock(&image_info->mutex);
+ INIT_WORK(&image_info->work, load_image_handler);
+
+ /* schedule work to load the image */
+ schedule_work(&image_info->work);
+
+ return 0;
+}
+
+/**
+ * nnpdev_unload_boot_image() - unmaps boot image for device
+ * @nnpdev: the device
+ *
+ * This function is called when the device no longer need the boot image
+ * in memory. either because it was already copied to the device or when
+ * the device is removed during the image load request is in progress.
+ * The function unmaps the device from the host resource.
+ *
+ * Return: error code or zero.
+ */
+int nnpdev_unload_boot_image(struct nnp_device *nnpdev)
+{
+ struct image_info *image_info = &nnpdev->boot_image;
+ int ret = 0;
+
+ mutex_lock(&image_info->mutex);
+ switch (image_info->state) {
+ case IMAGE_NONE:
+ ret = -EINVAL;
+ goto done;
+ case IMAGE_REQUESTED:
+ image_info->state = IMAGE_NONE;
+ mutex_unlock(&image_info->mutex);
+ cancel_work_sync(&image_info->work);
+ return 0;
+ case IMAGE_LOAD_FAILED:
+ case IMAGE_AVAILABLE:
+ break;
+ }
+
+ if (image_info->hostres) {
+ nnp_hostres_unmap_device(image_info->hostres_map);
+ nnp_hostres_put(image_info->hostres);
+ image_info->hostres = NULL;
+ }
+
+ image_info->state = IMAGE_NONE;
+
+done:
+ mutex_unlock(&image_info->mutex);
+ return ret;
+}
diff --git a/drivers/misc/intel-nnpi/bootimage.h b/drivers/misc/intel-nnpi/bootimage.h
new file mode 100644
index 0000000..c5efe46
--- /dev/null
+++ b/drivers/misc/intel-nnpi/bootimage.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_BOOTIMAGE_H
+#define _NNPDRV_BOOTIMAGE_H
+
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+
+struct nnp_device;
+
+enum image_state {
+ IMAGE_NONE = 0,
+ IMAGE_REQUESTED,
+ IMAGE_LOAD_FAILED,
+ IMAGE_AVAILABLE
+};
+
+/**
+ * struct image_info - describes a boot image object
+ * @work: handle for placing the image load in a workqueue
+ * @state: state indicating whether it is loaded or load failed
+ * @mutex: protects accesses to @state and @hostres
+ * @load_fail_err: zero or error code if @state is IMAGE_LOAD_FAILED.
+ * @hostres: host resource object allocated for the image content
+ * @hostres_map: mapping object of host resource to device
+ *
+ * This structure describe a request to load boot image from disk,
+ * there is one such structure for each device.
+ */
+struct image_info {
+ struct work_struct work;
+ enum image_state state;
+ struct mutex mutex;
+ struct host_resource *hostres;
+ struct nnpdev_mapping *hostres_map;
+};
+
+void nnpdev_boot_image_init(struct image_info *boot_image);
+int nnpdev_load_boot_image(struct nnp_device *nnpdev);
+int nnpdev_unload_boot_image(struct nnp_device *nnpdev);
+
+#endif /* _NNPDRV_BOOTIMAGE_H */
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index a3c6a1d..f4fc975 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -3,13 +3,18 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/bitfield.h>
#include <linux/device.h>
#include <linux/idr.h>
#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/printk.h>
+#include "bootimage.h"
#include "device.h"
#include "host_chardev.h"
#include "msg_scheduler.h"
+#include "nnp_boot_defs.h"
static DEFINE_IDA(dev_ida);
@@ -18,6 +23,178 @@ bool nnpdev_no_devices(void)
return ida_is_empty(&dev_ida);
}
+static void send_sysinfo_request_to_bios(struct nnp_device *nnpdev)
+{
+ u64 cmd[3];
+
+ cmd[0] = FIELD_PREP(NNP_H2C_BIOS_SYS_INFO_REQ_QW0_OP_MASK,
+ NNP_IPC_H2C_OP_BIOS_PROTOCOL);
+ cmd[0] |= FIELD_PREP(NNP_H2C_BIOS_SYS_INFO_REQ_QW0_TYPE_MASK,
+ NNP_IPC_H2C_TYPE_SYSTEM_INFO_REQ);
+ cmd[0] |= FIELD_PREP(NNP_H2C_BIOS_SYS_INFO_REQ_QW0_SIZE_MASK,
+ 2 * sizeof(u64));
+
+ cmd[1] = (u64)nnpdev->bios_system_info_dma_addr;
+
+ cmd[2] = FIELD_PREP(NNP_H2C_BIOS_SYS_INFO_REQ_QW2_SIZE_MASK,
+ NNP_PAGE_SIZE);
+
+ nnpdev->ops->cmdq_flush(nnpdev);
+
+ nnpdev->ops->cmdq_write_mesg(nnpdev, cmd, 3);
+}
+
+/**
+ * build_bios_version_string() - builds printable string of bios version string
+ * @nnpdev: pointer to device structure
+ *
+ * Initializes nnpdev->bios_version_str with printable string of bios version
+ * from bios_system_info page.
+ */
+static void build_bios_version_string(struct nnp_device *nnpdev)
+{
+ unsigned int i;
+ __le16 *v;
+
+ if (!nnpdev->bios_system_info)
+ return;
+
+ /*
+ * The bios version string in the bios's system info page
+ * holds __le16 for each character in the version string.
+ * (see struct nnp_c2h_bios_version)
+ * Here we convert it to string of chars by taking only the
+ * LSB from each 16-bit character
+ */
+ v = (__le16 *)&nnpdev->bios_system_info->bios_ver;
+
+ /* check that bios version string is corrected null terminated */
+ if (nnpdev->bios_system_info->bios_ver.null_terminator != 0)
+ return;
+
+ for (i = 0; i < NNP_BIOS_VERSION_LEN - 1 && v[i] != 0; ++i)
+ nnpdev->bios_version_str[i] = v[i];
+
+ nnpdev->bios_version_str[i] = '\0';
+}
+
+static int unload_boot_image(struct nnp_device *nnpdev)
+{
+ nnpdev->boot_image_loaded = false;
+ return nnpdev_unload_boot_image(nnpdev);
+}
+
+/**
+ * nnpdev_set_boot_state() - sets new device state.
+ * @nnpdev: pointer to device structure
+ * @mask: mask of device state bits defined in device.h
+ *
+ * This function sets new device status and handles the state machine of
+ * device boot flow.
+ * It is being called when various device notifications are received or
+ * some error conditions are detected.
+ *
+ * The following flow describes the communication flow with the NNP-I card's
+ * BIOS during the device boot flow, this function gets called when device
+ * state changes when progressing in this flow:
+ * 1) The device report its boot state through the "card doorbell" register,
+ * that signals an interrupt to the host and the "pci" layer in the driver
+ * calls the nnpdev_card_doorbell_value_changed function.
+ * 2) When the device signals that it is "Ready to boot", the host driver
+ * sends it through the "command queue" an address of page in host memory.
+ * 3) The card BIOS fills the page of memory with card system info and change
+ * the doorbell value to "sysinfo ready"
+ * 4) The host driver then initiate the boot image loading.
+ * 5) When boot image is ready in memory, the host driver send a
+ * "Boot image ready" message and the card BIOS starts booting and changes
+ * the doorbell value to indicate success or failure.
+ * 6) When receiving indication about success/failure the host driver signals
+ * that the device no longer needs the boot image in memory.
+ * When all devices no longer need the image it will be removed.
+ */
+void nnpdev_set_boot_state(struct nnp_device *nnpdev, u32 mask)
+{
+ u32 state, prev_state;
+ bool becomes_ready = false;
+ int ret;
+
+ /*
+ * Save previous state and modify current state
+ * with the changed state mask
+ */
+ spin_lock(&nnpdev->lock);
+ prev_state = nnpdev->state;
+ if ((mask & NNP_DEVICE_CARD_BOOT_STATE_MASK) != 0) {
+ /*
+ * When boot state changes previous boot states are reset.
+ * also, device error conditions is cleared.
+ */
+ nnpdev->state &= ~(NNP_DEVICE_CARD_BOOT_STATE_MASK);
+ nnpdev->state &= ~(NNP_DEVICE_ERROR_MASK);
+ }
+ nnpdev->state |= mask;
+ state = nnpdev->state;
+ spin_unlock(&nnpdev->lock);
+
+ dev_dbg(nnpdev->dev,
+ "device state change 0x%x --> 0x%x\n", prev_state, state);
+
+ /* Unload boot image if boot started or failed */
+ if (nnpdev->boot_image_loaded &&
+ (((state & NNP_DEVICE_BOOT_STARTED) &&
+ !(prev_state & NNP_DEVICE_BOOT_STARTED)) ||
+ (state & NNP_DEVICE_BOOT_FAILED))) {
+ ret = unload_boot_image(nnpdev);
+ /* This should never fail */
+ if (ret)
+ dev_dbg(nnpdev->dev,
+ "Unexpected error while unloading boot image. rc=%d\n",
+ ret);
+ }
+
+ /* if in error state - no need to check rest of the states */
+ if (state & NNP_DEVICE_ERROR_MASK)
+ return;
+
+ if ((state & NNP_DEVICE_BOOT_BIOS_READY) &&
+ !(prev_state & NNP_DEVICE_BOOT_BIOS_READY)) {
+ becomes_ready = true;
+ nnpdev->is_recovery_bios = false;
+ }
+
+ if ((state & NNP_DEVICE_BOOT_RECOVERY_BIOS_READY) &&
+ !(prev_state & NNP_DEVICE_BOOT_RECOVERY_BIOS_READY)) {
+ becomes_ready = true;
+ nnpdev->is_recovery_bios = true;
+ }
+
+ if (becomes_ready ||
+ mask == NNP_DEVICE_BOOT_BIOS_READY ||
+ mask == NNP_DEVICE_BOOT_RECOVERY_BIOS_READY) {
+ if (!becomes_ready)
+ dev_dbg(nnpdev->dev, "Re-sending sysinfo page to bios!!\n");
+
+ /* Send request to fill system_info buffer */
+ send_sysinfo_request_to_bios(nnpdev);
+ return;
+ }
+
+ /* Handle boot image request */
+ if ((state & NNP_DEVICE_BOOT_SYSINFO_READY) &&
+ !(prev_state & NNP_DEVICE_BOOT_SYSINFO_READY) &&
+ !nnpdev->boot_image_loaded) {
+ build_bios_version_string(nnpdev);
+ nnpdev->bios_system_info_valid = true;
+ nnpdev->boot_image_loaded = true;
+ ret = nnpdev_load_boot_image(nnpdev);
+
+ if (ret)
+ dev_err(nnpdev->dev,
+ "Unexpected error while loading boot image. rc=%d\n",
+ ret);
+ }
+}
+
/**
* nnpdev_init() - initialize NNP-I device structure.
* @nnpdev: device to be initialized
@@ -64,8 +241,33 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
goto err_msg_sched;
}
+ nnpdev->wq = create_singlethread_workqueue("nnpdev_wq");
+ if (!nnpdev->wq) {
+ ret = -ENOMEM;
+ goto err_cmdq;
+ }
+
+ /* setup memory for bios system info */
+ nnpdev->bios_system_info =
+ dma_alloc_coherent(nnpdev->dev, NNP_PAGE_SIZE,
+ &nnpdev->bios_system_info_dma_addr, GFP_KERNEL);
+ if (!nnpdev->bios_system_info) {
+ ret = -ENOMEM;
+ goto err_wq;
+ }
+
+ /* set host driver state to "Not ready" */
+ nnpdev->ops->set_host_doorbell_value(nnpdev, 0);
+
+ spin_lock_init(&nnpdev->lock);
+ nnpdev_boot_image_init(&nnpdev->boot_image);
+
return 0;
+err_wq:
+ destroy_workqueue(nnpdev->wq);
+err_cmdq:
+ nnp_msched_queue_destroy(nnpdev->cmdq);
err_msg_sched:
nnp_msched_destroy(nnpdev->cmdq_sched);
err_ida:
@@ -74,6 +276,71 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
}
EXPORT_SYMBOL(nnpdev_init);
+struct doorbell_work {
+ struct work_struct work;
+ struct nnp_device *nnpdev;
+ u32 val;
+};
+
+static void doorbell_changed_handler(struct work_struct *work)
+{
+ struct doorbell_work *req = container_of(work, struct doorbell_work,
+ work);
+ u32 boot_state, state = 0;
+ u32 error_state;
+ u32 doorbell_val = req->val;
+ struct nnp_device *nnpdev = req->nnpdev;
+
+ nnpdev->card_doorbell_val = doorbell_val;
+
+ error_state = FIELD_GET(NNP_CARD_ERROR_MASK, doorbell_val);
+ boot_state = FIELD_GET(NNP_CARD_BOOT_STATE_MASK, doorbell_val);
+
+ if (error_state) {
+ state = NNP_DEVICE_BOOT_FAILED;
+
+ switch (error_state) {
+ case NNP_CARD_ERROR_NOT_CAPSULE:
+ state |= NNP_DEVICE_CAPSULE_EXPECTED;
+ break;
+ case NNP_CARD_ERROR_CORRUPTED_IMAGE:
+ state |= NNP_DEVICE_CORRUPTED_BOOT_IMAGE;
+ break;
+ case NNP_CARD_ERROR_CAPSULE_FAILED:
+ state |= NNP_DEVICE_CAPSULE_FAILED;
+ break;
+ default:
+ break;
+ }
+ } else if (boot_state != nnpdev->curr_boot_state) {
+ nnpdev->curr_boot_state = boot_state;
+ switch (boot_state) {
+ case NNP_CARD_BOOT_STATE_BIOS_READY:
+ state = NNP_DEVICE_BOOT_BIOS_READY;
+ break;
+ case NNP_CARD_BOOT_STATE_RECOVERY_BIOS_READY:
+ state = NNP_DEVICE_BOOT_RECOVERY_BIOS_READY;
+ break;
+ case NNP_CARD_BOOT_STATE_BIOS_SYSINFO_READY:
+ state = NNP_DEVICE_BOOT_SYSINFO_READY;
+ break;
+ case NNP_CARD_BOOT_STATE_BOOT_STARTED:
+ state = NNP_DEVICE_BOOT_STARTED;
+ break;
+ case NNP_CARD_BOOT_STATE_BIOS_FLASH_STARTED:
+ state = NNP_DEVICE_BIOS_UPDATE_STARTED;
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (state)
+ nnpdev_set_boot_state(nnpdev, state);
+
+ kfree(req);
+}
+
/**
* nnpdev_card_doorbell_value_changed() - card doorbell changed notification
* @nnpdev: The nnp device
@@ -85,7 +352,18 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
void nnpdev_card_doorbell_value_changed(struct nnp_device *nnpdev,
u32 doorbell_val)
{
+ struct doorbell_work *req;
+
dev_dbg(nnpdev->dev, "Got card doorbell value 0x%x\n", doorbell_val);
+
+ req = kzalloc(sizeof(*req), GFP_KERNEL);
+ if (!req)
+ return;
+
+ req->nnpdev = nnpdev;
+ req->val = doorbell_val;
+ INIT_WORK(&req->work, doorbell_changed_handler);
+ queue_work(nnpdev->wq, &req->work);
}
EXPORT_SYMBOL(nnpdev_card_doorbell_value_changed);
@@ -100,6 +378,18 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
{
dev_dbg(nnpdev->dev, "Destroying NNP-I device\n");
+ /*
+ * If device is removed while boot image load is in-flight,
+ * stop the image load and flag it is not needed.
+ */
+ if (nnpdev->boot_image_loaded)
+ unload_boot_image(nnpdev);
+
+ destroy_workqueue(nnpdev->wq);
+
+ dma_free_coherent(nnpdev->dev, NNP_PAGE_SIZE, nnpdev->bios_system_info,
+ nnpdev->bios_system_info_dma_addr);
+
nnp_msched_destroy(nnpdev->cmdq_sched);
ida_simple_remove(&dev_ida, nnpdev->id);
}
@@ -121,3 +411,4 @@ static void __exit nnp_cleanup(void)
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Intel(R) NNPI Framework");
MODULE_AUTHOR("Intel Corporation");
+MODULE_FIRMWARE(NNP_FIRMWARE_NAME);
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 562bbc4..3745f5c 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -4,8 +4,50 @@
#ifndef _NNPDRV_DEVICE_H
#define _NNPDRV_DEVICE_H
+#include <linux/spinlock.h>
+#include <linux/workqueue.h>
+
+#include "bootimage.h"
+#include "ipc_protocol.h"
+#include "msg_scheduler.h"
+
#define NNP_MAX_DEVS 256
+#define NNP_FIRMWARE_NAME "intel/nnpi/disk.img"
+
+/* device state bits */
+#define NNP_DEVICE_BOOT_BIOS_READY BIT(1)
+#define NNP_DEVICE_BOOT_RECOVERY_BIOS_READY BIT(2)
+#define NNP_DEVICE_BOOT_SYSINFO_READY BIT(3)
+#define NNP_DEVICE_BOOT_STARTED BIT(4)
+#define NNP_DEVICE_BIOS_UPDATE_READY BIT(5)
+#define NNP_DEVICE_BIOS_UPDATE_STARTED BIT(6)
+#define NNP_DEVICE_BIOS_UPDATE_DONE BIT(7)
+#define NNP_DEVICE_CARD_DRIVER_READY BIT(8)
+#define NNP_DEVICE_CARD_READY BIT(9)
+#define NNP_DEVICE_CARD_ENABLED BIT(10)
+
+#define NNP_DEVICE_CARD_BOOT_STATE_MASK GENMASK(9, 1)
+
+#define NNP_DEVICE_ACTIVE_MASK (NNP_DEVICE_CARD_READY | \
+ NNP_DEVICE_CARD_ENABLED)
+
+#define NNP_DEVICE_FAILED_VERSION BIT(16)
+#define NNP_DEVICE_BOOT_FAILED BIT(17)
+#define NNP_DEVICE_HOST_DRIVER_ERROR BIT(18)
+#define NNP_DEVICE_KERNEL_CRASH BIT(20)
+#define NNP_DEVICE_PCI_ERROR BIT(21)
+#define NNP_DEVICE_CARD_IN_RESET BIT(22)
+#define NNP_DEVICE_FATAL_MCE_ERROR BIT(23)
+#define NNP_DEVICE_FATAL_DRAM_ECC_ERROR BIT(24)
+#define NNP_DEVICE_FATAL_ICE_ERROR BIT(25)
+#define NNP_DEVICE_HANG BIT(26)
+#define NNP_DEVICE_PROTOCOL_ERROR BIT(27)
+#define NNP_DEVICE_CAPSULE_EXPECTED BIT(28)
+#define NNP_DEVICE_CAPSULE_FAILED BIT(29)
+#define NNP_DEVICE_CORRUPTED_BOOT_IMAGE BIT(30)
+#define NNP_DEVICE_ERROR_MASK GENMASK(31, 16)
+
/**
* struct nnp_device - structure for NNP-I device info
* @ops: device operations implemented by the underlying device driver
@@ -15,6 +57,18 @@
* submissions to the device's command queue.
* @cmdq: input queue to @cmdq_sched used to schedule driver internal commands
* to be sent to the device.
+ * @wq: singlethread workqueue for processing device's response messages.
+ * @lock: protects accesses to @state
+ * @is_recovery_bios: true if device has booted from the recovery bios flash
+ * @boot_image_loaded: true if boot image load has started
+ * @bios_system_info_dma_addr: dma page allocated for bios system info.
+ * @bios_system_info: virtual pointer to bios system info page
+ * @bios_version_str: the device's started bios version string
+ * @bios_system_info_valid: true if @bios_system_info has been filled and valid
+ * @state: current device boot state mask (see device state bits above)
+ * @curr_boot_state: last boot state field received from device doorbell reg
+ * @card_doorbell_val: last received device doorbell register value.
+ * @boot_image: boot image object used to boot the card
*/
struct nnp_device {
const struct nnp_device_ops *ops;
@@ -23,6 +77,21 @@ struct nnp_device {
struct nnp_msched *cmdq_sched;
struct nnp_msched_queue *cmdq;
+
+ struct workqueue_struct *wq;
+ spinlock_t lock;
+ bool is_recovery_bios;
+ bool boot_image_loaded;
+
+ dma_addr_t bios_system_info_dma_addr;
+ struct nnp_c2h_system_info *bios_system_info;
+ char bios_version_str[NNP_BIOS_VERSION_LEN];
+ bool bios_system_info_valid;
+
+ u32 state;
+ u32 curr_boot_state;
+ u32 card_doorbell_val;
+ struct image_info boot_image;
};
/**
@@ -30,10 +99,12 @@ struct nnp_device {
* @cmdq_flush: empties the device command queue, discarding all queued
* commands.
* @cmdq_write_mesg: inserts a command message to the card's command queue.
+ * @set_host_doorbell_value: change the host doorbell value on device.
*/
struct nnp_device_ops {
int (*cmdq_flush)(struct nnp_device *hw_dev);
int (*cmdq_write_mesg)(struct nnp_device *nnpdev, u64 *msg, u32 size);
+ int (*set_host_doorbell_value)(struct nnp_device *nnpdev, u32 value);
};
bool nnpdev_no_devices(void);
@@ -47,4 +118,10 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
void nnpdev_destroy(struct nnp_device *nnpdev);
void nnpdev_card_doorbell_value_changed(struct nnp_device *nnpdev,
u32 doorbell_val);
+
+/*
+ * Framework internal functions (not exported)
+ */
+void nnpdev_set_boot_state(struct nnp_device *nnpdev, u32 mask);
+
#endif
diff --git a/drivers/misc/intel-nnpi/nnp_pcie.c b/drivers/misc/intel-nnpi/nnp_pcie.c
index 7aa9074..3840a53 100644
--- a/drivers/misc/intel-nnpi/nnp_pcie.c
+++ b/drivers/misc/intel-nnpi/nnp_pcie.c
@@ -375,9 +375,25 @@ static int nnp_cmdq_flush(struct nnp_device *nnpdev)
return 0;
}
+static int nnp_set_host_doorbell_value(struct nnp_device *nnpdev, u32 value)
+{
+ struct nnp_pci *nnp_pci = container_of(nnpdev, struct nnp_pci, nnpdev);
+
+ /*
+ * The SELF_RESET bit is set only by the h/w layer,
+ * do not allow higher layer to set it
+ */
+ value &= ~NNP_HOST_DRV_REQUEST_SELF_RESET_MASK;
+
+ nnp_mmio_write(nnp_pci, ELBI_PCI_HOST_DOORBELL_VALUE, value);
+
+ return 0;
+}
+
static struct nnp_device_ops nnp_device_ops = {
.cmdq_flush = nnp_cmdq_flush,
.cmdq_write_mesg = nnp_cmdq_write_mesg,
+ .set_host_doorbell_value = nnp_set_host_doorbell_value,
};
static void set_host_boot_state(struct nnp_pci *nnp_pci, int boot_state)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 09/15] misc: nnpi: Process device response messages
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (7 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 08/15] misc: nnpi: Boot NNP-I device Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 10/15] misc: nnpi: Query and verify device protocol Guy Zadicario
` (6 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
The nnpdev_process_messages function handles all messages coming
from a NNP-I device. Based on an opcode field attached to each message,
it calls the correct response processing handler. The function is
called from the NNP-I device driver, from a threaded interrupt handler, when
responses arrive in the HW response queue.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/device.c | 136 +++++++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/device.h | 10 +++
drivers/misc/intel-nnpi/nnp_pcie.c | 3 +
3 files changed, 149 insertions(+)
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index f4fc975..2e6ab82 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -23,6 +23,142 @@ bool nnpdev_no_devices(void)
return ida_is_empty(&dev_ida);
}
+/**
+ * handle_bios_protocol() - process response coming from card's BIOS.
+ * @nnpdev: The nnp device
+ * @msgbuf: pointer to response message content
+ * @avail_qwords: number of 64-bit units available in @msgbuf
+ *
+ * IPC protocol with card's BIOS may have different response sizes.
+ * @avail_qwords is the number of 64-bit units available in @msgbuf buffer.
+ * If the actual response size is larger then available data in the buffer,
+ * the function returns 0 to indicate that this is a partial response. Otherwise
+ * the actual response size is returned (in units of qwords).
+ *
+ * Return: 0 if @msgbuf contains a partial response otherwise the number of
+ * qwords of the response in @msgbuf.
+ */
+static int handle_bios_protocol(struct nnp_device *nnpdev, const u64 *msgbuf,
+ int avail_qwords)
+{
+ int msg_size, msg_qwords;
+
+ msg_size = FIELD_GET(NNP_C2H_BIOS_PROTOCOL_TYPE_MASK, msgbuf[0]);
+
+ /* The +1 is because size field does not include header */
+ msg_qwords = DIV_ROUND_UP(msg_size, 8) + 1;
+
+ if (msg_qwords > avail_qwords)
+ return 0;
+
+ return msg_qwords;
+}
+
+typedef int (*response_handler)(struct nnp_device *nnpdev, const u64 *msgbuf,
+ int avail_qwords);
+
+static response_handler resp_handlers[NNP_IPC_C2H_OPCODE_LAST + 1] = {
+ [NNP_IPC_C2H_OP_BIOS_PROTOCOL] = handle_bios_protocol
+};
+
+/**
+ * nnpdev_process_messages() - process response messages from nnpi device
+ * @nnpdev: The nnp device
+ * @hw_msg: pointer to response message content
+ * @hw_nof_msg: number of 64-bit units available in hw_msg buffer.
+ *
+ * This function is called from the PCIe device driver when response messages
+ * are arrived in the HWQ. It is called in sequence, should not be re-entrant.
+ * The function may not block !
+ */
+void nnpdev_process_messages(struct nnp_device *nnpdev, u64 *hw_msg,
+ unsigned int hw_nof_msg)
+{
+ int j = 0;
+ int msg_size;
+ u64 *msg;
+ unsigned int nof_msg;
+ bool fatal_protocol_error = false;
+
+ /* ignore any response if protocol error detected */
+ if ((nnpdev->state & NNP_DEVICE_PROTOCOL_ERROR) != 0)
+ return;
+
+ /*
+ * If we have pending messages from previous round
+ * copy the new messages to the pending list and process
+ * the pending list.
+ * otherwise process the messages received from HW directly
+ */
+ msg = hw_msg;
+ nof_msg = hw_nof_msg;
+ if (nnpdev->response_num_msgs > 0) {
+ /*
+ * Check to prevent response buffer overrun.
+ * This should never happen since the buffer is twice
+ * the size of the HW response queue. This check is
+ * for safety and debug purposes.
+ */
+ if (hw_nof_msg + nnpdev->response_num_msgs >=
+ NNP_DEVICE_RESPONSE_BUFFER_LEN) {
+ dev_dbg(nnpdev->dev,
+ "device response buffer would overrun: %d + %d !!\n",
+ nnpdev->response_num_msgs, hw_nof_msg);
+ return;
+ }
+
+ memcpy(&nnpdev->response_buf[nnpdev->response_num_msgs], hw_msg,
+ hw_nof_msg * sizeof(u64));
+ msg = nnpdev->response_buf;
+ nof_msg = nnpdev->response_num_msgs + hw_nof_msg;
+ }
+
+ /*
+ * loop for each message
+ */
+ do {
+ int op_code = FIELD_GET(NNP_C2H_OP_MASK, msg[j]);
+ response_handler handler;
+
+ /* opcodes above OP_BIOS_PROTOCOL are not yet supported */
+ if (op_code > NNP_IPC_C2H_OP_BIOS_PROTOCOL) {
+ fatal_protocol_error = true;
+ break;
+ }
+
+ /* dispatch the message request */
+ handler = resp_handlers[op_code];
+ if (!handler) {
+ /* Should not happen! */
+ dev_dbg(nnpdev->dev,
+ "Unknown response opcode received %d (0x%llx)\n",
+ op_code, msg[j]);
+ fatal_protocol_error = true;
+ break;
+ }
+
+ msg_size = (*handler)(nnpdev, &msg[j], (nof_msg - j));
+
+ j += msg_size;
+ } while (j < nof_msg || !msg_size);
+
+ if (fatal_protocol_error)
+ nnpdev->state |= NNP_DEVICE_PROTOCOL_ERROR;
+
+ /*
+ * If unprocessed messages left, copy it to the pending messages buffer
+ * for the next time this function will be called.
+ */
+ if (j < nof_msg) {
+ memcpy(&nnpdev->response_buf[0], &msg[j],
+ (nof_msg - j) * sizeof(u64));
+ nnpdev->response_num_msgs = nof_msg - j;
+ } else {
+ nnpdev->response_num_msgs = 0;
+ }
+}
+EXPORT_SYMBOL(nnpdev_process_messages);
+
static void send_sysinfo_request_to_bios(struct nnp_device *nnpdev)
{
u64 cmd[3];
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 3745f5c..b83f67a 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -48,6 +48,9 @@
#define NNP_DEVICE_CORRUPTED_BOOT_IMAGE BIT(30)
#define NNP_DEVICE_ERROR_MASK GENMASK(31, 16)
+#define NNP_DEVICE_RESPONSE_FIFO_LEN 16
+#define NNP_DEVICE_RESPONSE_BUFFER_LEN (NNP_DEVICE_RESPONSE_FIFO_LEN * 2)
+
/**
* struct nnp_device - structure for NNP-I device info
* @ops: device operations implemented by the underlying device driver
@@ -61,6 +64,8 @@
* @lock: protects accesses to @state
* @is_recovery_bios: true if device has booted from the recovery bios flash
* @boot_image_loaded: true if boot image load has started
+ * @response_buf: buffer of device response messages arrived from "pci" layer.
+ * @response_num_msgs: number of qwords available in @response_buf
* @bios_system_info_dma_addr: dma page allocated for bios system info.
* @bios_system_info: virtual pointer to bios system info page
* @bios_version_str: the device's started bios version string
@@ -83,6 +88,9 @@ struct nnp_device {
bool is_recovery_bios;
bool boot_image_loaded;
+ u64 response_buf[NNP_DEVICE_RESPONSE_BUFFER_LEN];
+ unsigned int response_num_msgs;
+
dma_addr_t bios_system_info_dma_addr;
struct nnp_c2h_system_info *bios_system_info;
char bios_version_str[NNP_BIOS_VERSION_LEN];
@@ -118,6 +126,8 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
void nnpdev_destroy(struct nnp_device *nnpdev);
void nnpdev_card_doorbell_value_changed(struct nnp_device *nnpdev,
u32 doorbell_val);
+void nnpdev_process_messages(struct nnp_device *nnpdev, u64 *hw_msg,
+ unsigned int hw_nof_msg);
/*
* Framework internal functions (not exported)
diff --git a/drivers/misc/intel-nnpi/nnp_pcie.c b/drivers/misc/intel-nnpi/nnp_pcie.c
index 3840a53..4c9b4c6 100644
--- a/drivers/misc/intel-nnpi/nnp_pcie.c
+++ b/drivers/misc/intel-nnpi/nnp_pcie.c
@@ -150,6 +150,9 @@ static void nnp_process_commands(struct nnp_pci *nnp_pci)
response_pci_control |= FIELD_PREP(RESPQ_READ_PTR_MASK, read_pointer);
nnp_mmio_write(nnp_pci, ELBI_RESPONSE_PCI_CONTROL,
response_pci_control);
+
+ nnpdev_process_messages(&nnp_pci->nnpdev, nnp_pci->response_buf,
+ avail_slots);
}
static void mask_all_interrupts(struct nnp_pci *nnp_pci)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 10/15] misc: nnpi: Query and verify device protocol
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (8 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 09/15] misc: nnpi: Process device response messages Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 11/15] misc: nnpi: Create comm channel from app to device Guy Zadicario
` (5 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Check that the card booted SW stack is compatible with the driver
running on the host. The IPC protocol between the driver and cards's
SW stack may change from time to time when releasing new versions of
the card boot image.
When the card boots and signals to the host that it is booted and
ready (through doorbell change interrupt), the driver sends
it a "query version" command. In response, the device sends back a
"query version reply" response with information of the IPC protocol
version which it supports. Only when the version check passes, the
device is considered booted and ready for operation.
If the version check fails, the device is put in error state. In order
to recover from this error condition, a device reset is required
(probably preceded by installation of a new, compatile boot image).
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/device.c | 145 +++++++++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/device.h | 17 +++++
2 files changed, 162 insertions(+)
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 2e6ab82..7bcab563 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -24,6 +24,119 @@ bool nnpdev_no_devices(void)
}
/**
+ * process_query_version_reply() - process a "query_version_reply" response
+ * @work: work struct of the calling work
+ *
+ * This function processes a "query_version_reply" response message from
+ * the card which is sent as reply to query_version command submitted
+ * earlier.
+ * The function checks that the IPC protocol version that is supported by the
+ * device matches the one supported by the driver. If there is no match the
+ * device state is put in error.
+ * There are two IPC protocol versions which are checked:
+ * 'protocol_version': is IPC protocol version of command and response messages
+ * That are built (for commands) and processed by this kernel mode
+ * driver. The protocol is defined in ipc_include/ipc_protocol.h
+ * A mismatch is possible in cases that the device has booted with
+ * a wrong/older version of the card boot image.
+ * 'chan_protocol_version': is IPC protocol of command and responses which are
+ * supported by the device but are built and processed in user-space.
+ * The structure of the commands and responses are mostly opaque to
+ * the kernel mode driver. This separation allows to update the
+ * device boot image and user-space library to support new sets
+ * of commands without changing the kernel driver.
+ * The restriction for such commands and responses is that the lowest
+ * 16-bits of the command/response are defined to include the
+ * command/response opcode and the channel id.
+ * The kernel driver should also know for each possible command and
+ * response opcode the size of the message. This info is received
+ * from the device within this "query_version_reply" response
+ * encoded in the chan_resp_op_size and chan_cmd_op_size fields
+ * of the response.
+ */
+static void process_query_version_reply(struct work_struct *work)
+{
+ struct query_version_work *query_version_work;
+ struct nnp_device *nnpdev;
+ u32 protocol_version;
+ u32 card_boot_state;
+ u32 val;
+
+ query_version_work =
+ container_of(work, struct query_version_work, work);
+ nnpdev = container_of(query_version_work, struct nnp_device,
+ query_version_work);
+ protocol_version = NNP_IPC_PROTOCOL_VERSION;
+ card_boot_state = FIELD_GET(NNP_CARD_BOOT_STATE_MASK,
+ nnpdev->card_doorbell_val);
+
+ nnpdev->protocol_version =
+ query_version_work->protocol_version;
+ nnpdev->chan_protocol_version =
+ query_version_work->chan_protocol_version;
+
+ /*
+ * NOTE: The card firmware and host driver protocol version must
+ * exactly match in the major and minor version components.
+ * There is no backwards compatibility on the protocol!
+ * When a device is put in a protocol version error state, the
+ * user must install a matching device firmware and reset the device
+ * in order to allow the device to function.
+ */
+ if (NNP_VERSION_MAJOR(query_version_work->protocol_version) !=
+ NNP_VERSION_MAJOR(protocol_version) ||
+ NNP_VERSION_MINOR(query_version_work->protocol_version) !=
+ NNP_VERSION_MINOR(protocol_version) ||
+ query_version_work->chan_resp_op_size == 0) {
+ nnpdev_set_boot_state(nnpdev, NNP_DEVICE_FAILED_VERSION);
+ /* set host driver state in doorbell register */
+ val = FIELD_PREP(NNP_HOST_DRV_STATE_MASK,
+ NNP_HOST_DRV_STATE_VERSION_ERROR);
+ nnpdev->ops->set_host_doorbell_value(nnpdev, val);
+ } else if (card_boot_state == NNP_CARD_BOOT_STATE_DRV_READY) {
+ nnpdev_set_boot_state(nnpdev, NNP_DEVICE_CARD_DRIVER_READY);
+ } else if (card_boot_state == NNP_CARD_BOOT_STATE_CARD_READY) {
+ /* Card driver finished initialization */
+ nnpdev_set_boot_state(nnpdev,
+ NNP_DEVICE_CARD_DRIVER_READY |
+ NNP_DEVICE_CARD_READY |
+ NNP_DEVICE_CARD_ENABLED);
+ }
+
+ query_version_work->running = false;
+}
+
+static int handle_query_version_reply3(struct nnp_device *nnpdev,
+ const u64 *msgbuf, int avail_qwords)
+{
+ int msg_qwords = 3; /* QUERY_VERSION_REPLY3 response len is 3 qwords */
+
+ if (avail_qwords < msg_qwords)
+ return 0;
+
+ /*
+ * This should not happen, but if it does, just ignore the message
+ * There is no fear in race condition on "running" flag as only
+ * single version reply message should be processed after each
+ * device reset.
+ */
+ if (nnpdev->query_version_work.running)
+ return msg_qwords;
+
+ nnpdev->query_version_work.running = true;
+ nnpdev->query_version_work.protocol_version =
+ FIELD_GET(NNP_C2H_VERSION_REPLY_QW0_PROT_VER_MASK, msgbuf[0]);
+ nnpdev->query_version_work.chan_protocol_version =
+ FIELD_GET(NNP_C2H_VERSION_REPLY_QW0_CHAN_VER_MASK, msgbuf[0]);
+ nnpdev->query_version_work.chan_resp_op_size = msgbuf[1];
+ nnpdev->query_version_work.chan_cmd_op_size = msgbuf[2];
+
+ queue_work(nnpdev->wq, &nnpdev->query_version_work.work);
+
+ return msg_qwords;
+}
+
+/**
* handle_bios_protocol() - process response coming from card's BIOS.
* @nnpdev: The nnp device
* @msgbuf: pointer to response message content
@@ -58,6 +171,7 @@ typedef int (*response_handler)(struct nnp_device *nnpdev, const u64 *msgbuf,
int avail_qwords);
static response_handler resp_handlers[NNP_IPC_C2H_OPCODE_LAST + 1] = {
+ [NNP_IPC_C2H_OP_QUERY_VERSION_REPLY3] = handle_query_version_reply3,
[NNP_IPC_C2H_OP_BIOS_PROTOCOL] = handle_bios_protocol
};
@@ -329,6 +443,18 @@ void nnpdev_set_boot_state(struct nnp_device *nnpdev, u32 mask)
"Unexpected error while loading boot image. rc=%d\n",
ret);
}
+
+ /* Handle transition to active state */
+ if (((state & NNP_DEVICE_CARD_DRIVER_READY) ||
+ (state & NNP_DEVICE_CARD_READY)) &&
+ !(prev_state & NNP_DEVICE_CARD_DRIVER_READY) &&
+ !(prev_state & NNP_DEVICE_CARD_READY)) {
+ u32 val;
+
+ /* set host driver state to "Driver ready" */
+ val = FIELD_PREP(NNP_HOST_DRV_STATE_MASK, NNP_HOST_DRV_STATE_READY);
+ nnpdev->ops->set_host_doorbell_value(nnpdev, val);
+ }
}
/**
@@ -364,6 +490,7 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
*/
nnpdev->dev = dev;
nnpdev->ops = ops;
+ nnpdev->protocol_version = 0;
nnpdev->cmdq_sched = nnp_msched_create(nnpdev);
if (!nnpdev->cmdq_sched) {
@@ -397,6 +524,7 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
spin_lock_init(&nnpdev->lock);
nnpdev_boot_image_init(&nnpdev->boot_image);
+ INIT_WORK(&nnpdev->query_version_work.work, process_query_version_reply);
return 0;
@@ -426,6 +554,7 @@ static void doorbell_changed_handler(struct work_struct *work)
u32 error_state;
u32 doorbell_val = req->val;
struct nnp_device *nnpdev = req->nnpdev;
+ u64 query_cmd;
nnpdev->card_doorbell_val = doorbell_val;
@@ -466,6 +595,22 @@ static void doorbell_changed_handler(struct work_struct *work)
case NNP_CARD_BOOT_STATE_BIOS_FLASH_STARTED:
state = NNP_DEVICE_BIOS_UPDATE_STARTED;
break;
+ case NNP_CARD_BOOT_STATE_DRV_READY:
+ case NNP_CARD_BOOT_STATE_CARD_READY:
+ /* card is up - send "query_version" command */
+ query_cmd = FIELD_PREP(NNP_H2C_OP_MASK,
+ NNP_IPC_H2C_OP_QUERY_VERSION);
+ if (nnp_msched_queue_msg(nnpdev->cmdq, query_cmd) ||
+ nnp_msched_queue_sync(nnpdev->cmdq))
+ dev_err(nnpdev->dev, "Query version msg error\n");
+ break;
+
+ case NNP_CARD_BOOT_STATE_NOT_READY:
+ /* card is down reset the device boot and error state */
+ spin_lock(&nnpdev->lock);
+ nnpdev->state = 0;
+ spin_unlock(&nnpdev->lock);
+ break;
default:
break;
}
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index b83f67a..63bc54d 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -51,6 +51,15 @@
#define NNP_DEVICE_RESPONSE_FIFO_LEN 16
#define NNP_DEVICE_RESPONSE_BUFFER_LEN (NNP_DEVICE_RESPONSE_FIFO_LEN * 2)
+struct query_version_work {
+ struct work_struct work;
+ u64 chan_resp_op_size;
+ u64 chan_cmd_op_size;
+ u16 protocol_version;
+ u16 chan_protocol_version;
+ bool running;
+};
+
/**
* struct nnp_device - structure for NNP-I device info
* @ops: device operations implemented by the underlying device driver
@@ -71,9 +80,13 @@
* @bios_version_str: the device's started bios version string
* @bios_system_info_valid: true if @bios_system_info has been filled and valid
* @state: current device boot state mask (see device state bits above)
+ * @protocol_version: version of host->card IPC protocol
+ * @chan_protocol_version: version of user-space->card IPC protocol
* @curr_boot_state: last boot state field received from device doorbell reg
* @card_doorbell_val: last received device doorbell register value.
* @boot_image: boot image object used to boot the card
+ * @query_version_work: work struct used to schedule processing of version
+ * reply response message arrived from card.
*/
struct nnp_device {
const struct nnp_device_ops *ops;
@@ -98,8 +111,12 @@ struct nnp_device {
u32 state;
u32 curr_boot_state;
+ unsigned short protocol_version;
+ unsigned short chan_protocol_version;
u32 card_doorbell_val;
struct image_info boot_image;
+
+ struct query_version_work query_version_work;
};
/**
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 11/15] misc: nnpi: Create comm channel from app to device
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (9 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 10/15] misc: nnpi: Query and verify device protocol Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 12/15] misc: nnpi: Route device response messages Guy Zadicario
` (4 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Establish bi-directional communication channels between user-mode
processes and NNP-I devices. Each command channel object ("channel")
holds a queue of messages from a single user-mode connection to a
single NNP-I device, as well as a ring-buffer to hold response messages
from this NNP-I device back to the user-mode process.
Messages to the NNP-I device are put by the channel into a command
queue. Response messages coming back from the device are being routed
to the intended channel's ring-buffer, where they are consumed by this
channel's user-mode connection. Routing of messages to and from the
device is done based on a channel's 10-bit unique id, which is included
in the messages.
The interface for consuming responses from the ring-buffer and writing
command messages into the msg_scheduler's command queue will be added
in a future patch. This patch only adds the channel creation code and
response message routing to the targeted channel.
When creating a "command channel", the user should give an open file
descriptor to the /dev/nnpi_host device. This file descriptor
associates the channel with a particular "nnp_user" object. The channel
can only reference host resources created by that "user".
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/Makefile | 2 +-
drivers/misc/intel-nnpi/cmd_chan.c | 314 +++++++++++++++++++++
drivers/misc/intel-nnpi/cmd_chan.h | 72 +++++
drivers/misc/intel-nnpi/device.c | 113 +++++++-
drivers/misc/intel-nnpi/device.h | 17 +-
drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h | 3 +
6 files changed, 516 insertions(+), 5 deletions(-)
create mode 100644 drivers/misc/intel-nnpi/cmd_chan.c
create mode 100644 drivers/misc/intel-nnpi/cmd_chan.h
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index e46c89f..b3bab2a 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -6,7 +6,7 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o \
- bootimage.o
+ bootimage.o cmd_chan.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/cmd_chan.c b/drivers/misc/intel-nnpi/cmd_chan.c
new file mode 100644
index 0000000..b5518e0
--- /dev/null
+++ b/drivers/misc/intel-nnpi/cmd_chan.c
@@ -0,0 +1,314 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/anon_inodes.h>
+#include <linux/dev_printk.h>
+#include <linux/file.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+
+#include "cmd_chan.h"
+#include "host_chardev.h"
+#include "ipc_protocol.h"
+#include "nnp_user.h"
+
+#define RESPQ_INIT_BUF_SIZE SZ_2K /* must be power of 2 */
+#define RESPQ_MAX_BUF_SIZE SZ_1M /* must be power of 2 */
+
+static inline int respq_free_bytes(struct nnp_chan *chan)
+{
+ return CIRC_SPACE(chan->respq.head, chan->respq.tail, chan->respq_size);
+}
+
+static inline void respq_push(struct nnp_chan *chan, void *buf, int count)
+{
+ char *dst = chan->respq.buf + chan->respq.head;
+ int t = CIRC_SPACE_TO_END(chan->respq.head, chan->respq.tail,
+ chan->respq_size);
+
+ if (t >= count) {
+ memcpy(dst, buf, count);
+ } else {
+ memcpy(dst, buf, t);
+ memcpy(chan->respq.buf, (u8 *)buf + t, count - t);
+ }
+ chan->respq.head = (chan->respq.head + count) & (chan->respq_size - 1);
+}
+
+static inline void respq_pop(struct nnp_chan *chan, void *buf, int count)
+{
+ char *src = chan->respq.buf + chan->respq.tail;
+ int t = CIRC_CNT_TO_END(chan->respq.head, chan->respq.tail,
+ chan->respq_size);
+
+ if (t >= count) {
+ memcpy(buf, src, count);
+ } else {
+ memcpy(buf, src, t);
+ memcpy((u8 *)buf + t, chan->respq.buf, count - t);
+ }
+ chan->respq.tail = (chan->respq.tail + count) & (chan->respq_size - 1);
+}
+
+/**
+ * nnpdev_chan_create() - creates a command channel object
+ * @nnpdev: the device
+ * @host_fd: opened file descriptor to "/dev/nnpi_host"
+ * @min_id: minimum range for allocating ipc channel id for that channel
+ * @max_id: maximum range for allocating ipc channel id for that channel
+ * @get_device_events: true if this channel needs to receive device-level
+ * responses (not originated to specific channel).
+ *
+ * This function create a "command channel" and assign it a unique id within
+ * the range [@min_id..@max_id]. channels in id range [0, 255] are assumed to be
+ * used for inference related operations and have slightly special semantics.
+ *
+ * Return: pointer to created channel or error.
+ */
+struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
+ unsigned int min_id, unsigned int max_id,
+ bool get_device_events)
+{
+ struct nnp_chan *cmd_chan;
+ int chan_id;
+ int ret;
+ unsigned int max_proto_id = BIT(NNP_IPC_CHANNEL_BITS) - 1;
+
+ if (min_id > max_proto_id)
+ return ERR_PTR(-EINVAL);
+ if (max_id > max_proto_id)
+ max_id = max_proto_id;
+ if (max_id < min_id)
+ return ERR_PTR(-EINVAL);
+
+ ret = ida_simple_get(&nnpdev->cmd_chan_ida, min_id, max_id + 1,
+ GFP_KERNEL);
+ if (ret < 0)
+ return ERR_PTR(ret);
+ chan_id = ret;
+
+ cmd_chan = kzalloc(sizeof(*cmd_chan), GFP_KERNEL);
+ if (!cmd_chan) {
+ ret = -ENOMEM;
+ goto err_ida;
+ }
+
+ cmd_chan->respq_buf = kmalloc(RESPQ_INIT_BUF_SIZE, GFP_KERNEL);
+ if (!cmd_chan->respq_buf) {
+ ret = -ENOMEM;
+ goto err_alloc;
+ }
+ cmd_chan->respq_size = RESPQ_INIT_BUF_SIZE;
+ cmd_chan->respq.buf = cmd_chan->respq_buf;
+ spin_lock_init(&cmd_chan->respq_lock);
+
+ cmd_chan->host_file = nnp_host_file_get(host_fd);
+ if (!cmd_chan->host_file) {
+ ret = -EINVAL;
+ goto err_respq;
+ }
+
+ cmd_chan->cmdq = nnp_msched_queue_create(nnpdev->cmdq_sched);
+ if (!cmd_chan->cmdq) {
+ ret = -ENOMEM;
+ goto err_file_get;
+ }
+
+ kref_init(&cmd_chan->ref);
+ cmd_chan->chan_id = chan_id;
+ cmd_chan->nnpdev = nnpdev;
+ cmd_chan->get_device_events = get_device_events;
+
+ cmd_chan->nnp_user = cmd_chan->host_file->private_data;
+ nnp_user_get(cmd_chan->nnp_user);
+
+ init_waitqueue_head(&cmd_chan->resp_waitq);
+ mutex_init(&cmd_chan->dev_mutex);
+
+ /*
+ * Add channel to the channel hash
+ */
+ spin_lock(&nnpdev->lock);
+ hash_add(nnpdev->cmd_chan_hash, &cmd_chan->hash_node, cmd_chan->chan_id);
+
+ spin_unlock(&nnpdev->lock);
+
+ return cmd_chan;
+
+err_file_get:
+ fput(cmd_chan->host_file);
+err_respq:
+ kfree(cmd_chan->respq_buf);
+err_alloc:
+ kfree(cmd_chan);
+err_ida:
+ ida_simple_remove(&nnpdev->cmd_chan_ida, chan_id);
+ return ERR_PTR(ret);
+}
+
+static void nnp_chan_release(struct kref *kref)
+{
+ struct nnp_chan *cmd_chan = container_of(kref, struct nnp_chan, ref);
+
+ nnp_chan_disconnect(cmd_chan);
+
+ nnp_user_put(cmd_chan->nnp_user);
+
+ kfree(cmd_chan->respq_buf);
+ kfree(cmd_chan);
+}
+
+void nnp_chan_get(struct nnp_chan *cmd_chan)
+{
+ kref_get(&cmd_chan->ref);
+}
+
+void nnp_chan_put(struct nnp_chan *cmd_chan)
+{
+ kref_put(&cmd_chan->ref, nnp_chan_release);
+}
+
+/**
+ * nnp_chan_disconnect() - disconnect the channel from the NNP-I device object
+ * @cmd_chan: the command channel object
+ *
+ * This function is called when the channel is released or the NNP-I device is
+ * being removed. It disconnect the channel from the nnp_device object.
+ * A disconnected channel can no longer become connected again and cannot
+ * be used to communicate with any device.
+ */
+void nnp_chan_disconnect(struct nnp_chan *cmd_chan)
+{
+ struct nnp_device *nnpdev;
+
+ mutex_lock(&cmd_chan->dev_mutex);
+ if (!cmd_chan->nnpdev) {
+ mutex_unlock(&cmd_chan->dev_mutex);
+ return;
+ }
+
+ nnpdev = cmd_chan->nnpdev;
+ cmd_chan->nnpdev = NULL;
+ spin_lock(&nnpdev->lock);
+ hash_del(&cmd_chan->hash_node);
+ spin_unlock(&nnpdev->lock);
+ mutex_unlock(&cmd_chan->dev_mutex);
+
+ nnp_msched_queue_sync(cmd_chan->cmdq);
+ nnp_msched_queue_destroy(cmd_chan->cmdq);
+
+ ida_simple_remove(&nnpdev->cmd_chan_ida, cmd_chan->chan_id);
+}
+
+static int resize_respq(struct nnp_chan *cmd_chan)
+{
+ unsigned int avail_size;
+ unsigned int new_size;
+ char *new_buf;
+
+ new_size = min_t(unsigned int, cmd_chan->respq_size * 2, RESPQ_MAX_BUF_SIZE);
+
+ /* do not try to resize if already in maximum size */
+ if (new_size == cmd_chan->respq_size)
+ return -ENOMEM;
+
+ new_buf = kmalloc(new_size, GFP_KERNEL);
+ if (!new_buf)
+ return -ENOMEM;
+
+ /* copy data from old to new ring buffer */
+ spin_lock(&cmd_chan->respq_lock);
+ avail_size = CIRC_CNT(cmd_chan->respq.head, cmd_chan->respq.tail,
+ cmd_chan->respq_size);
+ if (avail_size > 0)
+ respq_pop(cmd_chan, new_buf, avail_size);
+ kfree(cmd_chan->respq_buf);
+ cmd_chan->respq_buf = new_buf;
+ cmd_chan->respq_size = new_size;
+ cmd_chan->respq.buf = cmd_chan->respq_buf;
+ cmd_chan->respq.tail = 0;
+ cmd_chan->respq.head = avail_size;
+ spin_unlock(&cmd_chan->respq_lock);
+ dev_dbg(cmd_chan->nnpdev->dev, "channel respq resized to %d\n", new_size);
+
+ return 0;
+}
+
+/**
+ * try_add_response() - adds a response message to respq if enough space exist
+ * @cmd_chan: the command channel object
+ * @hw_msg: response message arrived from device
+ * @size: size in bytes of the response
+ *
+ * Return: zero on success, -ENOSPC if message does not fit
+ */
+static int try_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size)
+{
+ spin_lock(&cmd_chan->respq_lock);
+
+ /* Would the response fit in the buffer? */
+ if (respq_free_bytes(cmd_chan) < size + sizeof(size)) {
+ spin_unlock(&cmd_chan->respq_lock);
+ return -ENOSPC;
+ }
+
+ /* push the response message to the ring buffer */
+ respq_push(cmd_chan, &size, sizeof(size));
+ respq_push(cmd_chan, hw_msg, size);
+
+ spin_unlock(&cmd_chan->respq_lock);
+
+ wake_up_all(&cmd_chan->resp_waitq);
+
+ return 0;
+}
+
+/**
+ * nnp_chan_add_response() - adds a response message targeting this channel
+ * @cmd_chan: the command channel object
+ * @hw_msg: response message arrived from device
+ * @size: size in bytes of the response
+ *
+ * This function is being called when a response arrived from the NNP-I card
+ * which targets to a specific command channel object.
+ * The function puts the response message in a ring buffer and will later be
+ * consumed by user space through a call to read(2) on the channel' file
+ * descriptor.
+ *
+ * Return: error code or zero on success.
+ */
+int nnp_chan_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size)
+{
+ int ret;
+
+retry:
+ ret = try_add_response(cmd_chan, hw_msg, size);
+ if (ret == -ENOSPC) {
+ /*
+ * This should *rarely* happen in normal system
+ * operation since the ring-buffer is big enough.
+ * We will get here only if the user application sleeps
+ * for a *very* long time without draining the responses.
+ * Try to resize the response buffer when it does
+ * happen, but up to a maximum value.
+ * If resize failed, we have no choice but to lose the
+ * response. Only the application that uses that channel
+ * will get affected.
+ */
+ ret = resize_respq(cmd_chan);
+ if (!ret)
+ goto retry;
+ }
+
+ if (ret) {
+ if (!cmd_chan->resp_lost)
+ dev_err_ratelimited(cmd_chan->nnpdev->dev,
+ "Response queue full for channel %d losing response!\n",
+ cmd_chan->chan_id);
+ cmd_chan->resp_lost++;
+ }
+
+ return ret;
+}
diff --git a/drivers/misc/intel-nnpi/cmd_chan.h b/drivers/misc/intel-nnpi/cmd_chan.h
new file mode 100644
index 0000000..8cb1a5e
--- /dev/null
+++ b/drivers/misc/intel-nnpi/cmd_chan.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef NNPDRV_CMD_CHAN_H
+#define NNPDRV_CMD_CHAN_H
+
+#include <linux/circ_buf.h>
+#include <linux/hashtable.h>
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+
+#include "device.h"
+
+/**
+ * struct nnp_chan - structure object for user<->device communication
+ * @ref: refcount for this object
+ * @nnpdev: the device this channel is connected to. May be NULL after device
+ * disconnects (on device removal or reset).
+ * @chan_id: the ipc channel id for this channel
+ * @hash_node: node to include this object in list of channels
+ * hash is in (cmd_chan_hash in nnp_device).
+ * @get_device_events: true if device-level events received from card should
+ * be sent over this channel to user.
+ * @cmdq: message queue added to msg_scheduler, for user commands to be sent
+ * to the device.
+ * @host_file: reference to opened "/dev/nnpi_host" object which defines the
+ * nnp_user object this channel connects to.
+ * @nnp_user: the nnp_user this channel belongs to.
+ * the channel can reference host resources created by this
+ * nnp_user object.
+ * @dev_mutex: protects @nnpdev
+ * @resp_waitq: waitqueue used for waiting for response messages be available.
+ * @respq: circular buffer object that receive response messages from device.
+ * @respq_lock: protects @respq
+ * @respq_buf: buffer space allocated for circular response buffer.
+ * @respq_size: current allocated size of circular response buffer.
+ * @resp_lost: number of response messages lost due to response buffer full.
+ */
+struct nnp_chan {
+ struct kref ref;
+ struct nnp_device *nnpdev;
+ u16 chan_id;
+ struct hlist_node hash_node;
+ bool get_device_events;
+
+ struct nnp_msched_queue *cmdq;
+ struct file *host_file;
+ struct nnp_user_info *nnp_user;
+
+ struct mutex dev_mutex;
+ wait_queue_head_t resp_waitq;
+
+ struct circ_buf respq;
+ spinlock_t respq_lock;
+ char *respq_buf;
+ unsigned int respq_size;
+ unsigned int resp_lost;
+};
+
+struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
+ unsigned int min_id, unsigned int max_id,
+ bool get_device_events);
+
+void nnp_chan_get(struct nnp_chan *cmd_chan);
+void nnp_chan_put(struct nnp_chan *cmd_chan);
+void nnp_chan_disconnect(struct nnp_chan *cmd_chan);
+
+int nnp_chan_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size);
+
+#endif
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 7bcab563..3902876 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -11,6 +11,7 @@
#include <linux/printk.h>
#include "bootimage.h"
+#include "cmd_chan.h"
#include "device.h"
#include "host_chardev.h"
#include "msg_scheduler.h"
@@ -61,6 +62,9 @@ static void process_query_version_reply(struct work_struct *work)
u32 protocol_version;
u32 card_boot_state;
u32 val;
+ u64 chan_resp_op_size;
+ u64 chan_cmd_op_size;
+ int i;
query_version_work =
container_of(work, struct query_version_work, work);
@@ -70,6 +74,15 @@ static void process_query_version_reply(struct work_struct *work)
card_boot_state = FIELD_GET(NNP_CARD_BOOT_STATE_MASK,
nnpdev->card_doorbell_val);
+ chan_resp_op_size = query_version_work->chan_resp_op_size;
+ chan_cmd_op_size = query_version_work->chan_cmd_op_size;
+ for (i = 0; i < NNP_IPC_NUM_USER_OPS; i++) {
+ nnpdev->ipc_chan_resp_op_size[i] = chan_resp_op_size & 0x3;
+ chan_resp_op_size >>= 2;
+ nnpdev->ipc_chan_cmd_op_size[i] = chan_cmd_op_size & 0x3;
+ chan_cmd_op_size >>= 2;
+ }
+
nnpdev->protocol_version =
query_version_work->protocol_version;
nnpdev->chan_protocol_version =
@@ -167,6 +180,38 @@ static int handle_bios_protocol(struct nnp_device *nnpdev, const u64 *msgbuf,
return msg_qwords;
}
+struct nnp_chan *nnpdev_find_channel(struct nnp_device *nnpdev, u16 chan_id)
+{
+ struct nnp_chan *cmd_chan;
+
+ spin_lock(&nnpdev->lock);
+ hash_for_each_possible(nnpdev->cmd_chan_hash, cmd_chan, hash_node, chan_id)
+ if (cmd_chan->chan_id == chan_id) {
+ nnp_chan_get(cmd_chan);
+ spin_unlock(&nnpdev->lock);
+ return cmd_chan;
+ }
+ spin_unlock(&nnpdev->lock);
+
+ return NULL;
+}
+
+static void disconnect_all_channels(struct nnp_device *nnpdev)
+{
+ struct nnp_chan *cmd_chan;
+ int i;
+
+restart:
+ spin_lock(&nnpdev->lock);
+ hash_for_each(nnpdev->cmd_chan_hash, i, cmd_chan, hash_node) {
+ spin_unlock(&nnpdev->lock);
+ nnp_chan_disconnect(cmd_chan);
+ nnp_chan_put(cmd_chan);
+ goto restart;
+ }
+ spin_unlock(&nnpdev->lock);
+}
+
typedef int (*response_handler)(struct nnp_device *nnpdev, const u64 *msgbuf,
int avail_qwords);
@@ -175,6 +220,50 @@ typedef int (*response_handler)(struct nnp_device *nnpdev, const u64 *msgbuf,
[NNP_IPC_C2H_OP_BIOS_PROTOCOL] = handle_bios_protocol
};
+static int dispatch_chan_message(struct nnp_device *nnpdev, u64 *hw_msg,
+ unsigned int size)
+{
+ int op_code = FIELD_GET(NNP_C2H_CHAN_MSG_OP_MASK, hw_msg[0]);
+ int chan_id = FIELD_GET(NNP_C2H_CHAN_MSG_CHAN_ID_MASK, hw_msg[0]);
+ struct nnp_chan *chan;
+ int msg_size;
+
+ if (op_code < NNP_IPC_MIN_USER_OP ||
+ op_code > NNP_IPC_MAX_USER_OP) {
+ /* Should not happen! */
+ dev_err(nnpdev->dev,
+ "chan response opcode out-of-range received %d (0x%llx)\n",
+ op_code, *hw_msg);
+ return -EINVAL;
+ }
+
+ msg_size = nnpdev->ipc_chan_resp_op_size[op_code - NNP_IPC_MIN_USER_OP];
+ if (msg_size == 0) {
+ /* Should not happen! */
+ dev_err(nnpdev->dev,
+ "Unknown response chan opcode received %d (0x%llx)\n",
+ op_code, *hw_msg);
+ return -EINVAL;
+ }
+
+ /* Check for partial message */
+ if (size < msg_size)
+ return -ENOSPC;
+
+ chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (!chan) {
+ dev_err(nnpdev->dev,
+ "Got response for invalid channel chan_id=%d 0x%llx\n",
+ chan_id, *hw_msg);
+ return msg_size;
+ }
+
+ nnp_chan_add_response(chan, hw_msg, msg_size * 8);
+ nnp_chan_put(chan);
+
+ return msg_size;
+}
+
/**
* nnpdev_process_messages() - process response messages from nnpi device
* @nnpdev: The nnp device
@@ -234,10 +323,18 @@ void nnpdev_process_messages(struct nnp_device *nnpdev, u64 *hw_msg,
int op_code = FIELD_GET(NNP_C2H_OP_MASK, msg[j]);
response_handler handler;
- /* opcodes above OP_BIOS_PROTOCOL are not yet supported */
+ /* opcodes above OP_BIOS_PROTOCOL are routed to a channel */
if (op_code > NNP_IPC_C2H_OP_BIOS_PROTOCOL) {
- fatal_protocol_error = true;
- break;
+ msg_size = dispatch_chan_message(nnpdev, &msg[j],
+ nof_msg - j);
+ if (msg_size < 0) {
+ if (msg_size != -ENOSPC)
+ fatal_protocol_error = true;
+ break;
+ }
+
+ j += msg_size;
+ continue;
}
/* dispatch the message request */
@@ -492,6 +589,11 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
nnpdev->ops = ops;
nnpdev->protocol_version = 0;
+ nnpdev->protocol_version = 0;
+
+ ida_init(&nnpdev->cmd_chan_ida);
+ hash_init(nnpdev->cmd_chan_hash);
+
nnpdev->cmdq_sched = nnp_msched_create(nnpdev);
if (!nnpdev->cmdq_sched) {
ret = -ENOMEM;
@@ -668,10 +770,15 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
destroy_workqueue(nnpdev->wq);
+ disconnect_all_channels(nnpdev);
dma_free_coherent(nnpdev->dev, NNP_PAGE_SIZE, nnpdev->bios_system_info,
nnpdev->bios_system_info_dma_addr);
nnp_msched_destroy(nnpdev->cmdq_sched);
+ /*
+ * nnpdev->cmd_chan_ida is empty after disconnect_all_channels,
+ * ida_destroy is not needed
+ */
ida_simple_remove(&dev_ida, nnpdev->id);
}
EXPORT_SYMBOL(nnpdev_destroy);
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 63bc54d..9b6383e 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -4,6 +4,8 @@
#ifndef _NNPDRV_DEVICE_H
#define _NNPDRV_DEVICE_H
+#include <linux/hashtable.h>
+#include <linux/idr.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
@@ -70,11 +72,13 @@ struct query_version_work {
* @cmdq: input queue to @cmdq_sched used to schedule driver internal commands
* to be sent to the device.
* @wq: singlethread workqueue for processing device's response messages.
- * @lock: protects accesses to @state
+ * @lock: protects accesses to @state and @cmd_chan_hash
* @is_recovery_bios: true if device has booted from the recovery bios flash
* @boot_image_loaded: true if boot image load has started
* @response_buf: buffer of device response messages arrived from "pci" layer.
* @response_num_msgs: number of qwords available in @response_buf
+ * @cmd_chan_ida: allocate channel ids to be used in ipc protocol.
+ * @cmd_chan_hash: maps command channel id to its struct pointer.
* @bios_system_info_dma_addr: dma page allocated for bios system info.
* @bios_system_info: virtual pointer to bios system info page
* @bios_version_str: the device's started bios version string
@@ -87,6 +91,9 @@ struct query_version_work {
* @boot_image: boot image object used to boot the card
* @query_version_work: work struct used to schedule processing of version
* reply response message arrived from card.
+ * @ipc_chan_resp_op_size: holds response size for each possible channel
+ * response.
+ * @ipc_chan_cmd_op_size: holds command size for each possible channel command.
*/
struct nnp_device {
const struct nnp_device_ops *ops;
@@ -104,6 +111,9 @@ struct nnp_device {
u64 response_buf[NNP_DEVICE_RESPONSE_BUFFER_LEN];
unsigned int response_num_msgs;
+ struct ida cmd_chan_ida;
+ DECLARE_HASHTABLE(cmd_chan_hash, 6);
+
dma_addr_t bios_system_info_dma_addr;
struct nnp_c2h_system_info *bios_system_info;
char bios_version_str[NNP_BIOS_VERSION_LEN];
@@ -117,6 +127,9 @@ struct nnp_device {
struct image_info boot_image;
struct query_version_work query_version_work;
+
+ u8 ipc_chan_resp_op_size[NNP_IPC_NUM_USER_OPS];
+ u8 ipc_chan_cmd_op_size[NNP_IPC_NUM_USER_OPS];
};
/**
@@ -151,4 +164,6 @@ void nnpdev_process_messages(struct nnp_device *nnpdev, u64 *hw_msg,
*/
void nnpdev_set_boot_state(struct nnp_device *nnpdev, u32 mask);
+struct nnp_chan *nnpdev_find_channel(struct nnp_device *nnpdev, u16 chan_id);
+
#endif
diff --git a/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h b/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
index 59b4a79..037c362 100644
--- a/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
+++ b/drivers/misc/intel-nnpi/ipc_include/ipc_protocol.h
@@ -11,6 +11,9 @@
#define IPC_OP_MAX BIT(6)
#define NNP_IPC_OPCODE_MASK GENMASK(5, 0)
+#define NNP_IPC_MIN_USER_OP 32
+#define NNP_IPC_MAX_USER_OP 63
+#define NNP_IPC_NUM_USER_OPS (NNP_IPC_MAX_USER_OP - NNP_IPC_MIN_USER_OP + 1)
#define NNP_MSG_SIZE(msg) (sizeof(msg) / sizeof(__le64))
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 12/15] misc: nnpi: Route device response messages
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (10 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 11/15] misc: nnpi: Create comm channel from app to device Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 13/15] misc: nnpi: Expose command channel file interface Guy Zadicario
` (3 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Route both types of messages coming from the NNP-I card - event
report messages, which are handled by the driver, and command-response
messages, which should be routed to a specific command channel object
("channel").
Event report messages are device-level messages which are not
associated with a specific channel. They are typically initiated by the
NNP-I card, to indicate an error, status change, or any event which
is not a response to a message sent from a spcific channel. These event
report messages are handled by the driver.
In contrast, command-response messages are associated with a specific
channel, and are typically sent from the NNP-I card in response to a
message sent from a channel to the card. These command-response
messages are added to the intended channel ring-buffer for consumption
by this channel.
The list of messages, of both types, coming from the card, is defined
in ipc_include/ipc_c2h_events.h included in this patch.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/cmd_chan.h | 6 +
drivers/misc/intel-nnpi/device.c | 104 +++++++++++
drivers/misc/intel-nnpi/device.h | 2 +
.../misc/intel-nnpi/ipc_include/ipc_c2h_events.h | 198 +++++++++++++++++++++
4 files changed, 310 insertions(+)
create mode 100644 drivers/misc/intel-nnpi/ipc_include/ipc_c2h_events.h
diff --git a/drivers/misc/intel-nnpi/cmd_chan.h b/drivers/misc/intel-nnpi/cmd_chan.h
index 8cb1a5e..2be88c6 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.h
+++ b/drivers/misc/intel-nnpi/cmd_chan.h
@@ -4,6 +4,7 @@
#ifndef NNPDRV_CMD_CHAN_H
#define NNPDRV_CMD_CHAN_H
+#include <linux/bitfield.h>
#include <linux/circ_buf.h>
#include <linux/hashtable.h>
#include <linux/kref.h>
@@ -12,6 +13,7 @@
#include <linux/spinlock.h>
#include "device.h"
+#include "ipc_c2h_events.h"
/**
* struct nnp_chan - structure object for user<->device communication
@@ -21,6 +23,7 @@
* @chan_id: the ipc channel id for this channel
* @hash_node: node to include this object in list of channels
* hash is in (cmd_chan_hash in nnp_device).
+ * @card_critical_error_msg: last critical event report received from device
* @get_device_events: true if device-level events received from card should
* be sent over this channel to user.
* @cmdq: message queue added to msg_scheduler, for user commands to be sent
@@ -43,6 +46,7 @@ struct nnp_chan {
struct nnp_device *nnpdev;
u16 chan_id;
struct hlist_node hash_node;
+ u64 card_critical_error_msg;
bool get_device_events;
struct nnp_msched_queue *cmdq;
@@ -59,6 +63,8 @@ struct nnp_chan {
unsigned int resp_lost;
};
+#define chan_broken(chan) FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, (chan)->card_critical_error_msg)
+
struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
unsigned int min_id, unsigned int max_id,
bool get_device_events);
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 3902876..1064edc 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -14,6 +14,7 @@
#include "cmd_chan.h"
#include "device.h"
#include "host_chardev.h"
+#include "ipc_c2h_events.h"
#include "msg_scheduler.h"
#include "nnp_boot_defs.h"
@@ -212,11 +213,113 @@ static void disconnect_all_channels(struct nnp_device *nnpdev)
spin_unlock(&nnpdev->lock);
}
+static void nnpdev_submit_device_event_to_channels(struct nnp_device *nnpdev,
+ u64 event_msg)
+{
+ struct nnp_chan *cmd_chan;
+ int i;
+ unsigned int event_code;
+ bool should_wake = false;
+ bool is_card_fatal;
+
+ event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, event_msg);
+ is_card_fatal = is_card_fatal_event(event_code);
+
+ spin_lock(&nnpdev->lock);
+ hash_for_each(nnpdev->cmd_chan_hash, i, cmd_chan, hash_node) {
+ /*
+ * Update channel's card critical error,
+ * but do not override it if a more sever "fatal_drv" error
+ * event is already set.
+ */
+ if (is_card_fatal &&
+ !is_card_fatal_drv_event(chan_broken(cmd_chan))) {
+ cmd_chan->card_critical_error_msg = event_msg;
+ should_wake = true;
+ }
+
+ /* Send the event message to the channel (if needed) */
+ if (is_card_fatal || cmd_chan->get_device_events)
+ nnp_chan_add_response(cmd_chan, &event_msg, sizeof(event_msg));
+ }
+ spin_unlock(&nnpdev->lock);
+
+ if (should_wake)
+ wake_up_all(&nnpdev->waitq);
+}
+
+/*
+ * this function handle device-level event report message.
+ * which is usually affect the entire device and not a single channel
+ */
+static void process_device_event(struct nnp_device *nnpdev, u64 event_msg)
+{
+ /* submit the event to all channels requested to get device events */
+ nnpdev_submit_device_event_to_channels(nnpdev, event_msg);
+}
+
+struct event_report_work {
+ struct work_struct work;
+ struct nnp_device *nnpdev;
+ u64 event_msg;
+};
+
+static void device_event_report_handler(struct work_struct *work)
+{
+ struct event_report_work *req =
+ container_of(work, struct event_report_work, work);
+
+ process_device_event(req->nnpdev, req->event_msg);
+
+ kfree(req);
+}
+
+static int handle_event_report(struct nnp_device *nnpdev, const u64 *msgbuf,
+ int avail_qwords)
+{
+ int msg_qwords = 1; /* EVENT_REPORT response len is 1 qword */
+ struct event_report_work *req;
+ u64 event_msg;
+
+ if (avail_qwords < msg_qwords)
+ return 0;
+
+ event_msg = msgbuf[0];
+ if (FIELD_GET(NNP_C2H_EVENT_REPORT_CHAN_VALID_MASK, event_msg)) {
+ struct nnp_chan *cmd_chan;
+ unsigned int chan_id;
+
+ chan_id = FIELD_GET(NNP_C2H_EVENT_REPORT_CHAN_ID_MASK, event_msg);
+ cmd_chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (cmd_chan) {
+ nnp_chan_add_response(cmd_chan, &event_msg, sizeof(event_msg));
+ nnp_chan_put(cmd_chan);
+ } else {
+ dev_dbg(nnpdev->dev,
+ "Got Event Report for non existing channel id %d\n",
+ chan_id);
+ }
+ return msg_qwords;
+ }
+
+ req = kzalloc(sizeof(*req), GFP_NOWAIT);
+ if (!req)
+ return msg_qwords;
+
+ req->event_msg = event_msg;
+ req->nnpdev = nnpdev;
+ INIT_WORK(&req->work, device_event_report_handler);
+ queue_work(nnpdev->wq, &req->work);
+
+ return msg_qwords;
+}
+
typedef int (*response_handler)(struct nnp_device *nnpdev, const u64 *msgbuf,
int avail_qwords);
static response_handler resp_handlers[NNP_IPC_C2H_OPCODE_LAST + 1] = {
[NNP_IPC_C2H_OP_QUERY_VERSION_REPLY3] = handle_query_version_reply3,
+ [NNP_IPC_C2H_OP_EVENT_REPORT] = handle_event_report,
[NNP_IPC_C2H_OP_BIOS_PROTOCOL] = handle_bios_protocol
};
@@ -593,6 +696,7 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
ida_init(&nnpdev->cmd_chan_ida);
hash_init(nnpdev->cmd_chan_hash);
+ init_waitqueue_head(&nnpdev->waitq);
nnpdev->cmdq_sched = nnp_msched_create(nnpdev);
if (!nnpdev->cmdq_sched) {
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index 9b6383e..c37f1da 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -79,6 +79,7 @@ struct query_version_work {
* @response_num_msgs: number of qwords available in @response_buf
* @cmd_chan_ida: allocate channel ids to be used in ipc protocol.
* @cmd_chan_hash: maps command channel id to its struct pointer.
+ * @waitq: used to wait for device response messages
* @bios_system_info_dma_addr: dma page allocated for bios system info.
* @bios_system_info: virtual pointer to bios system info page
* @bios_version_str: the device's started bios version string
@@ -113,6 +114,7 @@ struct nnp_device {
struct ida cmd_chan_ida;
DECLARE_HASHTABLE(cmd_chan_hash, 6);
+ wait_queue_head_t waitq;
dma_addr_t bios_system_info_dma_addr;
struct nnp_c2h_system_info *bios_system_info;
diff --git a/drivers/misc/intel-nnpi/ipc_include/ipc_c2h_events.h b/drivers/misc/intel-nnpi/ipc_include/ipc_c2h_events.h
new file mode 100644
index 0000000..5ca1b8e
--- /dev/null
+++ b/drivers/misc/intel-nnpi/ipc_include/ipc_c2h_events.h
@@ -0,0 +1,198 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNP_IPC_C2H_EVENTS_H
+#define _NNP_IPC_C2H_EVENTS_H
+
+/**
+ * The following describes the possible values for a c2h_event_report message
+ * sent from card to host to report on some error or other events.
+ *
+ * The c2h_event_report message has the following fields available to describe
+ * the event:
+ * event_code - 7 bits value describing the type of event
+ * event_val - 8 bits value - interpretation depends on event_code
+ * chan_id - the protocol id of the channel in which the event was
+ * occurred.
+ * obj_id - 16 bits, interpretation depends on event_code, usually used
+ * to hold an inference object protocol ID.
+ * obj_id_2 - 16 bits, in case obj_id is not enough to describe the object
+ * In this file we define the possible values for the above fields and document
+ * each field meaning for each possible event_code.
+ */
+
+/**
+ * Event codes ranges
+ *
+ * error codes are grouped into the following ranges:
+ * 0 - 3 ==> non error events generated by daemon/runtime
+ * 4 - 47 ==> non error events generated by card kernel driver
+ * 48 - 51 ==> non-critical error events generated by daemon/runtime
+ * 52 - 95 ==> non-critical error events generatd by kernel driver
+ * 96 - 103 ==> context-critical error events generated by daemon/runtime
+ * 104 - 111 ==> context-critical error events generated by kernel driver
+ * 112 - 119 ==> card-critical error events generated by daemon/runtime
+ * 120 - 127 ==> card-critical error events generated by kernel driver
+ *
+ * context-critical error event is one that puts the infer context in an
+ * un-recoverable error state.
+ * card-critical error event is one that make the card not useful for inference
+ * request until it is reset.
+ */
+#define EVENT_NON_ERR_START 0
+#define EVENT_NON_ERR_DRV_START 4
+#define EVENT_ERR_START 48
+#define EVENT_ERR_DRV_START 52
+#define EVENT_CONTEXT_FATAL_START 96
+#define EVENT_CONTEXT_FATAL_DRV_START 104
+#define EVENT_CARD_FATAL_START 112
+#define EVENT_CARD_FATAL_DRV_START 120
+
+#define is_context_fatal_event(e) ({ \
+ int e_ = (e); \
+ (e_ >= EVENT_CONTEXT_FATAL_START && e_ < EVENT_CARD_FATAL_START); \
+})
+
+#define is_card_fatal_event(e) ((e) >= EVENT_CARD_FATAL_START)
+#define is_card_fatal_drv_event(e) ((e) >= EVENT_CARD_FATAL_DRV_START)
+
+#define NNP_IPC_RUNTIME_DONE (EVENT_NON_ERR_START + 1)
+/* MAX offset for EVENT_NON_ERR_START is 3 */
+
+/* non-error event codes */
+#define NNP_IPC_CREATE_CONTEXT_SUCCESS (EVENT_NON_ERR_DRV_START + 0)
+#define NNP_IPC_CREATE_DEVRES_SUCCESS (EVENT_NON_ERR_DRV_START + 1)
+#define NNP_IPC_CREATE_COPY_SUCCESS (EVENT_NON_ERR_DRV_START + 2)
+#define NNP_IPC_EXECUTE_COPY_SUCCESS (EVENT_NON_ERR_DRV_START + 3)
+#define NNP_IPC_DEVRES_DESTROYED (EVENT_NON_ERR_DRV_START + 4)
+#define NNP_IPC_COPY_DESTROYED (EVENT_NON_ERR_DRV_START + 5)
+#define NNP_IPC_CONTEXT_DESTROYED (EVENT_NON_ERR_DRV_START + 6)
+#define NNP_IPC_CREATE_DEVNET_SUCCESS (EVENT_NON_ERR_DRV_START + 7)
+#define NNP_IPC_DEVNET_DESTROYED (EVENT_NON_ERR_DRV_START + 8)
+#define NNP_IPC_CREATE_INFREQ_SUCCESS (EVENT_NON_ERR_DRV_START + 9)
+#define NNP_IPC_INFREQ_DESTROYED (EVENT_NON_ERR_DRV_START + 10)
+#define NNP_IPC_RECOVER_CONTEXT_SUCCESS (EVENT_NON_ERR_DRV_START + 11)
+#define NNP_IPC_THERMAL_TRIP_EVENT (EVENT_NON_ERR_DRV_START + 12)
+#define NNP_IPC_DEVNET_ADD_RES_SUCCESS (EVENT_NON_ERR_DRV_START + 13)
+#define NNP_IPC_DEVICE_STATE_CHANGED (EVENT_NON_ERR_DRV_START + 14)
+#define NNP_IPC_DEVNET_RESOURCES_RESERVATION_SUCCESS \
+ (EVENT_NON_ERR_DRV_START + 15)
+#define NNP_IPC_DEVNET_RESOURCES_RELEASE_SUCCESS (EVENT_NON_ERR_DRV_START + 16)
+#define NNP_IPC_CREATE_CHANNEL_SUCCESS (EVENT_NON_ERR_DRV_START + 17)
+#define NNP_IPC_CHANNEL_DESTROYED (EVENT_NON_ERR_DRV_START + 18)
+#define NNP_IPC_CHANNEL_SET_RB_SUCCESS (EVENT_NON_ERR_DRV_START + 19)
+#define NNP_IPC_CHANNEL_MAP_HOSTRES_SUCCESS (EVENT_NON_ERR_DRV_START + 20)
+#define NNP_IPC_CHANNEL_UNMAP_HOSTRES_SUCCESS (EVENT_NON_ERR_DRV_START + 21)
+#define NNP_IPC_ABORT_REQUEST (EVENT_NON_ERR_DRV_START + 22)
+#define NNP_IPC_GET_FIFO (EVENT_NON_ERR_DRV_START + 23)
+#define NNP_IPC_CREATE_CMD_SUCCESS (EVENT_NON_ERR_DRV_START + 24)
+#define NNP_IPC_CMD_DESTROYED (EVENT_NON_ERR_DRV_START + 25)
+#define NNP_IPC_EXECUTE_CMD_COMPLETE (EVENT_NON_ERR_DRV_START + 26)
+#define NNP_IPC_DEVNET_SET_PROPERTY_SUCCESS (EVENT_NON_ERR_DRV_START + 27)
+#define NNP_IPC_EXECUTE_CPYLST_SUCCESS (EVENT_NON_ERR_DRV_START + 28)
+#define NNP_IPC_GET_CR_FIFO_REPLY (EVENT_NON_ERR_DRV_START + 29)
+#define NNP_IPC_P2P_PEERS_CONNECTED (EVENT_NON_ERR_DRV_START + 30)
+#define NNP_IPC_P2P_PEER_DEV_UPDATED (EVENT_NON_ERR_DRV_START + 31)
+#define NNP_IPC_EXECUTE_COPY_SUBRES_SUCCESS (EVENT_NON_ERR_DRV_START + 32)
+/* MAX offset for EVENT_NON_ERR_DRV_START is 43 */
+
+/* non-critical error event codes */
+#define NNP_IPC_CREATE_CONTEXT_FAILED (EVENT_ERR_DRV_START + 0)
+#define NNP_IPC_CREATE_DEVRES_FAILED (EVENT_ERR_DRV_START + 1)
+#define NNP_IPC_CREATE_COPY_FAILED (EVENT_ERR_DRV_START + 2)
+#define NNP_IPC_DESTROY_CONTEXT_FAILED (EVENT_ERR_DRV_START + 3)
+#define NNP_IPC_DESTROY_DEVRES_FAILED (EVENT_ERR_DRV_START + 4)
+#define NNP_IPC_DESTROY_COPY_FAILED (EVENT_ERR_DRV_START + 5)
+#define NNP_IPC_CREATE_SYNC_FAILED (EVENT_ERR_DRV_START + 6)
+#define NNP_IPC_ERROR_SUB_RESOURCE_LOAD_FAILED (EVENT_ERR_DRV_START + 7)
+#define NNP_IPC_CREATE_DEVNET_FAILED (EVENT_ERR_DRV_START + 8)
+#define NNP_IPC_DESTROY_DEVNET_FAILED (EVENT_ERR_DRV_START + 9)
+#define NNP_IPC_CREATE_INFREQ_FAILED (EVENT_ERR_DRV_START + 10)
+#define NNP_IPC_DESTROY_INFREQ_FAILED (EVENT_ERR_DRV_START + 11)
+#define NNP_IPC_RECOVER_CONTEXT_FAILED (EVENT_ERR_DRV_START + 12)
+#define NNP_IPC_ERROR_MCE_CORRECTABLE (EVENT_ERR_DRV_START + 13)
+#define NNP_IPC_ERROR_MCE_UNCORRECTABLE (EVENT_ERR_DRV_START + 14)
+#define NNP_IPC_DEVNET_ADD_RES_FAILED (EVENT_ERR_DRV_START + 15)
+#define NNP_IPC_DEVNET_RESOURCES_RESERVATION_FAILED (EVENT_ERR_DRV_START + 16)
+#define NNP_IPC_DEVNET_RESOURCES_RELEASE_FAILED (EVENT_ERR_DRV_START + 17)
+#define NNP_IPC_CREATE_CHANNEL_FAILED (EVENT_ERR_DRV_START + 18)
+#define NNP_IPC_DESTROY_CHANNEL_FAILED (EVENT_ERR_DRV_START + 19)
+#define NNP_IPC_CHANNEL_SET_RB_FAILED (EVENT_ERR_DRV_START + 20)
+#define NNP_IPC_CREATE_CMD_FAILED (EVENT_ERR_DRV_START + 21)
+#define NNP_IPC_DESTROY_CMD_FAILED (EVENT_ERR_DRV_START + 22)
+#define NNP_IPC_CHANNEL_MAP_HOSTRES_FAILED (EVENT_ERR_DRV_START + 23)
+#define NNP_IPC_CHANNEL_UNMAP_HOSTRES_FAILED (EVENT_ERR_DRV_START + 24)
+#define NNP_IPC_DEVNET_SET_PROPERTY_FAILED (EVENT_ERR_DRV_START + 25)
+#define NNP_IPC_ERROR_DRAM_ECC_CORRECTABLE (EVENT_ERR_DRV_START + 26)
+#define NNP_IPC_EXECUTE_COPY_FAILED (EVENT_ERR_DRV_START + 27)
+#define NNP_IPC_SCHEDULE_INFREQ_FAILED (EVENT_ERR_DRV_START + 28)
+#define NNP_IPC_EXECUTE_CPYLST_FAILED (EVENT_ERR_DRV_START + 29)
+#define NNP_IPC_EXECUTE_COPY_SUBRES_FAILED (EVENT_ERR_DRV_START + 30)
+#define NNP_IPC_EC_FAILED_TO_RELEASE_CREDIT (EVENT_ERR_DRV_START + 31)
+#define NNP_IPC_DMA_HANG_DETECTED (EVENT_ERR_DRV_START + 32)
+/* MAX offset for EVENT_ERR_DRV_START is 43 */
+
+/* context critical error event codes */
+#define NNP_IPC_ERROR_RUNTIME_LAUNCH (EVENT_CONTEXT_FATAL_START + 0)
+#define NNP_IPC_ERROR_RUNTIME_DIED (EVENT_CONTEXT_FATAL_START + 1)
+/* MAX offset for EVENT_CONTEXT_FATAL_START is 7 */
+
+#define NNP_IPC_CONTEXT_EXEC_ERROR (EVENT_CONTEXT_FATAL_DRV_START + 0)
+#define NNP_IPC_CTX_DRAM_ECC_UNCORRECTABLE (EVENT_CONTEXT_FATAL_DRV_START + 1)
+/* MAX offset for EVENT_CONTEXT_FATAL_DRV_START is 7 */
+
+/* card critical error event codes */
+#define NNP_IPC_ERROR_OS_CRASHED (EVENT_CARD_FATAL_START + 0)
+#define NNP_IPC_ERROR_DRAM_ECC_UNCORRECTABLE_FATAL (EVENT_CARD_FATAL_START + 1)
+#define NNP_IPC_ERROR_FATAL_ICE_ERROR (EVENT_CARD_FATAL_START + 2)
+/* MAX offset for EVENT_CARD_FATAL_START is 7 */
+
+/* card critical and driver fatal*/
+#define NNP_IPC_ERROR_PCI_ERROR (EVENT_CARD_FATAL_DRV_START + 0)
+#define NNP_IPC_ERROR_MCE_UNCORRECTABLE_FATAL (EVENT_CARD_FATAL_DRV_START + 1)
+#define NNP_IPC_ERROR_CARD_RESET (EVENT_CARD_FATAL_DRV_START + 2)
+#define NNP_IPC_ERROR_CHANNEL_KILLED (EVENT_CARD_FATAL_DRV_START + 3)
+#define NNP_IPC_ERROR_PROTOCOL_ERROR (EVENT_CARD_FATAL_DRV_START + 4)
+#define NNP_IPC_FATAL_DMA_HANG_DETECTED (EVENT_CARD_FATAL_DRV_START + 5)
+/* MAX offset for EVENT_CARD_FATAL_DRV_START is 7 */
+
+enum event_val {
+ NNP_IPC_NO_ERROR = 0,
+ NNP_IPC_NO_SUCH_CONTEXT = 1,
+ NNP_IPC_NO_SUCH_DEVRES = 2,
+ NNP_IPC_NO_SUCH_COPY = 3,
+ NNP_IPC_NO_SUCH_NET = 4,
+ NNP_IPC_NO_SUCH_INFREQ = 5,
+ NNP_IPC_ALREADY_EXIST = 6,
+ NNP_IPC_NO_DAEMON = 7,
+ NNP_IPC_NO_MEMORY = 8,
+ NNP_IPC_RUNTIME_FAILED = 9,
+ NNP_IPC_RUNTIME_LAUNCH_FAILED = 10,
+ NNP_IPC_DMA_ERROR = 11,
+ NNP_IPC_RUNTIME_NOT_SUPPORTED = 12,
+ NNP_IPC_RUNTIME_INVALID_EXECUTABLE_NETWORK_BINARY = 13,
+ NNP_IPC_RUNTIME_INFER_MISSING_RESOURCE = 14,
+ NNP_IPC_RUNTIME_INFER_EXEC_ERROR = 15,
+ NNP_IPC_RUNTIME_INFER_SCHEDULE_ERROR = 16,
+ NNP_IPC_CONTEXT_BROKEN = 17,
+ NNP_IPC_DEVNET_RESERVE_INSUFFICIENT_RESOURCES = 18,
+ NNP_IPC_TIMEOUT_EXCEEDED = 19,
+ NNP_IPC_ECC_ALLOC_FAILED = 20,
+ NNP_IPC_NO_SUCH_CHANNEL = 21,
+ NNP_IPC_NO_SUCH_CMD = 22,
+ NNP_IPC_NO_SUCH_HOSTRES = 23,
+ NNP_IPC_DEVNET_EDIT_BUSY = 24,
+ NNP_IPC_DEVNET_EDIT_ERROR = 25,
+ NNP_IPC_NOT_SUPPORTED = 26,
+ NNP_IPC_ICEDRV_INFER_EXEC_ERROR = 27,
+ NNP_IPC_ICEDRV_INFER_EXEC_ERROR_NEED_RESET = 28,
+ NNP_IPC_ICEDRV_INFER_EXEC_ERROR_NEED_CARD_RESET = 29,
+ NNP_IPC_NO_EXEC_ERRORS = 30,
+ NNP_IPC_IO_ERROR = 31,
+ NNP_IPC_INPUT_IS_DIRTY = 32,
+
+ /* Non failure events */
+ NNP_IPC_CMDLIST_FINISHED = 128,
+};
+
+#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 13/15] misc: nnpi: Expose command channel file interface
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (11 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 12/15] misc: nnpi: Route device response messages Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 14/15] misc: nnpi: Create command channel from userspace Guy Zadicario
` (2 subsequent siblings)
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Expose an anon file descriptor interface to a command channel object
which allows user-space to send commands to the device by writing to
that file as well as consume device response messages by reading the
file.
When the file is released (closed), a channel shut-down sequence
starts. First, a message is sent to the device notifying it that the
channel is closing. Once the response to this message is received from
the device, the command channel object is destroyed.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/cmd_chan.c | 362 +++++++++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/cmd_chan.h | 22 ++-
drivers/misc/intel-nnpi/device.c | 71 ++++++++
3 files changed, 454 insertions(+), 1 deletion(-)
diff --git a/drivers/misc/intel-nnpi/cmd_chan.c b/drivers/misc/intel-nnpi/cmd_chan.c
index b5518e0..89ae604 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.c
+++ b/drivers/misc/intel-nnpi/cmd_chan.c
@@ -4,13 +4,16 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/anon_inodes.h>
+#include <linux/bitfield.h>
#include <linux/dev_printk.h>
#include <linux/file.h>
#include <linux/minmax.h>
+#include <linux/poll.h>
#include <linux/slab.h>
#include "cmd_chan.h"
#include "host_chardev.h"
+#include "ipc_c2h_events.h"
#include "ipc_protocol.h"
#include "nnp_user.h"
@@ -52,6 +55,258 @@ static inline void respq_pop(struct nnp_chan *chan, void *buf, int count)
chan->respq.tail = (chan->respq.tail + count) & (chan->respq_size - 1);
}
+static inline void respq_unpop(struct nnp_chan *chan, int count)
+{
+ chan->respq.tail = (chan->respq.tail - count) & (chan->respq_size - 1);
+}
+
+enum respq_state {
+ RESPQ_EMPTY = 0,
+ RESPQ_MSG_AVAIL,
+ RESPQ_DISCONNECTED
+};
+
+/**
+ * respq_state() - check if a response message is available to be popped
+ * @chan: the cmd_chan object
+ *
+ * Checks if new response message is available or channel has been destroyed.
+ *
+ * Return:
+ * * RESPQ_EMPTY - response queue is empty
+ * * RESPQ_MSG_AVAIL - a response message is available in the queue
+ * * RESPQ_DISCONNECTED - the channel in a destroyed state
+ */
+static enum respq_state respq_state(struct nnp_chan *chan)
+{
+ bool ret;
+
+ mutex_lock(&chan->dev_mutex);
+ if (chan->state == NNP_CHAN_DESTROYED) {
+ mutex_unlock(&chan->dev_mutex);
+ return RESPQ_DISCONNECTED;
+ }
+
+ spin_lock(&chan->respq_lock);
+ /*
+ * response messages are pushed into the respq ring-buffer by pushing
+ * the size of the message (as u32) followed by message content.
+ * So an entire message is available only if more than sizeof(u32)
+ * bytes are available (there is no message with zero size).
+ */
+ if (CIRC_CNT(chan->respq.head, chan->respq.tail, chan->respq_size) >
+ sizeof(u32))
+ ret = RESPQ_MSG_AVAIL;
+ else
+ ret = RESPQ_EMPTY;
+ spin_unlock(&chan->respq_lock);
+
+ mutex_unlock(&chan->dev_mutex);
+
+ return ret;
+}
+
+static inline int is_cmd_chan_file(struct file *f);
+
+static int cmd_chan_file_release(struct inode *inode, struct file *f)
+{
+ struct nnp_chan *chan = f->private_data;
+ struct file *host_file;
+
+ if (!is_cmd_chan_file(f))
+ return -EINVAL;
+
+ nnp_chan_send_destroy(chan);
+
+ host_file = chan->host_file;
+ nnp_chan_put(chan);
+ fput(host_file);
+
+ return 0;
+}
+
+/**
+ * cmd_chan_file_read() - reads a single response message arrived from device
+ * @f: cmd_chan file descriptor
+ * @buf: buffer to receive the message
+ * @size: size of buf, must be at least 16 qwords (16 * sizeof(u64))
+ * @off: ignored.
+ *
+ * This function will block and wait until interrupted or a response
+ * message from device is available.
+ * When message(s) are available, it reads a single message, copy it to
+ * @buf and returns the message size.
+ * The given @buf and @size must be large enough to receive the largest
+ * possible response which is 16 qwords, otherwise -EINVAL is returned.
+ * The function returns the size of the received message, a return value
+ * of zero means that corrupted message has been detected and no more reads
+ * can be made from this channel.
+ *
+ * Return: if positive, the size in bytes of the read message.
+ * zero, if a corrupted message has been detected.
+ * error code otherwise
+ */
+static ssize_t cmd_chan_file_read(struct file *f, char __user *buf, size_t size,
+ loff_t *off)
+{
+ struct nnp_chan *chan = f->private_data;
+ u64 msg[NNP_DEVICE_RESPONSE_FIFO_LEN];
+ enum respq_state state;
+ u32 msg_size;
+ int ret;
+
+ if (!is_cmd_chan_file(f))
+ return -EINVAL;
+
+ if (size < sizeof(msg))
+ return -EINVAL;
+
+ /*
+ * wait for response message to be available, interrupted or channel
+ * has been destroyed on us.
+ */
+ ret = wait_event_interruptible(chan->resp_waitq,
+ (state = respq_state(chan)) != RESPQ_EMPTY);
+ if (ret < 0)
+ return ret;
+
+ if (state == RESPQ_DISCONNECTED)
+ return -EPIPE;
+
+ spin_lock(&chan->respq_lock);
+ respq_pop(chan, &msg_size, sizeof(msg_size));
+ /*
+ * Check msg_size does not overrun msg size.
+ * This will never happen unless the response ring buffer got
+ * corrupted in some way.
+ * We detect it here for safety and return zero
+ */
+ if (msg_size > sizeof(msg)) {
+ /*
+ * unpop the bad size to let subsequent read attempts
+ * to fail as well.
+ */
+ respq_unpop(chan, sizeof(msg_size));
+ spin_unlock(&chan->respq_lock);
+ return 0;
+ }
+ respq_pop(chan, msg, msg_size);
+ spin_unlock(&chan->respq_lock);
+
+ if (copy_to_user(buf, msg, msg_size))
+ return -EFAULT;
+
+ return (ssize_t)msg_size;
+}
+
+/**
+ * cmd_chan_file_write() - schedule a command message to be sent to the device.
+ * @f: a cmd_chan file descriptor
+ * @buf: the command message content
+ * @size: size in bytes of the message, must be multiple of 8 and not larger
+ * than 3 qwords.
+ * @off: ignored
+ *
+ * This function reads a command message from buffer and puts it in the
+ * channel's message queue to schedule it to be delivered to the device.
+ * The function returns when the message is copied to the message scheduler
+ * queue without waiting for it to be sent out.
+ * A valid command message size must be qword aligned and not larger than
+ * the maximum size the message scheduler support, which is 3 qwords.
+ *
+ * The function also validate the command content and fail if the chan_id
+ * field of the command header does not belong to the same channel of this
+ * file descriptor, or the command opcode is out of range, or the command
+ * size does not fit the size of this opcode.
+ *
+ * Return: the size of the message written or error code.
+ */
+static ssize_t cmd_chan_file_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *off)
+{
+ struct nnp_chan *chan = f->private_data;
+ u64 msg[MSG_SCHED_MAX_MSG_SIZE];
+ unsigned int chan_id, opcode;
+ unsigned int op;
+ int rc = 0;
+
+ if (!is_cmd_chan_file(f))
+ return -EINVAL;
+
+ /*
+ * size must be positive, multiple of 8 bytes and
+ * cannot exceed maximum message size
+ */
+ if (!size || size > sizeof(msg) || (size & 0x7) != 0)
+ return -EINVAL;
+
+ if (copy_from_user(msg, buf, size))
+ return -EFAULT;
+
+ /*
+ * Check chan_id, opcode and message size are valid
+ */
+ opcode = FIELD_GET(NNP_H2C_CHAN_MSG_OP_MASK, msg[0]);
+ chan_id = FIELD_GET(NNP_H2C_CHAN_MSG_CHAN_ID_MASK, msg[0]);
+ if (chan_id != chan->chan_id)
+ return -EINVAL;
+ if (opcode < NNP_IPC_MIN_USER_OP)
+ return -EINVAL;
+ op = opcode - NNP_IPC_MIN_USER_OP;
+
+ mutex_lock(&chan->dev_mutex);
+ if (!chan->nnpdev) {
+ /* The device was removed */
+ mutex_unlock(&chan->dev_mutex);
+ return -EPIPE;
+ }
+ if (size != chan->nnpdev->ipc_chan_cmd_op_size[op] * 8) {
+ mutex_unlock(&chan->dev_mutex);
+ return -EINVAL;
+ }
+
+ if (!is_card_fatal_drv_event(chan_broken(chan)))
+ rc = nnp_msched_queue_add_msg(chan->cmdq, msg, size / 8);
+ mutex_unlock(&chan->dev_mutex);
+
+ if (rc < 0)
+ return rc;
+
+ return size;
+}
+
+static unsigned int cmd_chan_file_poll(struct file *f, struct poll_table_struct *pt)
+{
+ struct nnp_chan *chan = f->private_data;
+ unsigned int mask = POLLOUT | POLLWRNORM;
+ enum respq_state state;
+
+ if (!is_cmd_chan_file(f))
+ return 0;
+
+ poll_wait(f, &chan->resp_waitq, pt);
+ state = respq_state(chan);
+ if (state != RESPQ_EMPTY)
+ mask |= POLLIN | POLLRDNORM;
+ if (state == RESPQ_DISCONNECTED)
+ mask |= POLLHUP;
+
+ return mask;
+}
+
+static const struct file_operations nnp_chan_fops = {
+ .owner = THIS_MODULE,
+ .release = cmd_chan_file_release,
+ .read = cmd_chan_file_read,
+ .write = cmd_chan_file_write,
+ .poll = cmd_chan_file_poll,
+};
+
+static inline int is_cmd_chan_file(struct file *f)
+{
+ return f->f_op == &nnp_chan_fops;
+}
+
/**
* nnpdev_chan_create() - creates a command channel object
* @nnpdev: the device
@@ -119,6 +374,7 @@ struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
kref_init(&cmd_chan->ref);
cmd_chan->chan_id = chan_id;
cmd_chan->nnpdev = nnpdev;
+ cmd_chan->fd = -1;
cmd_chan->get_device_events = get_device_events;
cmd_chan->nnp_user = cmd_chan->host_file->private_data;
@@ -154,6 +410,17 @@ static void nnp_chan_release(struct kref *kref)
nnp_chan_disconnect(cmd_chan);
+ /*
+ * If a chan file was created (through nnp_chan_create_file),
+ * the host_file was already put when the file has released, otherwise
+ * we put it here.
+ * This is because we want to release host_file once the channel file
+ * has been closed even though the channel object may continue to exist
+ * until the card will send a respond that it was destroyed.
+ */
+ if (cmd_chan->fd < 0)
+ fput(cmd_chan->host_file);
+
nnp_user_put(cmd_chan->nnp_user);
kfree(cmd_chan->respq_buf);
@@ -170,6 +437,99 @@ void nnp_chan_put(struct nnp_chan *cmd_chan)
kref_put(&cmd_chan->ref, nnp_chan_release);
}
+int nnp_chan_create_file(struct nnp_chan *cmd_chan)
+{
+ /*
+ * get refcount to the channel that will drop when
+ * the file is released.
+ */
+ nnp_chan_get(cmd_chan);
+
+ cmd_chan->fd = anon_inode_getfd("nnpi_chan", &nnp_chan_fops, cmd_chan,
+ O_RDWR | O_CLOEXEC);
+ if (cmd_chan->fd < 0)
+ nnp_chan_put(cmd_chan);
+
+ return cmd_chan->fd;
+}
+
+/**
+ * nnp_chan_set_destroyed() - atomically mark the channel "destroyed"
+ * @chan: the cmd_chan
+ *
+ * This function sets the command channel state to "destroyed" and returns
+ * the previous destroyed state.
+ * This function should be called once the channel has been destructed on the
+ * device and a "channel destroyed" response message arrived.
+ *
+ * Return: true if the channel was already marked destroyed.
+ */
+bool nnp_chan_set_destroyed(struct nnp_chan *chan)
+{
+ bool ret;
+
+ mutex_lock(&chan->dev_mutex);
+ ret = (chan->state == NNP_CHAN_DESTROYED);
+ chan->state = NNP_CHAN_DESTROYED;
+ mutex_unlock(&chan->dev_mutex);
+
+ wake_up_all(&chan->resp_waitq);
+
+ return ret;
+}
+
+/**
+ * nnp_chan_send_destroy() - sends a "destroy channel" command to device
+ * @chan: the cmd_chan to destroy.
+ *
+ * This function sends a command to the device to destroy a command channel,
+ * The channel object remains to exist, it will be dropped only when the device
+ * send back a "channel destroyed" response message.
+ * In case the device is in critical error state, we treat it as not
+ * functional, and the function will immediately drop the channel object without
+ * sending any command and will return with success.
+ *
+ * Return: 0 on success, error value otherwise.
+ */
+int nnp_chan_send_destroy(struct nnp_chan *chan)
+{
+ u64 cmd;
+ int ret = 0;
+ bool do_put = false;
+
+ mutex_lock(&chan->dev_mutex);
+ if (chan->state == NNP_CHAN_DESTROYED || !chan->nnpdev)
+ goto done;
+
+ cmd = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_OP);
+ cmd |= FIELD_PREP(NNP_H2C_CHANNEL_OP_CHAN_ID_MASK, chan->chan_id);
+ cmd |= FIELD_PREP(NNP_H2C_CHANNEL_OP_DESTROY_MASK, 1);
+
+ chan->event_msg = 0;
+
+ /*
+ * If card is in critical state (or was during the channel lifetime)
+ * we destroy the channel.
+ * otherwise, we send a destroy command to card and will destroy when
+ * the destroy reply arrives.
+ */
+ if (is_card_fatal_drv_event(chan_broken(chan))) {
+ chan->state = NNP_CHAN_DESTROYED;
+ do_put = true;
+ goto done;
+ }
+
+ ret = nnp_msched_queue_msg(chan->cmdq, cmd);
+
+done:
+ mutex_unlock(&chan->dev_mutex);
+ if (do_put) {
+ wake_up_all(&chan->resp_waitq);
+ nnp_chan_put(chan);
+ }
+ return ret;
+}
+
/**
* nnp_chan_disconnect() - disconnect the channel from the NNP-I device object
* @cmd_chan: the command channel object
@@ -194,8 +554,10 @@ void nnp_chan_disconnect(struct nnp_chan *cmd_chan)
spin_lock(&nnpdev->lock);
hash_del(&cmd_chan->hash_node);
spin_unlock(&nnpdev->lock);
+ cmd_chan->state = NNP_CHAN_DESTROYED;
mutex_unlock(&cmd_chan->dev_mutex);
+ wake_up_all(&cmd_chan->resp_waitq);
nnp_msched_queue_sync(cmd_chan->cmdq);
nnp_msched_queue_destroy(cmd_chan->cmdq);
diff --git a/drivers/misc/intel-nnpi/cmd_chan.h b/drivers/misc/intel-nnpi/cmd_chan.h
index 2be88c6..d60abf4 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.h
+++ b/drivers/misc/intel-nnpi/cmd_chan.h
@@ -16,6 +16,16 @@
#include "ipc_c2h_events.h"
/**
+ * enum nnp_chan_state - indicate special state of a command channel
+ * @NNP_CHAN_NORMAL: channel is in normal state.
+ * @NNP_CHAN_DESTROYED: channel should be treated as no-longer-exist on card.
+ */
+enum nnp_chan_state {
+ NNP_CHAN_NORMAL = 0,
+ NNP_CHAN_DESTROYED,
+};
+
+/**
* struct nnp_chan - structure object for user<->device communication
* @ref: refcount for this object
* @nnpdev: the device this channel is connected to. May be NULL after device
@@ -23,9 +33,11 @@
* @chan_id: the ipc channel id for this channel
* @hash_node: node to include this object in list of channels
* hash is in (cmd_chan_hash in nnp_device).
+ * @event_msg: ipc event response received from device during create channel
* @card_critical_error_msg: last critical event report received from device
* @get_device_events: true if device-level events received from card should
* be sent over this channel to user.
+ * @fd: file descriptor created for the channel (implements read/write)
* @cmdq: message queue added to msg_scheduler, for user commands to be sent
* to the device.
* @host_file: reference to opened "/dev/nnpi_host" object which defines the
@@ -33,7 +45,8 @@
* @nnp_user: the nnp_user this channel belongs to.
* the channel can reference host resources created by this
* nnp_user object.
- * @dev_mutex: protects @nnpdev
+ * @dev_mutex: protects @nnpdev and @state
+ * @state: the current state of this channel.
* @resp_waitq: waitqueue used for waiting for response messages be available.
* @respq: circular buffer object that receive response messages from device.
* @respq_lock: protects @respq
@@ -46,15 +59,18 @@ struct nnp_chan {
struct nnp_device *nnpdev;
u16 chan_id;
struct hlist_node hash_node;
+ u64 event_msg;
u64 card_critical_error_msg;
bool get_device_events;
+ int fd;
struct nnp_msched_queue *cmdq;
struct file *host_file;
struct nnp_user_info *nnp_user;
struct mutex dev_mutex;
wait_queue_head_t resp_waitq;
+ enum nnp_chan_state state;
struct circ_buf respq;
spinlock_t respq_lock;
@@ -73,6 +89,10 @@ struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
void nnp_chan_put(struct nnp_chan *cmd_chan);
void nnp_chan_disconnect(struct nnp_chan *cmd_chan);
+int nnp_chan_create_file(struct nnp_chan *cmd_chan);
+int nnp_chan_send_destroy(struct nnp_chan *chan);
+bool nnp_chan_set_destroyed(struct nnp_chan *chan);
+
int nnp_chan_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size);
#endif
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 1064edc..ece19c0 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -246,6 +246,47 @@ static void nnpdev_submit_device_event_to_channels(struct nnp_device *nnpdev,
if (should_wake)
wake_up_all(&nnpdev->waitq);
+
+ /*
+ * On card fatal event, we consider the device dead and there is
+ * no point communicating with it. The user will destroy the channel
+ * and initiate a device reset to fix this.
+ * We disconnect all channels and set each as "destroyed" since the
+ * NNP_IPC_CHANNEL_DESTROYED response, which normally do that, will
+ * never arrive.
+ */
+ if (is_card_fatal_drv_event(event_code))
+ disconnect_all_channels(nnpdev);
+}
+
+static void handle_channel_destroy(struct nnp_device *nnpdev, u64 event_msg)
+{
+ struct nnp_chan *cmd_chan;
+ unsigned int chan_id;
+
+ chan_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+ cmd_chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (!cmd_chan) {
+ dev_err(nnpdev->dev,
+ "Got channel destroyed reply for not existing channel %d\n",
+ chan_id);
+ return;
+ }
+
+ /*
+ * Channel is destroyed on device. Put the main ref of cmd_chan if it
+ * did not already done.
+ * There is one possible case that the channel will be already marked
+ * as destroyed when we get here. This is when we got some card fatal
+ * event, which caused us to flag the channel as destroyed, but later
+ * the "destroy channel" response has arrived from the device
+ * (unexpected).
+ */
+ if (!nnp_chan_set_destroyed(cmd_chan))
+ nnp_chan_put(cmd_chan);
+
+ /* put against the get from find_channel */
+ nnp_chan_put(cmd_chan);
}
/*
@@ -254,6 +295,36 @@ static void nnpdev_submit_device_event_to_channels(struct nnp_device *nnpdev,
*/
static void process_device_event(struct nnp_device *nnpdev, u64 event_msg)
{
+ unsigned int event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, event_msg);
+ unsigned int obj_id, event_val;
+
+ if (!is_card_fatal_event(event_code)) {
+ switch (event_code) {
+ case NNP_IPC_DESTROY_CHANNEL_FAILED:
+ obj_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+ event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, event_msg);
+ dev_err(nnpdev->dev,
+ "Channel destroyed failed channel %d val %d\n",
+ obj_id, event_val);
+ /*
+ * We should not enter this case never as the card will
+ * send this response only when the driver requested to
+ * destroy a not-exist channel, which means a driver
+ * bug.
+ * To handle the case we continue and destroy the channel
+ * on the host side.
+ */
+ fallthrough;
+ case NNP_IPC_CHANNEL_DESTROYED:
+ handle_channel_destroy(nnpdev, event_msg);
+ break;
+ default:
+ dev_err(nnpdev->dev,
+ "Unknown event received - %u\n", event_code);
+ return;
+ }
+ }
+
/* submit the event to all channels requested to get device events */
nnpdev_submit_device_event_to_channels(nnpdev, event_msg);
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 14/15] misc: nnpi: Create command channel from userspace
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (12 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 13/15] misc: nnpi: Expose command channel file interface Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-13 8:57 ` [PATCH v2 15/15] misc: nnpi: Map host resources to device channel Guy Zadicario
2021-05-14 8:33 ` [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Greg KH
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Expose a character device for each NNP-I device (/dev/nnpi%d) with
IOCTL interface. Using this character device, user-space can create a
command channel object, through which it can send and receive messages
to and from the device.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/Makefile | 2 +-
drivers/misc/intel-nnpi/cmd_chan.c | 11 +
drivers/misc/intel-nnpi/cmd_chan.h | 1 +
drivers/misc/intel-nnpi/device.c | 50 ++++-
drivers/misc/intel-nnpi/device.h | 11 +
drivers/misc/intel-nnpi/device_chardev.c | 348 +++++++++++++++++++++++++++++++
drivers/misc/intel-nnpi/device_chardev.h | 14 ++
include/uapi/misc/intel_nnpi.h | 43 ++++
8 files changed, 478 insertions(+), 2 deletions(-)
create mode 100644 drivers/misc/intel-nnpi/device_chardev.c
create mode 100644 drivers/misc/intel-nnpi/device_chardev.h
diff --git a/drivers/misc/intel-nnpi/Makefile b/drivers/misc/intel-nnpi/Makefile
index b3bab2a..a294cf0c 100644
--- a/drivers/misc/intel-nnpi/Makefile
+++ b/drivers/misc/intel-nnpi/Makefile
@@ -6,7 +6,7 @@
obj-$(CONFIG_INTEL_NNPI) := intel_nnpi.o intel_nnpi_pcie.o
intel_nnpi-y := device.o msg_scheduler.o hostres.o host_chardev.o nnp_user.o \
- bootimage.o cmd_chan.o
+ bootimage.o cmd_chan.o device_chardev.o
intel_nnpi_pcie-y := nnp_pcie.o
diff --git a/drivers/misc/intel-nnpi/cmd_chan.c b/drivers/misc/intel-nnpi/cmd_chan.c
index 89ae604..0ad281a 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.c
+++ b/drivers/misc/intel-nnpi/cmd_chan.c
@@ -557,6 +557,17 @@ void nnp_chan_disconnect(struct nnp_chan *cmd_chan)
cmd_chan->state = NNP_CHAN_DESTROYED;
mutex_unlock(&cmd_chan->dev_mutex);
+ /*
+ * If the channel is not in critical state,
+ * put it in critical state and wake any user
+ * which might wait for the device.
+ */
+ if (!chan_drv_fatal(cmd_chan)) {
+ cmd_chan->card_critical_error_msg = FIELD_PREP(NNP_C2H_EVENT_REPORT_CODE_MASK,
+ NNP_IPC_ERROR_CHANNEL_KILLED);
+ wake_up_all(&nnpdev->waitq);
+ }
+
wake_up_all(&cmd_chan->resp_waitq);
nnp_msched_queue_sync(cmd_chan->cmdq);
nnp_msched_queue_destroy(cmd_chan->cmdq);
diff --git a/drivers/misc/intel-nnpi/cmd_chan.h b/drivers/misc/intel-nnpi/cmd_chan.h
index d60abf4..3eb5c1c 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.h
+++ b/drivers/misc/intel-nnpi/cmd_chan.h
@@ -80,6 +80,7 @@ struct nnp_chan {
};
#define chan_broken(chan) FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, (chan)->card_critical_error_msg)
+#define chan_drv_fatal(chan) (is_card_fatal_drv_event(chan_broken(chan)))
struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
unsigned int min_id, unsigned int max_id,
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index ece19c0..17746d2 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -13,6 +13,7 @@
#include "bootimage.h"
#include "cmd_chan.h"
#include "device.h"
+#include "device_chardev.h"
#include "host_chardev.h"
#include "ipc_c2h_events.h"
#include "msg_scheduler.h"
@@ -259,6 +260,22 @@ static void nnpdev_submit_device_event_to_channels(struct nnp_device *nnpdev,
disconnect_all_channels(nnpdev);
}
+static void handle_channel_create_response(struct nnp_device *nnpdev, u64 event_msg)
+{
+ struct nnp_chan *cmd_chan;
+ unsigned int chan_id;
+
+ chan_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+
+ cmd_chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (!cmd_chan)
+ return;
+
+ cmd_chan->event_msg = event_msg;
+ nnp_chan_put(cmd_chan);
+ wake_up_all(&nnpdev->waitq);
+}
+
static void handle_channel_destroy(struct nnp_device *nnpdev, u64 event_msg)
{
struct nnp_chan *cmd_chan;
@@ -300,6 +317,10 @@ static void process_device_event(struct nnp_device *nnpdev, u64 event_msg)
if (!is_card_fatal_event(event_code)) {
switch (event_code) {
+ case NNP_IPC_CREATE_CHANNEL_SUCCESS:
+ case NNP_IPC_CREATE_CHANNEL_FAILED:
+ handle_channel_create_response(nnpdev, event_msg);
+ break;
case NNP_IPC_DESTROY_CHANNEL_FAILED:
obj_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, event_msg);
@@ -796,6 +817,11 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
goto err_wq;
}
+ /* Create the character device interface to this device */
+ ret = nnpdev_cdev_create(nnpdev);
+ if (ret)
+ goto err_sys_info;
+
/* set host driver state to "Not ready" */
nnpdev->ops->set_host_doorbell_value(nnpdev, 0);
@@ -805,6 +831,9 @@ int nnpdev_init(struct nnp_device *nnpdev, struct device *dev,
return 0;
+err_sys_info:
+ dma_free_coherent(nnpdev->dev, NNP_PAGE_SIZE, nnpdev->bios_system_info,
+ nnpdev->bios_system_info_dma_addr);
err_wq:
destroy_workqueue(nnpdev->wq);
err_cmdq:
@@ -946,6 +975,10 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
destroy_workqueue(nnpdev->wq);
disconnect_all_channels(nnpdev);
+
+ /* destroy character device */
+ nnpdev_cdev_destroy(nnpdev);
+
dma_free_coherent(nnpdev->dev, NNP_PAGE_SIZE, nnpdev->bios_system_info,
nnpdev->bios_system_info_dma_addr);
@@ -960,13 +993,28 @@ void nnpdev_destroy(struct nnp_device *nnpdev)
static int __init nnp_init(void)
{
- return nnp_init_host_interface();
+ int ret;
+
+ ret = nnp_init_host_interface();
+ if (ret)
+ return ret;
+
+ ret = nnpdev_cdev_class_init();
+ if (ret)
+ goto err_class;
+
+ return 0;
+
+err_class:
+ nnp_release_host_interface();
+ return ret;
}
subsys_initcall(nnp_init);
static void __exit nnp_cleanup(void)
{
nnp_release_host_interface();
+ nnpdev_cdev_class_cleanup();
/* dev_ida is already empty here - no point calling ida_destroy */
}
module_exit(nnp_cleanup);
diff --git a/drivers/misc/intel-nnpi/device.h b/drivers/misc/intel-nnpi/device.h
index c37f1da..e7e66d6 100644
--- a/drivers/misc/intel-nnpi/device.h
+++ b/drivers/misc/intel-nnpi/device.h
@@ -4,8 +4,10 @@
#ifndef _NNPDRV_DEVICE_H
#define _NNPDRV_DEVICE_H
+#include <linux/cdev.h>
#include <linux/hashtable.h>
#include <linux/idr.h>
+#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
@@ -53,6 +55,9 @@
#define NNP_DEVICE_RESPONSE_FIFO_LEN 16
#define NNP_DEVICE_RESPONSE_BUFFER_LEN (NNP_DEVICE_RESPONSE_FIFO_LEN * 2)
+#define NNP_MAX_CHANNEL_ID 1023 /* has 10 bits in ipc protocol */
+#define NNP_MAX_INF_CONTEXT_CHANNEL_ID 255 /* [0, 255] are reserved for inference contexts */
+
struct query_version_work {
struct work_struct work;
u64 chan_resp_op_size;
@@ -92,6 +97,9 @@ struct query_version_work {
* @boot_image: boot image object used to boot the card
* @query_version_work: work struct used to schedule processing of version
* reply response message arrived from card.
+ * @cdev: cdev object of NNP-I device char dev.
+ * @chardev: character device for this device
+ * @cdev_clients: list of opened struct file to the chardev of this device.
* @ipc_chan_resp_op_size: holds response size for each possible channel
* response.
* @ipc_chan_cmd_op_size: holds command size for each possible channel command.
@@ -130,6 +138,9 @@ struct nnp_device {
struct query_version_work query_version_work;
+ struct cdev cdev;
+ struct device *chardev;
+ struct list_head cdev_clients;
u8 ipc_chan_resp_op_size[NNP_IPC_NUM_USER_OPS];
u8 ipc_chan_cmd_op_size[NNP_IPC_NUM_USER_OPS];
};
diff --git a/drivers/misc/intel-nnpi/device_chardev.c b/drivers/misc/intel-nnpi/device_chardev.c
new file mode 100644
index 0000000..e4bb168
--- /dev/null
+++ b/drivers/misc/intel-nnpi/device_chardev.c
@@ -0,0 +1,348 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitfield.h>
+#include <linux/cdev.h>
+#include <linux/device.h>
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/printk.h>
+#include <linux/slab.h>
+
+#include <uapi/misc/intel_nnpi.h>
+
+#include "cmd_chan.h"
+#include "device_chardev.h"
+#include "ipc_c2h_events.h"
+
+static dev_t devnum;
+static struct class *class;
+
+/**
+ * struct device_client - structure for opened device char device file
+ * @node: list node to include this struct in a list of clients
+ * (nnpdev->cdev_clients).
+ * @nnpdev: the NNP-I device associated with the opened chardev
+ * @mutex: protects @nnpdev
+ *
+ * NOTE: @nnpdev may become NULL if the underlying NNP-I device has removed.
+ * Any ioctl request on the char device in this state will fail with
+ * -ENODEV
+ */
+struct device_client {
+ struct list_head node;
+ struct nnp_device *nnpdev;
+ struct mutex mutex;
+};
+
+/* protects nnpdev->cdev_clients list (for all nnp devices) */
+static DEFINE_MUTEX(clients_mutex);
+
+#define NNPDRV_DEVICE_DEV_NAME "nnpi"
+
+static inline bool is_nnp_device_file(struct file *f);
+
+static int nnp_device_open(struct inode *inode, struct file *f)
+{
+ struct device_client *client;
+ struct nnp_device *nnpdev;
+
+ if (!is_nnp_device_file(f))
+ return -EINVAL;
+
+ if (!inode->i_cdev)
+ return -EINVAL;
+
+ client = kzalloc(sizeof(*client), GFP_KERNEL);
+ if (!client)
+ return -ENOMEM;
+
+ nnpdev = container_of(inode->i_cdev, struct nnp_device, cdev);
+ client->nnpdev = nnpdev;
+ mutex_init(&client->mutex);
+ f->private_data = client;
+
+ mutex_lock(&clients_mutex);
+ list_add_tail(&client->node, &nnpdev->cdev_clients);
+ mutex_unlock(&clients_mutex);
+
+ return 0;
+}
+
+static void disconnect_client_locked(struct device_client *client)
+{
+ lockdep_assert_held(&clients_mutex);
+
+ mutex_lock(&client->mutex);
+ if (!client->nnpdev) {
+ mutex_unlock(&client->mutex);
+ return;
+ }
+ client->nnpdev = NULL;
+ list_del(&client->node);
+ mutex_unlock(&client->mutex);
+}
+
+static int nnp_device_release(struct inode *inode, struct file *f)
+{
+ struct device_client *client = f->private_data;
+
+ if (!is_nnp_device_file(f))
+ return -EINVAL;
+
+ mutex_lock(&clients_mutex);
+ disconnect_client_locked(client);
+ mutex_unlock(&clients_mutex);
+ kfree(client);
+ f->private_data = NULL;
+
+ return 0;
+}
+
+static int event_val_to_nnp_error(enum event_val event_val)
+{
+ switch (event_val) {
+ case NNP_IPC_NO_ERROR:
+ return 0;
+ case NNP_IPC_NO_MEMORY:
+ return -ENOMEM;
+ default:
+ return -EFAULT;
+ }
+}
+
+static int send_create_chan_req(struct nnp_device *nnpdev, struct nnp_chan *chan)
+{
+ unsigned int event_code, event_val;
+ u64 cmd;
+ int ret;
+
+ cmd = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_OP);
+ cmd |= FIELD_PREP(NNP_H2C_CHANNEL_OP_CHAN_ID_MASK, chan->chan_id);
+ cmd |= FIELD_PREP(NNP_H2C_CHANNEL_OP_UID_MASK, 0);
+ cmd |= FIELD_PREP(NNP_H2C_CHANNEL_OP_PRIV_MASK, 1);
+
+ ret = nnp_msched_queue_msg(nnpdev->cmdq, cmd);
+ if (ret < 0)
+ return NNPER_DEVICE_ERROR;
+
+ /*
+ * wait until card has respond to the create request or fatal
+ * card error has been detected.
+ */
+ wait_event(nnpdev->waitq, chan->event_msg || chan_drv_fatal(chan));
+ if (!chan->event_msg)
+ return NNPER_DEVICE_ERROR;
+
+ event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, chan->event_msg);
+ event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, chan->event_msg);
+ if (event_code == NNP_IPC_CREATE_CHANNEL_FAILED)
+ return event_val_to_nnp_error(event_val);
+
+ return 0;
+}
+
+static long create_channel(struct device_client *cinfo, void __user *arg,
+ unsigned int size)
+{
+ struct nnp_device *nnpdev = cinfo->nnpdev;
+ struct ioctl_nnpi_create_channel req;
+ unsigned int io_size = sizeof(req);
+ struct nnp_chan *chan;
+ long ret = 0;
+ u32 error_mask;
+
+ /* only single size structure is currently supported */
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ /* o_errno must be cleared on entry */
+ if (req.o_errno)
+ return -EINVAL;
+
+ if (req.i_max_id < req.i_min_id ||
+ req.i_max_id > NNP_MAX_CHANNEL_ID)
+ return -EINVAL;
+
+ /*
+ * Do not allow create command channel if device is in
+ * error state.
+ * However allow new non infer context channels in case
+ * of fatal ICE error in order to allow retrieve debug
+ * information.
+ */
+ error_mask = NNP_DEVICE_ERROR_MASK;
+ if (req.i_max_id > NNP_MAX_INF_CONTEXT_CHANNEL_ID)
+ error_mask &= ~(NNP_DEVICE_FATAL_ICE_ERROR);
+
+ if ((nnpdev->state & error_mask) ||
+ !(nnpdev->state & NNP_DEVICE_CARD_DRIVER_READY) ||
+ (req.i_max_id <= NNP_MAX_INF_CONTEXT_CHANNEL_ID &&
+ (nnpdev->state & NNP_DEVICE_ACTIVE_MASK) !=
+ NNP_DEVICE_ACTIVE_MASK)) {
+ req.o_errno = NNPER_DEVICE_NOT_READY;
+ goto done;
+ }
+
+ /* Validate channel protocol version */
+ if (NNP_VERSION_MAJOR(req.i_protocol_version) !=
+ NNP_VERSION_MAJOR(nnpdev->chan_protocol_version) ||
+ NNP_VERSION_MINOR(req.i_protocol_version) !=
+ NNP_VERSION_MINOR(nnpdev->chan_protocol_version)) {
+ req.o_errno = NNPER_VERSIONS_MISMATCH;
+ goto done;
+ }
+
+ /* create the channel object */
+ chan = nnpdev_chan_create(nnpdev, req.i_host_fd, req.i_min_id, req.i_max_id,
+ req.i_get_device_events);
+ if (IS_ERR(chan)) {
+ ret = PTR_ERR(chan);
+ goto done;
+ }
+
+ /* create the channel on card */
+ req.o_errno = send_create_chan_req(nnpdev, chan);
+ if (req.o_errno)
+ goto err_destroy;
+
+ req.o_channel_id = chan->chan_id;
+
+ /* Attach file descriptor to the channel object */
+ req.o_fd = nnp_chan_create_file(chan);
+
+ /* remove channel object if failed */
+ if (req.o_fd < 0) {
+ /* the channel already created on card - send a destroy request */
+ nnp_chan_send_destroy(chan);
+ ret = req.o_fd;
+ }
+
+ goto done;
+
+err_destroy:
+ /* the channel was not created on card - destroy it now */
+ if (!nnp_chan_set_destroyed(chan))
+ nnp_chan_put(chan);
+done:
+ if (!ret && copy_to_user(arg, &req, io_size))
+ return -EFAULT;
+
+ return ret;
+}
+
+static long nnp_device_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
+{
+ struct device_client *client = f->private_data;
+ unsigned int ioc_nr, size;
+ long ret;
+
+ if (!is_nnp_device_file(f))
+ return -ENOTTY;
+
+ if (_IOC_TYPE(cmd) != 'D')
+ return -EINVAL;
+
+ mutex_lock(&client->mutex);
+ if (!client->nnpdev) {
+ mutex_unlock(&client->mutex);
+ return -ENODEV;
+ }
+
+ ioc_nr = _IOC_NR(cmd);
+ size = _IOC_SIZE(cmd);
+
+ switch (ioc_nr) {
+ case _IOC_NR(IOCTL_NNPI_DEVICE_CREATE_CHANNEL):
+ ret = create_channel(client, (void __user *)arg, size);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ mutex_unlock(&client->mutex);
+
+ return ret;
+}
+
+static const struct file_operations nnp_device_fops = {
+ .owner = THIS_MODULE,
+ .open = nnp_device_open,
+ .release = nnp_device_release,
+ .unlocked_ioctl = nnp_device_ioctl,
+ .compat_ioctl = nnp_device_ioctl,
+};
+
+static inline bool is_nnp_device_file(struct file *f)
+{
+ return f->f_op == &nnp_device_fops;
+}
+
+int nnpdev_cdev_create(struct nnp_device *nnpdev)
+{
+ int ret;
+
+ INIT_LIST_HEAD(&nnpdev->cdev_clients);
+
+ cdev_init(&nnpdev->cdev, &nnp_device_fops);
+ nnpdev->cdev.owner = THIS_MODULE;
+ ret = cdev_add(&nnpdev->cdev, MKDEV(MAJOR(devnum), nnpdev->id), 1);
+ if (ret)
+ return ret;
+
+ nnpdev->chardev = device_create(class, NULL, MKDEV(MAJOR(devnum), nnpdev->id),
+ nnpdev, NNPI_DEVICE_DEV_FMT, nnpdev->id);
+ if (IS_ERR(nnpdev->chardev)) {
+ cdev_del(&nnpdev->cdev);
+ return PTR_ERR(nnpdev->chardev);
+ }
+
+ return 0;
+}
+
+void nnpdev_cdev_destroy(struct nnp_device *nnpdev)
+{
+ struct device_client *client, *tmp;
+
+ device_destroy(class, MKDEV(MAJOR(devnum), nnpdev->id));
+
+ /* disconnect all chardev clients from the device */
+ mutex_lock(&clients_mutex);
+ list_for_each_entry_safe(client, tmp, &nnpdev->cdev_clients, node)
+ disconnect_client_locked(client);
+ mutex_unlock(&clients_mutex);
+
+ cdev_del(&nnpdev->cdev);
+}
+
+int nnpdev_cdev_class_init(void)
+{
+ int ret;
+
+ ret = alloc_chrdev_region(&devnum, 0, NNP_MAX_DEVS,
+ NNPDRV_DEVICE_DEV_NAME);
+ if (ret < 0)
+ return ret;
+
+ class = class_create(THIS_MODULE, NNPDRV_DEVICE_DEV_NAME);
+ if (IS_ERR(class)) {
+ ret = PTR_ERR(class);
+ unregister_chrdev_region(devnum, NNP_MAX_DEVS);
+ return ret;
+ }
+
+ return 0;
+}
+
+void nnpdev_cdev_class_cleanup(void)
+{
+ class_destroy(class);
+ unregister_chrdev_region(devnum, NNP_MAX_DEVS);
+}
+
diff --git a/drivers/misc/intel-nnpi/device_chardev.h b/drivers/misc/intel-nnpi/device_chardev.h
new file mode 100644
index 0000000..0db919d
--- /dev/null
+++ b/drivers/misc/intel-nnpi/device_chardev.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2019-2021 Intel Corporation */
+
+#ifndef _NNPDRV_DEVICE_CHARDEV_H
+#define _NNPDRV_DEVICE_CHARDEV_H
+
+#include "device.h"
+
+int nnpdev_cdev_create(struct nnp_device *nnpdev);
+void nnpdev_cdev_destroy(struct nnp_device *nnpdev);
+int nnpdev_cdev_class_init(void);
+void nnpdev_cdev_class_cleanup(void);
+
+#endif
diff --git a/include/uapi/misc/intel_nnpi.h b/include/uapi/misc/intel_nnpi.h
index 5114aea..620a5d4 100644
--- a/include/uapi/misc/intel_nnpi.h
+++ b/include/uapi/misc/intel_nnpi.h
@@ -134,6 +134,49 @@ struct nnpdrv_ioctl_destroy_hostres {
__u32 o_errno;
};
+/*
+ * ioctls for /dev/nnpi%d device
+ */
+#define NNPI_DEVICE_DEV_FMT "nnpi%u"
+
+/**
+ * IOCTL_NNPI_DEVICE_CREATE_CHANNEL:
+ *
+ * A request to create a new communication "channel" with an NNP-I device.
+ * This channel can be used to send command and receive responses from the
+ * device.
+ */
+#define IOCTL_NNPI_DEVICE_CREATE_CHANNEL \
+ _IOWR('D', 0, struct ioctl_nnpi_create_channel)
+
+/**
+ * struct ioctl_nnpi_create_channel - IOCTL_NNPI_DEVICE_CREATE_CHANNEL payload
+ * @i_host_fd: opened file descriptor to /dev/nnpi_host
+ * @i_min_id: minimum range for channel id allocation
+ * @i_max_id: maximum range for channel id allocation
+ * @i_get_device_events: if true, device-level event responses will be
+ * delivered to be read from the channel.
+ * @i_protocol_version: The NNP_IPC_CHAN_PROTOCOL_VERSION the user-space has
+ * compiled with.
+ * @o_fd: returns file-descriptor through which commands/responses can be
+ * write/read.
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ * @o_channel_id: returns the unique id of the channel
+ *
+ * Argument structure for IOCTL_NNPI_DEVICE_CREATE_CHANNEL ioctl.
+ */
+struct ioctl_nnpi_create_channel {
+ __s32 i_host_fd;
+ __u32 i_min_id;
+ __u32 i_max_id;
+ __s32 i_get_device_events;
+ __u32 i_protocol_version;
+ __s32 o_fd;
+ __u32 o_errno;
+ __u16 o_channel_id;
+};
+
/****************************************************************
* Error code values - errors returned in o_errno fields of
* above structures.
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v2 15/15] misc: nnpi: Map host resources to device channel
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (13 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 14/15] misc: nnpi: Create command channel from userspace Guy Zadicario
@ 2021-05-13 8:57 ` Guy Zadicario
2021-05-14 8:33 ` [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Greg KH
15 siblings, 0 replies; 20+ messages in thread
From: Guy Zadicario @ 2021-05-13 8:57 UTC (permalink / raw)
To: gregkh, linux-kernel
Cc: olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon, guy.zadicario
Provide an IOCTL interface for mapping and unmapping host resources to a
channel, through the device's /dev/nnpi%d char device. The mapping gets a
uniqueue ID and the page list of the host resource is transferred to the
device. Later, commands to the device can reference the resource by the
channel ID and map ID.
There is a special interface to map host resources which serve as
host-to-card and card-to-host ring buffers. These ring buffers can be
referenced later by the ring-buffer direction and index, rather than by
a map ID.
Signed-off-by: Guy Zadicario <guy.zadicario@intel.com>
Reviewed-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
---
drivers/misc/intel-nnpi/cmd_chan.c | 103 ++++++++
drivers/misc/intel-nnpi/cmd_chan.h | 37 ++-
drivers/misc/intel-nnpi/device.c | 58 +++-
drivers/misc/intel-nnpi/device_chardev.c | 441 +++++++++++++++++++++++++++++++
include/uapi/misc/intel_nnpi.h | 111 ++++++++
5 files changed, 748 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/intel-nnpi/cmd_chan.c b/drivers/misc/intel-nnpi/cmd_chan.c
index 0ad281a..d98d02b 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.c
+++ b/drivers/misc/intel-nnpi/cmd_chan.c
@@ -383,6 +383,9 @@ struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
init_waitqueue_head(&cmd_chan->resp_waitq);
mutex_init(&cmd_chan->dev_mutex);
+ ida_init(&cmd_chan->hostres_map_ida);
+ hash_init(cmd_chan->hostres_hash);
+
/*
* Add channel to the channel hash
*/
@@ -411,6 +414,11 @@ static void nnp_chan_release(struct kref *kref)
nnp_chan_disconnect(cmd_chan);
/*
+ * cmd_chan->hostres_map_ida is empty at this point,
+ * calling ida_destroy is not needed
+ */
+
+ /*
* If a chan file was created (through nnp_chan_create_file),
* the host_file was already put when the file has released, otherwise
* we put it here.
@@ -542,6 +550,9 @@ int nnp_chan_send_destroy(struct nnp_chan *chan)
void nnp_chan_disconnect(struct nnp_chan *cmd_chan)
{
struct nnp_device *nnpdev;
+ struct chan_hostres_map *hostres_map;
+ struct hlist_node *tmp;
+ int i;
mutex_lock(&cmd_chan->dev_mutex);
if (!cmd_chan->nnpdev) {
@@ -573,6 +584,32 @@ void nnp_chan_disconnect(struct nnp_chan *cmd_chan)
nnp_msched_queue_destroy(cmd_chan->cmdq);
ida_simple_remove(&nnpdev->cmd_chan_ida, cmd_chan->chan_id);
+
+ /*
+ * Unmap ring buffers
+ */
+ for (i = 0; i < NNP_IPC_MAX_CHANNEL_RB; i++) {
+ if (cmd_chan->h2c_rb_hostres_map[i]) {
+ nnp_hostres_unmap_device(cmd_chan->h2c_rb_hostres_map[i]);
+ cmd_chan->h2c_rb_hostres_map[i] = NULL;
+ }
+ if (cmd_chan->c2h_rb_hostres_map[i]) {
+ nnp_hostres_unmap_device(cmd_chan->c2h_rb_hostres_map[i]);
+ cmd_chan->c2h_rb_hostres_map[i] = NULL;
+ }
+ }
+
+ /*
+ * Destroy all host resource maps
+ */
+ mutex_lock(&cmd_chan->dev_mutex);
+ hash_for_each_safe(cmd_chan->hostres_hash, i, tmp, hostres_map, hash_node) {
+ hash_del(&hostres_map->hash_node);
+ ida_simple_remove(&cmd_chan->hostres_map_ida, hostres_map->id);
+ nnp_hostres_unmap_device(hostres_map->hostres_map);
+ kfree(hostres_map);
+ }
+ mutex_unlock(&cmd_chan->dev_mutex);
}
static int resize_respq(struct nnp_chan *cmd_chan)
@@ -685,3 +722,69 @@ int nnp_chan_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size)
return ret;
}
+
+int nnp_chan_set_ringbuf(struct nnp_chan *chan, bool h2c, unsigned int id,
+ struct nnpdev_mapping *hostres_map)
+{
+ if (id >= NNP_IPC_MAX_CHANNEL_RB)
+ return -EINVAL;
+
+ mutex_lock(&chan->dev_mutex);
+ if (h2c) {
+ if (chan->h2c_rb_hostres_map[id])
+ nnp_hostres_unmap_device(chan->h2c_rb_hostres_map[id]);
+ chan->h2c_rb_hostres_map[id] = hostres_map;
+ } else {
+ if (chan->c2h_rb_hostres_map[id])
+ nnp_hostres_unmap_device(chan->c2h_rb_hostres_map[id]);
+ chan->c2h_rb_hostres_map[id] = hostres_map;
+ }
+ mutex_unlock(&chan->dev_mutex);
+
+ return 0;
+}
+
+static struct chan_hostres_map *find_map(struct nnp_chan *chan, unsigned int map_id)
+{
+ struct chan_hostres_map *hostres_map;
+
+ lockdep_assert_held(&chan->dev_mutex);
+
+ hash_for_each_possible(chan->hostres_hash, hostres_map, hash_node,
+ map_id)
+ if (hostres_map->id == map_id)
+ return hostres_map;
+
+ return NULL;
+}
+
+struct chan_hostres_map *nnp_chan_find_map(struct nnp_chan *chan, unsigned int map_id)
+{
+ struct chan_hostres_map *map;
+
+ mutex_lock(&chan->dev_mutex);
+ map = find_map(chan, map_id);
+ mutex_unlock(&chan->dev_mutex);
+
+ return map;
+}
+
+int nnp_chan_unmap_hostres(struct nnp_chan *chan, unsigned int map_id)
+{
+ struct chan_hostres_map *hostres_map;
+
+ mutex_lock(&chan->dev_mutex);
+ hostres_map = find_map(chan, map_id);
+ if (!hostres_map) {
+ mutex_unlock(&chan->dev_mutex);
+ return -ENXIO;
+ }
+ hash_del(&hostres_map->hash_node);
+ ida_simple_remove(&chan->hostres_map_ida, hostres_map->id);
+ nnp_hostres_unmap_device(hostres_map->hostres_map);
+ mutex_unlock(&chan->dev_mutex);
+
+ kfree(hostres_map);
+
+ return 0;
+}
diff --git a/drivers/misc/intel-nnpi/cmd_chan.h b/drivers/misc/intel-nnpi/cmd_chan.h
index 3eb5c1c..2913595 100644
--- a/drivers/misc/intel-nnpi/cmd_chan.h
+++ b/drivers/misc/intel-nnpi/cmd_chan.h
@@ -19,10 +19,13 @@
* enum nnp_chan_state - indicate special state of a command channel
* @NNP_CHAN_NORMAL: channel is in normal state.
* @NNP_CHAN_DESTROYED: channel should be treated as no-longer-exist on card.
+ * @NNP_CHAN_RB_OP_IN_FLIGHT: a ring-buffer create or destroy command has been
+ * sent to device and response did not yet arrived.
*/
enum nnp_chan_state {
NNP_CHAN_NORMAL = 0,
NNP_CHAN_DESTROYED,
+ NNP_CHAN_RB_OP_IN_FLIGHT,
};
/**
@@ -45,14 +48,20 @@ enum nnp_chan_state {
* @nnp_user: the nnp_user this channel belongs to.
* the channel can reference host resources created by this
* nnp_user object.
- * @dev_mutex: protects @nnpdev and @state
+ * @dev_mutex: protects @nnpdev, @state, @hostres_hash and @hostres_map_ida
* @state: the current state of this channel.
+ * @hostres_map_ida: generate ipc ids for hostres mapping
+ * @hostres_hash: hash table to store all host resource mapping, key is ipc id
* @resp_waitq: waitqueue used for waiting for response messages be available.
* @respq: circular buffer object that receive response messages from device.
* @respq_lock: protects @respq
* @respq_buf: buffer space allocated for circular response buffer.
* @respq_size: current allocated size of circular response buffer.
* @resp_lost: number of response messages lost due to response buffer full.
+ * @h2c_rb_hostres_map: host resource mapping used for each host-to-card ring buffer
+ * There may be up to 2 such ring buffers, both can be NULL.
+ * @c2h_rb_hostres_map: host resource mapping used for each card-to-host ring buffer
+ * There may be up to 2 such ring buffers, both can be NULL.
*/
struct nnp_chan {
struct kref ref;
@@ -72,11 +81,32 @@ struct nnp_chan {
wait_queue_head_t resp_waitq;
enum nnp_chan_state state;
+ struct ida hostres_map_ida;
+ DECLARE_HASHTABLE(hostres_hash, 6);
+
struct circ_buf respq;
spinlock_t respq_lock;
char *respq_buf;
unsigned int respq_size;
unsigned int resp_lost;
+
+ struct nnpdev_mapping *h2c_rb_hostres_map[NNP_IPC_MAX_CHANNEL_RB];
+ struct nnpdev_mapping *c2h_rb_hostres_map[NNP_IPC_MAX_CHANNEL_RB];
+};
+
+/**
+ * struct chan_hostres_map - holds host resource mapping to channel
+ *
+ * @id: ipc map id of the mapping
+ * @hash_node: node to include this mapping in @hostres_hash of nnpdrv_cmd_chan
+ * @hostres_map: the host resource mapping object
+ * @event_msg: device response to the map create request
+ */
+struct chan_hostres_map {
+ unsigned int id;
+ struct hlist_node hash_node;
+ struct nnpdev_mapping *hostres_map;
+ u64 event_msg;
};
#define chan_broken(chan) FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, (chan)->card_critical_error_msg)
@@ -96,4 +126,9 @@ struct nnp_chan *nnpdev_chan_create(struct nnp_device *nnpdev, int host_fd,
int nnp_chan_add_response(struct nnp_chan *cmd_chan, u64 *hw_msg, u32 size);
+int nnp_chan_set_ringbuf(struct nnp_chan *chan, bool h2c, unsigned int id,
+ struct nnpdev_mapping *hostres_map);
+
+struct chan_hostres_map *nnp_chan_find_map(struct nnp_chan *chan, unsigned int map_id);
+int nnp_chan_unmap_hostres(struct nnp_chan *chan, unsigned int map_id);
#endif
diff --git a/drivers/misc/intel-nnpi/device.c b/drivers/misc/intel-nnpi/device.c
index 17746d2..69ff555 100644
--- a/drivers/misc/intel-nnpi/device.c
+++ b/drivers/misc/intel-nnpi/device.c
@@ -276,6 +276,45 @@ static void handle_channel_create_response(struct nnp_device *nnpdev, u64 event_
wake_up_all(&nnpdev->waitq);
}
+static void handle_channel_map_hostres(struct nnp_device *nnpdev, u64 event_msg)
+{
+ struct chan_hostres_map *hostres_map;
+ unsigned int chan_id, map_id;
+ struct nnp_chan *cmd_chan;
+
+ chan_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+ cmd_chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (!cmd_chan)
+ return;
+
+ map_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID2_MASK, event_msg);
+ hostres_map = nnp_chan_find_map(cmd_chan, map_id);
+ if (!hostres_map)
+ goto put_chan;
+
+ hostres_map->event_msg = event_msg;
+ wake_up_all(&nnpdev->waitq);
+
+put_chan:
+ nnp_chan_put(cmd_chan);
+}
+
+static void handle_channel_unmap_hostres(struct nnp_device *nnpdev, u64 event_msg)
+{
+ unsigned int chan_id, map_id;
+ struct nnp_chan *cmd_chan;
+
+ chan_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+ cmd_chan = nnpdev_find_channel(nnpdev, chan_id);
+ if (!cmd_chan)
+ return;
+
+ map_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID2_MASK, event_msg);
+ nnp_chan_unmap_hostres(cmd_chan, map_id);
+
+ nnp_chan_put(cmd_chan);
+}
+
static void handle_channel_destroy(struct nnp_device *nnpdev, u64 event_msg)
{
struct nnp_chan *cmd_chan;
@@ -313,14 +352,20 @@ static void handle_channel_destroy(struct nnp_device *nnpdev, u64 event_msg)
static void process_device_event(struct nnp_device *nnpdev, u64 event_msg)
{
unsigned int event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, event_msg);
- unsigned int obj_id, event_val;
+ unsigned int obj_id, event_val, obj_id_2;
if (!is_card_fatal_event(event_code)) {
switch (event_code) {
case NNP_IPC_CREATE_CHANNEL_SUCCESS:
case NNP_IPC_CREATE_CHANNEL_FAILED:
+ case NNP_IPC_CHANNEL_SET_RB_SUCCESS:
+ case NNP_IPC_CHANNEL_SET_RB_FAILED:
handle_channel_create_response(nnpdev, event_msg);
break;
+ case NNP_IPC_CHANNEL_MAP_HOSTRES_SUCCESS:
+ case NNP_IPC_CHANNEL_MAP_HOSTRES_FAILED:
+ handle_channel_map_hostres(nnpdev, event_msg);
+ break;
case NNP_IPC_DESTROY_CHANNEL_FAILED:
obj_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, event_msg);
@@ -339,6 +384,17 @@ static void process_device_event(struct nnp_device *nnpdev, u64 event_msg)
case NNP_IPC_CHANNEL_DESTROYED:
handle_channel_destroy(nnpdev, event_msg);
break;
+ case NNP_IPC_CHANNEL_UNMAP_HOSTRES_FAILED:
+ obj_id = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID_MASK, event_msg);
+ obj_id_2 = FIELD_GET(NNP_C2H_EVENT_REPORT_OBJ_ID2_MASK, event_msg);
+ event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, event_msg);
+ dev_dbg(nnpdev->dev,
+ "Channel hostres unmap failed on device channel %d map %d val %d\n",
+ obj_id, obj_id_2, event_val);
+ fallthrough;
+ case NNP_IPC_CHANNEL_UNMAP_HOSTRES_SUCCESS:
+ handle_channel_unmap_hostres(nnpdev, event_msg);
+ break;
default:
dev_err(nnpdev->dev,
"Unknown event received - %u\n", event_code);
diff --git a/drivers/misc/intel-nnpi/device_chardev.c b/drivers/misc/intel-nnpi/device_chardev.c
index e4bb168..623352d 100644
--- a/drivers/misc/intel-nnpi/device_chardev.c
+++ b/drivers/misc/intel-nnpi/device_chardev.c
@@ -7,6 +7,7 @@
#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/kref.h>
+#include <linux/dma-map-ops.h>
#include <linux/list.h>
#include <linux/printk.h>
#include <linux/slab.h>
@@ -15,6 +16,7 @@
#include "cmd_chan.h"
#include "device_chardev.h"
+#include "nnp_user.h"
#include "ipc_c2h_events.h"
static dev_t devnum;
@@ -236,6 +238,431 @@ static long create_channel(struct device_client *cinfo, void __user *arg,
return ret;
}
+/**
+ * send_rb_op() - sends CHANNEL_RB_OP command and wait for reply
+ * @chan: the command channel
+ * @rb_op_cmd: the command to send
+ * @o_errno: returns zero or error code from device
+ *
+ * The function sends a "ring buffer operation" command to the device
+ * to either create or destroy a ring buffer object.
+ * This is a synchronous operation, the function will wait until a response
+ * from the device has arrived.
+ * If some other synchronous ring buffer operation is already in progress on
+ * the same channel, the function will fail.
+ *
+ * Return:
+ * * -EBUSY: Ring-buffer create/destroy operation is already in-flight.
+ * * -EPIPE: The channel is in critical error state or sending the command
+ * has failed.
+ * * 0: The command has sent successfully, the operation status is updated
+ * in o_errno, if o_errno is zero, then the create/destoy operation has
+ * succeeded, otherwise it indicates an error code received from
+ * device.
+ */
+static int send_rb_op(struct nnp_chan *chan, u64 rb_op_cmd, __u32 *o_errno)
+{
+ struct nnp_device *nnpdev = chan->nnpdev;
+ unsigned int event_code, event_val;
+ int ret = -EPIPE;
+
+ *o_errno = 0;
+
+ mutex_lock(&chan->dev_mutex);
+ if (chan->state == NNP_CHAN_RB_OP_IN_FLIGHT) {
+ mutex_unlock(&chan->dev_mutex);
+ return -EBUSY;
+ } else if (chan->state == NNP_CHAN_DESTROYED) {
+ mutex_unlock(&chan->dev_mutex);
+ *o_errno = NNPER_DEVICE_ERROR;
+ return 0;
+ }
+ chan->state = NNP_CHAN_RB_OP_IN_FLIGHT;
+ mutex_unlock(&chan->dev_mutex);
+
+ chan->event_msg = 0;
+
+ /* send the command to card */
+ if (!chan_drv_fatal(chan))
+ ret = nnp_msched_queue_msg(nnpdev->cmdq, rb_op_cmd);
+
+ if (ret < 0)
+ goto done;
+
+ /* wait until card respond or card critical error is detected */
+ wait_event(nnpdev->waitq, chan->event_msg || chan_drv_fatal(chan));
+ if (!chan->event_msg) {
+ *o_errno = NNPER_DEVICE_ERROR;
+ ret = 0;
+ goto done;
+ }
+
+ event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, chan->event_msg);
+ if (event_code == NNP_IPC_CHANNEL_SET_RB_FAILED) {
+ event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, chan->event_msg);
+ *o_errno = event_val_to_nnp_error(event_val);
+ ret = 0;
+ }
+
+done:
+ mutex_lock(&chan->dev_mutex);
+ if (chan->state == NNP_CHAN_RB_OP_IN_FLIGHT)
+ chan->state = NNP_CHAN_NORMAL;
+ mutex_unlock(&chan->dev_mutex);
+ return ret;
+}
+
+static long create_channel_data_ringbuf(struct device_client *cinfo,
+ void __user *arg, unsigned int size)
+{
+ struct nnp_device *nnpdev = cinfo->nnpdev;
+ struct ioctl_nnpi_create_channel_data_ringbuf req;
+ struct user_hostres *hostres_entry = NULL;
+ struct nnp_user_info *nnp_user = NULL;
+ struct nnpdev_mapping *hostres_map;
+ unsigned int io_size = sizeof(req);
+ struct host_resource *hostres;
+ struct nnp_chan *chan = NULL;
+ unsigned long dma_pfn;
+ dma_addr_t page_list;
+ u64 rb_op_cmd;
+ int ret = 0;
+
+ /* only single size structure is currently supported */
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ if (req.i_id > NNP_IPC_MAX_CHANNEL_RB - 1)
+ return -EINVAL;
+
+ /* o_errno must be cleared on entry */
+ if (req.o_errno)
+ return -EINVAL;
+
+ chan = nnpdev_find_channel(nnpdev, req.i_channel_id);
+ if (!chan) {
+ req.o_errno = NNPER_NO_SUCH_CHANNEL;
+ goto done;
+ }
+
+ nnp_user = chan->nnp_user;
+ mutex_lock(&nnp_user->mutex);
+ hostres_entry = idr_find(&nnp_user->idr, req.i_hostres_handle);
+ if (!hostres_entry) {
+ req.o_errno = NNPER_NO_SUCH_RESOURCE;
+ goto unlock_user;
+ }
+
+ hostres = hostres_entry->hostres;
+
+ /* check the resource fit the direction */
+ if ((req.i_h2c && !nnp_hostres_is_input(hostres)) ||
+ (!req.i_h2c && !nnp_hostres_is_output(hostres))) {
+ req.o_errno = NNPER_INCOMPATIBLE_RESOURCES;
+ goto unlock_user;
+ }
+
+ hostres_map = nnp_hostres_map_device(hostres, nnpdev, false, &page_list, NULL);
+ if (IS_ERR(hostres_map)) {
+ ret = -EFAULT;
+ goto unlock_user;
+ }
+
+ /*
+ * Its OK to release the mutex here and let other
+ * thread destroy the hostres handle as we already
+ * mapped it (which ref counted)
+ */
+ mutex_unlock(&nnp_user->mutex);
+
+ dma_pfn = NNP_IPC_DMA_ADDR_TO_PFN(page_list);
+ rb_op_cmd = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_RB_OP);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_CHAN_ID_MASK, chan->chan_id);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_ID_MASK, req.i_id);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_HOST_PFN_MASK, dma_pfn);
+ if (req.i_h2c)
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_H2C_MASK, 1);
+
+ ret = send_rb_op(chan, rb_op_cmd, &req.o_errno);
+ if (!ret && !req.o_errno)
+ ret = nnp_chan_set_ringbuf(chan, req.i_h2c, req.i_id, hostres_map);
+
+ if (ret || req.o_errno)
+ nnp_hostres_unmap_device(hostres_map);
+
+ goto put_chan;
+
+unlock_user:
+ mutex_unlock(&nnp_user->mutex);
+put_chan:
+ nnp_chan_put(chan);
+done:
+ if (!ret && copy_to_user(arg, &req, io_size))
+ return -EFAULT;
+
+ return ret;
+}
+
+static long destroy_channel_data_ringbuf(struct device_client *cinfo,
+ void __user *arg, unsigned int size)
+{
+ struct ioctl_nnpi_destroy_channel_data_ringbuf req;
+ struct nnp_device *nnpdev = cinfo->nnpdev;
+ unsigned int io_size = sizeof(req);
+ struct nnp_chan *chan;
+ u64 rb_op_cmd;
+ int ret = 0;
+
+ /* only single size structure is currently supported */
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ /* we have one bit in ipc protocol for ringbuf id for each direction */
+ if (req.i_id > 1)
+ return -EINVAL;
+
+ /* o_errno must be cleared on entry */
+ if (req.o_errno)
+ return -EINVAL;
+
+ chan = nnpdev_find_channel(nnpdev, req.i_channel_id);
+ if (!chan) {
+ req.o_errno = NNPER_NO_SUCH_CHANNEL;
+ goto done;
+ }
+
+ rb_op_cmd = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_RB_OP);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_CHAN_ID_MASK, chan->chan_id);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_ID_MASK, req.i_id);
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_DESTROY_MASK, 1);
+ if (req.i_h2c)
+ rb_op_cmd |= FIELD_PREP(NNP_H2C_CHANNEL_RB_OP_H2C_MASK, 1);
+
+ ret = send_rb_op(chan, rb_op_cmd, &req.o_errno);
+ if (ret || req.o_errno)
+ goto put_chan;
+
+ ret = nnp_chan_set_ringbuf(chan, req.i_h2c, req.i_id, NULL);
+
+put_chan:
+ nnp_chan_put(chan);
+done:
+ if (!ret && copy_to_user(arg, &req, io_size))
+ return -EFAULT;
+
+ return ret;
+}
+
+static int send_map_hostres_req(struct nnp_device *nnpdev, struct nnp_chan *chan,
+ struct chan_hostres_map *hostres_map, dma_addr_t page_list)
+{
+ unsigned int event_code, event_val;
+ unsigned long dma_pfn;
+ u64 cmd[2];
+
+ dma_pfn = NNP_IPC_DMA_ADDR_TO_PFN(page_list);
+ cmd[0] = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_HOSTRES_OP);
+ cmd[0] |= FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW0_CHAN_ID_MASK,
+ chan->chan_id);
+ cmd[0] |= FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW0_ID_MASK, hostres_map->id);
+ cmd[1] = FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW1_HOST_PFN_MASK, dma_pfn);
+
+ /* do not send the map command if the device in a fatal error state */
+ if (chan_drv_fatal(chan))
+ return NNPER_DEVICE_ERROR;
+
+ /* send the hostres map command to card */
+ if (nnp_msched_queue_msg(chan->cmdq, cmd) < 0)
+ return NNPER_DEVICE_ERROR;
+
+ /* wait until card respond or card critical error is detected */
+ wait_event(nnpdev->waitq, hostres_map->event_msg || chan_drv_fatal(chan));
+
+ if (!hostres_map->event_msg)
+ return NNPER_DEVICE_ERROR;
+
+ event_code = FIELD_GET(NNP_C2H_EVENT_REPORT_CODE_MASK, hostres_map->event_msg);
+ if (event_code == NNP_IPC_CHANNEL_MAP_HOSTRES_FAILED) {
+ event_val = FIELD_GET(NNP_C2H_EVENT_REPORT_VAL_MASK, hostres_map->event_msg);
+ return event_val_to_nnp_error(event_val);
+ }
+
+ return 0;
+}
+
+static int do_map_hostres(struct nnp_device *nnpdev, struct nnp_chan *chan,
+ unsigned long hostres_handle)
+{
+ struct chan_hostres_map *hostres_map = NULL;
+ struct user_hostres *hostres_entry = NULL;
+ struct nnp_user_info *nnp_user;
+ struct host_resource *hostres;
+ dma_addr_t page_list;
+ int map_id;
+ int err;
+
+ nnp_user = chan->nnp_user;
+ mutex_lock(&nnp_user->mutex);
+ hostres_entry = idr_find(&nnp_user->idr, hostres_handle);
+ if (!hostres_entry) {
+ err = -NNPER_NO_SUCH_RESOURCE;
+ goto unlock_user;
+ }
+ hostres = hostres_entry->hostres;
+
+ hostres_map = kzalloc(sizeof(*hostres_map), GFP_KERNEL);
+ if (!hostres_map) {
+ err = -ENOMEM;
+ goto unlock_user;
+ }
+
+ mutex_lock(&chan->dev_mutex);
+ map_id = ida_simple_get(&chan->hostres_map_ida, 0, U16_MAX, GFP_KERNEL);
+ if (map_id < 0) {
+ err = -ENOMEM;
+ goto err_map;
+ }
+
+ hostres_map->hostres_map = nnp_hostres_map_device(hostres, nnpdev,
+ false, &page_list, NULL);
+ if (IS_ERR(hostres_map->hostres_map)) {
+ err = -EFAULT;
+ goto err_ida;
+ }
+
+ hostres_map->event_msg = 0;
+ hostres_map->id = map_id;
+
+ hash_add(chan->hostres_hash, &hostres_map->hash_node, hostres_map->id);
+ mutex_unlock(&chan->dev_mutex);
+ mutex_unlock(&nnp_user->mutex);
+
+ err = send_map_hostres_req(nnpdev, chan, hostres_map, page_list);
+ if (err) {
+ nnp_chan_unmap_hostres(chan, hostres_map->id);
+ return err;
+ }
+
+ return map_id;
+
+err_ida:
+ ida_simple_remove(&chan->hostres_map_ida, map_id);
+err_map:
+ mutex_unlock(&chan->dev_mutex);
+ kfree(hostres_map);
+unlock_user:
+ mutex_unlock(&nnp_user->mutex);
+ return err;
+}
+
+static long map_hostres(struct device_client *cinfo, void __user *arg,
+ unsigned int size)
+{
+ struct nnp_device *nnpdev = cinfo->nnpdev;
+ struct ioctl_nnpi_channel_map_hostres req;
+ unsigned int io_size = sizeof(req);
+ const struct dma_map_ops *ops;
+ struct nnp_chan *chan = NULL;
+ int ret;
+
+ /* only single size structure is currently supported */
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ chan = nnpdev_find_channel(nnpdev, req.i_channel_id);
+ if (!chan) {
+ req.o_errno = NNPER_NO_SUCH_CHANNEL;
+ goto done;
+ }
+
+ ret = do_map_hostres(nnpdev, chan, req.i_hostres_handle);
+ if (ret < 0) {
+ req.o_errno = -ret;
+ goto put_chan;
+ }
+
+ req.o_errno = 0;
+ req.o_map_id = ret;
+
+ ops = get_dma_ops(nnpdev->dev);
+ if (ops)
+ req.o_sync_needed = ops->sync_sg_for_cpu ? 1 : 0;
+ else
+ req.o_sync_needed =
+ !dev_is_dma_coherent(nnpdev->dev);
+
+ goto put_chan;
+
+put_chan:
+ nnp_chan_put(chan);
+done:
+ if (copy_to_user(arg, &req, io_size))
+ return -EFAULT;
+
+ return 0;
+}
+
+static long unmap_hostres(struct device_client *cinfo, void __user *arg,
+ unsigned int size)
+{
+ struct ioctl_nnpi_channel_unmap_hostres req;
+ struct nnp_device *nnpdev = cinfo->nnpdev;
+ struct chan_hostres_map *hostres_map;
+ unsigned int io_size = sizeof(req);
+ struct nnp_chan *chan = NULL;
+ u64 cmd[2];
+ long ret = 0;
+
+ /* only single size structure is currently supported */
+ if (size != io_size)
+ return -EINVAL;
+
+ if (copy_from_user(&req, arg, io_size))
+ return -EFAULT;
+
+ /* o_errno must be cleared on entry */
+ if (req.o_errno)
+ return -EINVAL;
+
+ chan = nnpdev_find_channel(nnpdev, req.i_channel_id);
+ if (!chan) {
+ req.o_errno = NNPER_NO_SUCH_CHANNEL;
+ goto done;
+ }
+
+ hostres_map = nnp_chan_find_map(chan, req.i_map_id);
+ if (!hostres_map) {
+ req.o_errno = NNPER_NO_SUCH_HOSTRES_MAP;
+ goto put_chan;
+ }
+
+ cmd[0] = FIELD_PREP(NNP_H2C_OP_MASK, NNP_IPC_H2C_OP_CHANNEL_HOSTRES_OP);
+ cmd[0] |= FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW0_CHAN_ID_MASK,
+ chan->chan_id);
+ cmd[0] |= FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW0_ID_MASK, req.i_map_id);
+ cmd[0] |= FIELD_PREP(NNP_H2C_CHANNEL_HOSTRES_QW0_UNMAP_MASK, 1);
+ cmd[1] = 0;
+
+ ret = nnp_msched_queue_msg(chan->cmdq, cmd);
+
+put_chan:
+ nnp_chan_put(chan);
+done:
+ if (!ret && copy_to_user(arg, &req, io_size))
+ return -EFAULT;
+
+ return ret;
+}
+
static long nnp_device_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
{
struct device_client *client = f->private_data;
@@ -261,6 +688,20 @@ static long nnp_device_ioctl(struct file *f, unsigned int cmd, unsigned long arg
case _IOC_NR(IOCTL_NNPI_DEVICE_CREATE_CHANNEL):
ret = create_channel(client, (void __user *)arg, size);
break;
+ case _IOC_NR(IOCTL_NNPI_DEVICE_CREATE_CHANNEL_RB):
+ ret = create_channel_data_ringbuf(client, (void __user *)arg,
+ size);
+ break;
+ case _IOC_NR(IOCTL_NNPI_DEVICE_DESTROY_CHANNEL_RB):
+ ret = destroy_channel_data_ringbuf(client, (void __user *)arg,
+ size);
+ break;
+ case _IOC_NR(IOCTL_NNPI_DEVICE_CHANNEL_MAP_HOSTRES):
+ ret = map_hostres(client, (void __user *)arg, size);
+ break;
+ case _IOC_NR(IOCTL_NNPI_DEVICE_CHANNEL_UNMAP_HOSTRES):
+ ret = unmap_hostres(client, (void __user *)arg, size);
+ break;
default:
ret = -EINVAL;
break;
diff --git a/include/uapi/misc/intel_nnpi.h b/include/uapi/misc/intel_nnpi.h
index 620a5d4..8d53af6 100644
--- a/include/uapi/misc/intel_nnpi.h
+++ b/include/uapi/misc/intel_nnpi.h
@@ -150,6 +150,43 @@ struct nnpdrv_ioctl_destroy_hostres {
_IOWR('D', 0, struct ioctl_nnpi_create_channel)
/**
+ * IOCTL_NNPI_DEVICE_CREATE_CHANNEL_RB:
+ *
+ * A request to create a data ring buffer for a command channel object.
+ * This is used to transfer data together with command to the device.
+ * A device command may include a data size fields which indicate how much data
+ * has pushed into that ring-buffer object.
+ */
+#define IOCTL_NNPI_DEVICE_CREATE_CHANNEL_RB \
+ _IOWR('D', 1, struct ioctl_nnpi_create_channel_data_ringbuf)
+
+/**
+ * IOCTL_NNPI_DEVICE_DESTROY_CHANNEL_RB:
+ *
+ * A request to destoy a data ring buffer allocated for a command channel.
+ */
+#define IOCTL_NNPI_DEVICE_DESTROY_CHANNEL_RB \
+ _IOWR('D', 2, struct ioctl_nnpi_destroy_channel_data_ringbuf)
+
+/**
+ * IOCTL_NNPI_DEVICE_CHANNEL_MAP_HOSTRES:
+ *
+ * A request to map a host resource to a command channel object.
+ * Device commands can include "map id" of this mapping for referencing
+ * a host resource.
+ */
+#define IOCTL_NNPI_DEVICE_CHANNEL_MAP_HOSTRES \
+ _IOWR('D', 3, struct ioctl_nnpi_channel_map_hostres)
+
+/**
+ * IOCTL_NNPI_DEVICE_CHANNEL_UNMAP_HOSTRES:
+ *
+ * A request to unmap a host resource previously mapped to a command channel.
+ */
+#define IOCTL_NNPI_DEVICE_CHANNEL_UNMAP_HOSTRES \
+ _IOWR('D', 4, struct ioctl_nnpi_channel_unmap_hostres)
+
+/**
* struct ioctl_nnpi_create_channel - IOCTL_NNPI_DEVICE_CREATE_CHANNEL payload
* @i_host_fd: opened file descriptor to /dev/nnpi_host
* @i_min_id: minimum range for channel id allocation
@@ -177,6 +214,80 @@ struct ioctl_nnpi_create_channel {
__u16 o_channel_id;
};
+/**
+ * struct ioctl_nnpi_create_channel_data_ringbuf
+ * @i_hostres_handle: handle of a host resource which will be used to hold
+ * the ring-buffer content.
+ * @i_channel_id: command channel id.
+ * @i_id: id of the ring buffer object (can be 0 or 1).
+ * @i_h2c: non-zero if this ring-buffer is for command submission use,
+ * otherwise it is for responses.
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * this is the payload for IOCTL_NNPI_DEVICE_CREATE_CHANNEL_RB ioctl
+ */
+struct ioctl_nnpi_create_channel_data_ringbuf {
+ __s32 i_hostres_handle;
+ __u32 i_channel_id;
+ __u32 i_id;
+ __u32 i_h2c;
+ __u32 o_errno;
+};
+
+/**
+ * struct ioctl_nnpi_destroy_channel_data_ringbuf
+ * @i_channel_id: command channel id.
+ * @i_id: id of the ring buffer object (can be 0 or 1).
+ * @i_h2c: true if this ring-buffer is for command submission use,
+ * otherwise it is for responses.
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * this is the payload for IOCTL_NNPI_DEVICE_DESTROY_CHANNEL_RB ioctl
+ */
+struct ioctl_nnpi_destroy_channel_data_ringbuf {
+ __u32 i_channel_id;
+ __u32 i_id;
+ __u32 i_h2c;
+ __u32 o_errno;
+};
+
+/**
+ * struct ioctl_nnpi_channel_map_hostres
+ * @i_hostres_handle: handle of a host resource to be mapped
+ * @i_channel_id: command channel id.
+ * @o_map_id: returns unique id of the mapping
+ * @o_sync_needed: returns non-zero if LOCK/UNLOCK_HOST_RESOURCE ioctls
+ * needs to be used before/after accessing the resource from cpu.
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * this is the payload for IOCTL_NNPI_DEVICE_CHANNEL_MAP_HOSTRES ioctl
+ */
+struct ioctl_nnpi_channel_map_hostres {
+ __s32 i_hostres_handle;
+ __u32 i_channel_id;
+ __u32 o_map_id;
+ __u32 o_sync_needed;
+ __u32 o_errno;
+};
+
+/**
+ * ioctl_nnpi_channel_unmap_hostres
+ * @i_channel_id: command channel id.
+ * @i_map_id: mapping id
+ * @o_errno: On input, must be set to 0.
+ * On output, 0 on success, one of the NNPERR_* error codes on error.
+ *
+ * This is the payload for IOCTL_NNPI_DEVICE_CHANNEL_UNMAP_HOSTRES ioctl
+ */
+struct ioctl_nnpi_channel_unmap_hostres {
+ __u32 i_channel_id;
+ __u32 i_map_id;
+ __u32 o_errno;
+};
+
/****************************************************************
* Error code values - errors returned in o_errno fields of
* above structures.
--
1.8.3.1
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device
2021-05-13 8:57 [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Guy Zadicario
` (14 preceding siblings ...)
2021-05-13 8:57 ` [PATCH v2 15/15] misc: nnpi: Map host resources to device channel Guy Zadicario
@ 2021-05-14 8:33 ` Greg KH
2021-06-15 14:58 ` Guy Zadicario
15 siblings, 1 reply; 20+ messages in thread
From: Greg KH @ 2021-05-14 8:33 UTC (permalink / raw)
To: Guy Zadicario
Cc: linux-kernel, olof, alexander.shishkin, andriy.shevchenko,
yochai.shefi-simchon
On Thu, May 13, 2021 at 11:57:10AM +0300, Guy Zadicario wrote:
> Hi,
>
> The following series is a driver for a new PCIe device from Intel named NNP-I
> (Nirvana Neural Processor for Inference). NNP-I is a PCIe connected compute
> device used for acceleration of AI deep learning inference applications in the
> data-center.
How does this differ from the "intel_gna" driver being proposed here:
https://lore.kernel.org/linux-doc/20210513110040.2268-1-maciej.kwapulinski@linux.intel.com/
Please work together to come up with a unified userspace api for these
devices, and hopefully, shared kernel code as well.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device
2021-05-14 8:33 ` [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device Greg KH
@ 2021-06-15 14:58 ` Guy Zadicario
2021-06-15 15:54 ` Greg KH
0 siblings, 1 reply; 20+ messages in thread
From: Guy Zadicario @ 2021-06-15 14:58 UTC (permalink / raw)
To: Greg KH
Cc: linux-kernel, olof, alexander.shishkin, Shevchenko, Andriy,
Shefi-simchon, Yochai
On Fri, May 14, 2021 at 11:33:31AM +0300, Greg KH wrote:
> On Thu, May 13, 2021 at 11:57:10AM +0300, Guy Zadicario wrote:
> > Hi,
> >
> > The following series is a driver for a new PCIe device from Intel named NNP-I
> > (Nirvana Neural Processor for Inference). NNP-I is a PCIe connected compute
> > device used for acceleration of AI deep learning inference applications in the
> > data-center.
>
> How does this differ from the "intel_gna" driver being proposed here:
> https://lore.kernel.org/linux-doc/20210513110040.2268-1-maciej.kwapulinski@linux.intel.com/
>
> Please work together to come up with a unified userspace api for these
> devices, and hopefully, shared kernel code as well.
>
Ok, after consulting, we will try to share api and some kernel code
between the two drivers.
The plan is to suggest a common framework for both devices
and let each device driver register with that framework,
some functionality will be common and some will be device driver
specific (same model as drm for gpus). At first I think the common part
will be small and will deal only with host memory management, later,
it might grow, if/when some other drivers will adopt it as well.
We will first change the NNP-I driver to use this framework.
The GNA driver will wait for this API to be accepted and only then will
adopt it and request to upstream.
Thanks,
Guy.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 00/15] misc: nnpi: New PCIe driver for Intel's NNP-I pcie device
2021-06-15 14:58 ` Guy Zadicario
@ 2021-06-15 15:54 ` Greg KH
0 siblings, 0 replies; 20+ messages in thread
From: Greg KH @ 2021-06-15 15:54 UTC (permalink / raw)
To: Guy Zadicario
Cc: linux-kernel, olof, alexander.shishkin, Shevchenko, Andriy,
Shefi-simchon, Yochai
On Tue, Jun 15, 2021 at 05:58:09PM +0300, Guy Zadicario wrote:
> On Fri, May 14, 2021 at 11:33:31AM +0300, Greg KH wrote:
> > On Thu, May 13, 2021 at 11:57:10AM +0300, Guy Zadicario wrote:
> > > Hi,
> > >
> > > The following series is a driver for a new PCIe device from Intel named NNP-I
> > > (Nirvana Neural Processor for Inference). NNP-I is a PCIe connected compute
> > > device used for acceleration of AI deep learning inference applications in the
> > > data-center.
> >
> > How does this differ from the "intel_gna" driver being proposed here:
> > https://lore.kernel.org/linux-doc/20210513110040.2268-1-maciej.kwapulinski@linux.intel.com/
> >
> > Please work together to come up with a unified userspace api for these
> > devices, and hopefully, shared kernel code as well.
> >
> Ok, after consulting, we will try to share api and some kernel code
> between the two drivers.
>
> The plan is to suggest a common framework for both devices
> and let each device driver register with that framework,
> some functionality will be common and some will be device driver
> specific (same model as drm for gpus). At first I think the common part
> will be small and will deal only with host memory management, later,
> it might grow, if/when some other drivers will adopt it as well.
>
> We will first change the NNP-I driver to use this framework.
> The GNA driver will wait for this API to be accepted and only then will
> adopt it and request to upstream.
Great, can you also look into what it would take to move the habanna
code to this new api as well? A new api doesn't mean much if there is
only one user :)
thanks,
greg k-h
^ permalink raw reply [flat|nested] 20+ messages in thread