* [PATCH 01/22] Add Vision Processing Unit (VPU) documentation.
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 02/22] dt-bindings: Add bindings for Keem Bay IPC driver mgross
` (21 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel; +Cc: markgross, mgross, adam.r.gretzinger, Jonathan Corbet
From: mark gross <mgross@linux.intel.com>
The Intel VPU needs a complicated SW stack to make it work. Add a
directory to hold VPU related documentation including an architectural
overview of the SW stack that the patches implement.
Cc: Jonathan Corbet <corbet@lwn.net>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Mark Gross <mgross@linux.intel.com>
---
Documentation/index.rst | 3 +-
Documentation/vpu/index.rst | 16 ++
Documentation/vpu/vpu-stack-overview.rst | 267 +++++++++++++++++++++++
3 files changed, 285 insertions(+), 1 deletion(-)
create mode 100644 Documentation/vpu/index.rst
create mode 100644 Documentation/vpu/vpu-stack-overview.rst
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 57719744774c..0a2cc0204e8f 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: GPL-2.0
+.. SPDX-License-Identifier: GPL-2.0-only
.. The Linux Kernel documentation master file, created by
@@ -137,6 +137,7 @@ needed).
misc-devices/index
scheduler/index
mhi/index
+ vpu/index
Architecture-agnostic documentation
-----------------------------------
diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
new file mode 100644
index 000000000000..7e290e048910
--- /dev/null
+++ b/Documentation/vpu/index.rst
@@ -0,0 +1,16 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+============================================
+Vision Processor Unit Documentation
+============================================
+
+This documentation contains information for the Intel VPU stack.
+
+.. class:: toc-title
+
+ Table of contents
+
+.. toctree::
+ :maxdepth: 2
+
+ vpu-stack-overview
diff --git a/Documentation/vpu/vpu-stack-overview.rst b/Documentation/vpu/vpu-stack-overview.rst
new file mode 100644
index 000000000000..53c06a7d9a52
--- /dev/null
+++ b/Documentation/vpu/vpu-stack-overview.rst
@@ -0,0 +1,267 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+======================
+Intel VPU architecture
+======================
+
+Overview
+========
+
+The Intel Movidius acquisition has developed a Vision Processing Unit (VPU)
+roadmap of products starting with Keem Bay (KMB). The HW configurations the
+VPU can support include:
+
+1. Standalone smart camera that does local CV processing in camera
+2. Standalone appliance or SBC device connected to a network and tethered
+ cameras doing local CV processing
+3. Embedded in a USB dongle or M.2 as an CV accelerator.
+4. Multiple VPU enabled SOC's on a PCIE card as a CV accelerator in a larger IA
+ box or server.
+
+Keem Bay is the first instance of this family of products. This document
+provides an architectural overview of the SW stack supporting the VPU enabled
+products.
+
+Keem Bay (KMB) is a Computer Vision AI processing SoC based on ARM A53 CPU that
+provides Edge neural network acceleration (inference) and includes a Vision
+Processing Unit (VPU) hardware. The ARM CPU SubSystem (CPUSS) interfaces
+locally to the VPU and enables integration/interfacing with a remote host over
+PCIe or USB or Ethernet interfaces. The interface between the CPUSS and the VPU
+is implemented with HW FIFOs (Control) and coherent memory mapping (Data) such
+that zero copy processing can happen within the VPU.
+
+The KMB can be used in all 4 of the above classes of designs.
+
+We refer to the 'local host' as being the ARM part of the SoC, while the
+'remote host' as the IA system hosting the KMB device(s). The KMB SoC boots
+from an eMMC via uBoot and ARM Linux compatible device tree interface with an
+expectation to fully boot within hundreds of milliseconds. There is also
+support for downloading the kernel and root file system image from a remote
+host.
+
+The eMMC can be updated with standard mender update process.
+See https://github.com/mendersoftware/mender
+
+The VPU is started and controlled from the A53 local host. Its firmware image
+is loaded using the drive FW helper KAPI's.
+
+The VPU IP FW payload consists of a SPARC ISA RTEMS bootloader and/or
+application binary.
+
+The interface allowing (remote or local) host clients to access VPU IP
+capabilities is realized through an abstracted programming model, which
+provides Remote Proxy APIs for a host CPU application to dynamically create and
+execute CV and NN workloads on the VPU. All frameworks exposed through
+programming model’s APIs are contained in the pre-compiled standard firmware
+image.
+
+There is a significant SW stack built up to support KMB and the use cases. The
+rest of this documentation provides an overview of the components of the stack.
+
+Keem Bay IPC
+============
+
+Directly interfaces with the KMB HW FIFOs to provide zero copy processing from
+the VPU. It implements the lowest level protocol for interacting with the VPU.
+
+The Keem Bay IPC mechanism is based on shared memory and hardware FIFOs,
+specifically there are:
+
+* Two 128-entry HW FIFOs, one for the CPU and one for the VPU.
+* Two shared memory regions, used as memory pool for allocating IPC buffers
+
+An IPC channel is a software abstraction allowing communication multiplexing,
+so that multiple applications / users can concurrently communicated to the VPU.
+IPC channels area conceptually similar to socket ports.
+
+There is a total of 1024 channels, each one identified by a channel ID, ranging
+from 0 to 1023.
+
+Channels are divided in two categories:
+
+* High-Speed (HS) channels, having IDs in the 0-9 range.
+* General-Purpose (GP) channels, having IDs in the 10-1023 range.
+
+HS channels have higher priority over GP channels and can be used by
+applications requiring higher throughput or lower latency.
+
+Since all the channels share the same HW resources (i.e., the HW FIFOs and the
+IPC memory pools), the Keem Bay IPC driver uses software queues to give a
+higher priority to HS channels.
+
+The driver supports a build-time configurable number of communication channels
+defined in a so called Channel Mapping Table.
+
+An IPC channel is full duplex: a pending operation from a certain channel does
+not block other operations on the same channel, regardless of their operation
+mode (blocking or non-blocking).
+
+Operation mode is individually selectable for each channel, per operation
+direction (read or write). All operations for that direction comply to
+selection.
+
+
+Keem Bay-VPU-IPC
+================
+
+This is the MMIO driver of the VPU IP block inside the SOC. It is a control
+driver mapping IPC channel communication to Xlink virtual channels.
+
+This driver provides the following functionality to other drivers in the
+communication stack:
+
+* VPU IP execution control (firmware load, start, reset)
+* VPU IP event notifications (device connected, device disconnected, WDT event)
+* VPU IP device status query (OFF, BUSY, READY, ERROR, RECOVERY)
+* Communication via the IPC protocol (wrapping the Keem Bay IPC driver and
+ exposing it to higher level Xlink layer)
+
+In addition to the above, the driver exposes SoC information (like stepping,
+device ID, etc.) to user-space via sysfs.
+
+This driver depends on the 'Keem Bay IPC' driver, which enables the Keem Bay
+IPC communication protocol.
+
+The driver uses the Firmware API to load the VPU firmware from user-space.
+
+Xlink-IPC
+=========
+This component is implementing the IPC specific Xlink protocol. It maps channel
+IDs to HW FIFO entries, using the Keem Bay VPU IPC driver.
+
+Some of the main functions this driver provides:
+
+* establishing a connection with an IPC device
+* obtaining a list with the available devices
+* obtaining the status for a device
+* booting a device
+* resetting a device
+* opening and closing channels
+* issuing read and write operations
+
+Xlink-core
+==========
+
+This component implements an abstracted set of control and communication APIs
+based on channel identification. It is intended to support VPU technology both
+at SoC level as well as at IP level, over multiple interfaces.
+
+It provides symmetrical services, where the producer and the consumer have
+the same privileges.
+
+Xlink driver has the ability to abstract several types of communication
+channels underneath, allowing the usage of different interfaces with the same
+function calls.
+
+Xlink services are available to both kernel and user space clients and include:
+
+* interface abstract control and communication API
+* multi device support
+* concurrent communication across 4096 communication channels (from 0 to
+ 0xFFF), with customizable properties
+* full duplex channels with multiprocess and multithread support
+* channel IDs can be mapped to desired physical interface (PCIE, USB, ETH, IPC)
+ via a Channel Mapping Table
+* asynchronous fast pass through mode: remote host data packets are directly
+ dispatched using interrupt systems running on local host to IPC calls for low
+ overhead
+* channel handshaking mechanism for peer to peer communication, without the
+ need of static channel preallocation
+* channel resource management
+* asynchronous data and device notifications to subscribers
+
+Xlink transports: PCIe, USB, ETH, IPCXLink-PCIe
+
+XLink-PCIE
+==========
+This is an endpoint driver that is mapping Xlink channel IDs to PCIE channels.
+
+This component ensures (remote)host-to-(local)host communication, and VPU IP
+communication via an asynchronous pass through mode, where PCIE data loads are
+directly dispatched to Xlink-IPC.
+
+The component builds and advertises Device IDs that can are used by local host
+application in case of multi device scenarios.
+
+XLink-USB
+==========
+This is an endpoint driver that is mapping Xlink channel IDs to bidirectional
+USB endpoints and supports CDC USB class protocol. More than one Xlink channels
+can be mapped to a single USB endpoint.
+
+This component ensures host-to-host communication, and, as well, asynchronous
+pass through communication, where USB transfer packets are directly dispatched
+to Xlink-IPC.
+
+The component builds and advertises Device IDs that can are used by local host
+application in case of multi device scenarios.
+
+XLink-ETH
+=========
+
+This is an endpoint driver that is mapping Xlink channel IDs to Ethernet
+sockets.
+
+This component ensures host-to-host communication, and, as well, asynchronous
+pass through communication, where Ethernet data loads are directly dispatched to
+Xlink-IPC.
+
+The component builds and advertises Device IDs that can are used by local host
+application in case of multi device scenarios.
+
+Assorted drivers that depend on this stack:
+
+Xlink-SMB
+=========
+The Intel Edge.AI Computer Vision platforms have to be monitored using platform
+devices like sensors, fan controller, IO expander etc. Some of these devices
+are memory mapped and some are i2c based. Either of these devices are not
+directly accessible to the host.
+
+The host here refers to the server to which the vision accelerators are
+connected over PCIe Interface. The Host needs to do a consolidated action based
+on the parameters of platform devices. In general, most of the standard devices
+(includes sensors, fan controller, IO expander etc) are I2C/SMBus based and are
+used to provide the status of the accelerator. Standard drivers for these
+devices are available based on i2c/smbus APIs.
+
+Instead of changing the sensor drivers to adapt to PCIe interface, a generic
+i2c adapter "xlink-smbus" which underneath uses xlink as physical medium is
+used. With xlink-smbus, the drivers for the platform devices doesn't need to
+undergo any interface change.
+
+TSEN
+====
+
+Thermal sensor driver for exporting thermal events to the local Arm64 host as
+well as to the remote X86 host if in the PCIe add in CV accelerator
+configuration.
+
+The driver receiving the junction temperature from different heating points
+inside the SOC. The driver will receive the temperature on SMBUS connection and
+forward over xlink-smb when in a remote host configuration.
+
+In Keem Bay, the four thermal junction temperature points are, Media Subsystem
+(mss), NN subsystem (nce), Compute subsystem (cse) and SOC(Maximum of mss, nce
+and cse)
+
+HDDL
+====
+
+- Exports details of temperature sensor, current sensor and fan controller
+ present in Intel Edge.AI Computer Vision platforms to IA host.
+- Enable Time sync of Intel Edge.AI Computer Vision platform with IA host.
+- Handles device connect and disconnect events.
+- Receives slave address from the IA host for memory mapped thermal sensors
+ present in SoC (Documentation/hwmon/intel_tsens_sensors.rst).
+- Registers i2c slave device for slaves present in Intel Edge.AI Computer
+ Vision platform
+
+
+VPUMGR (VPU Manager)
+====================
+
+Bridges firmware on VPU side and applications on CPU user-space, it assists
+firmware on VPU side serving multiple user space application processes on CPU
+side concurrently while also performing necessary data buffer management on
+behalf of VPU IP.
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 02/22] dt-bindings: Add bindings for Keem Bay IPC driver
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
2020-11-30 23:06 ` [PATCH 01/22] Add Vision Processing Unit (VPU) documentation mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 03/22] keembay-ipc: Add Keem Bay IPC module mgross
` (20 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Daniele Alessandrelli,
Rob Herring, devicetree
From: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Add DT binding documentation for the Intel Keem Bay IPC driver, which
enables communication between the Computing Sub-System (CSS) and the
Multimedia Sub-System (MSS) of the Intel Movidius SoC code named Keem
Bay.
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../bindings/soc/intel/intel,keembay-ipc.yaml | 63 +++++++++++++++++++
1 file changed, 63 insertions(+)
create mode 100644 Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
diff --git a/Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml b/Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
new file mode 100644
index 000000000000..6e21c54d8f34
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2020 Intel Corporation
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/soc/intel/intel,keembay-ipc.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Keem Bay IPC
+
+maintainers:
+ - Daniele Alessandrelli <daniele.alessandrelli@intel.com>
+
+description:
+ The Keem Bay IPC driver enables Inter-Processor Communication (IPC) with the
+ Visual Processor Unit (VPU) embedded in the Intel Movidius SoC code named
+ Keem Bay.
+
+properties:
+ compatible:
+ const: intel,keembay-ipc
+
+ reg:
+ items:
+ - description: The CSS (CPU) FIFO registers
+ - description: The MSS (VPU) FIFO registers
+
+ reg-names:
+ items:
+ - const: css_fifo
+ - const: mss_fifo
+
+ interrupts:
+ items:
+ - description: CSS FIFO not-empty interrupt
+
+ interrupt-controller: true
+
+ memory-region:
+ items:
+ - description: Reserved memory region used for CSS IPC buffers
+ - description: Reserved memory region used for MSS IPC buffers
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - memory-region
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ ipc@203300f0 {
+ compatible = "intel,keembay-ipc";
+ reg = <0x203300f0 0x310>, /* CPU TIM FIFO */
+ <0x208200f0 0x310>; /* VPU TIM FIFO */
+ reg-names = "css_fifo", "mss_fifo";
+ interrupts = <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>;
+ memory-region = <&css_ipc_reserved>, <&mss_ipc_reserved>;
+ };
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 03/22] keembay-ipc: Add Keem Bay IPC module
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
2020-11-30 23:06 ` [PATCH 01/22] Add Vision Processing Unit (VPU) documentation mgross
2020-11-30 23:06 ` [PATCH 02/22] dt-bindings: Add bindings for Keem Bay IPC driver mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 04/22] dt-bindings: Add bindings for Keem Bay VPU IPC driver mgross
` (19 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Daniele Alessandrelli,
Paul Walmsley, Palmer Dabbelt, Borislav Petkov, Damien Le Moal,
Peng Fan, Shawn Guo, Leonard Crestez
From: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
On the Intel Movidius SoC code named Keem Bay, communication between the
Computing Sub-System (CSS), i.e., the CPU, and the Multimedia Sub-System
(MSS), i.e., the VPU is enabled by the Keem Bay Inter-Processor
Communication (IPC) mechanism.
Add the driver for using Keem Bay IPC from within the Linux Kernel.
Keem Bay IPC uses the following terminology:
- Node: A processing entity that can use the IPC to communicate;
currently, we just have two nodes, CPU (CSS) and VPU (MSS).
- Link: Two nodes that can communicate over IPC form an IPC link
(currently, we just have one link, the one between the CPU
and VPU).
- Channel: An IPC link can provide multiple IPC channels. IPC channels
allow communication multiplexing, i.e., the same IPC link can
be used by different applications for different
communications. Each channel is identified by a channel ID,
which must be unique within a single IPC link. Channels are
divided in two categories, High-Speed (HS) channels and
General-Purpose (GP) channels. HS channels have higher
priority over GP channels.
Keem Bay IPC mechanism is based on shared memory and hardware FIFOs.
Both the CPU and the VPU have their own hardware FIFO. When the CPU
wants to send an IPC message to the VPU, it writes to the VPU FIFO (MSS
FIFO); similarly, when MSS wants to send an IPC message to the CPU, it
writes to the CPU FIFO (CSS FIFO).
A FIFO entry is simply a pointer to an IPC buffer (aka IPC header)
stored in a portion of memory shared between the CPU and the VPU.
Specifically, the FIFO entry contains the (VPU) physical address of the
IPC buffer being transferred.
In turn, the IPC buffer contains the (VPU) physical address of the
payload (which must be located in shared memory too) as well as other
information (payload size, IPC channel ID, etc.).
Each IPC node instantiates a pool of IPC buffers from its own IPC buffer
memory region. When instantiated, IPC buffers are marked as free. When
the node needs to send an IPC message, it gets the first free buffer it
finds (from its own pool), marks it as allocated (used), and puts its
physical address into the IPC FIFO of the destination node. The
destination node (which is notified by an interrupt when there are
entries pending in its FIFO) extract the physical address from the FIFO
and process the IPC buffer, marking it as free once done (so that the
sender can reuse the buffer).
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmerdabbelt@google.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Damien Le Moal <damien.lemoal@wdc.com>
Cc: Peng Fan <peng.fan@nxp.com>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Leonard Crestez <leonard.crestez@nxp.com>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
MAINTAINERS | 8 +
drivers/soc/Kconfig | 1 +
drivers/soc/Makefile | 1 +
drivers/soc/intel/Kconfig | 17 +
drivers/soc/intel/Makefile | 4 +
drivers/soc/intel/keembay-ipc.c | 1669 +++++++++++++++++++++++++
include/linux/soc/intel/keembay-ipc.h | 30 +
7 files changed, 1730 insertions(+)
create mode 100644 drivers/soc/intel/Kconfig
create mode 100644 drivers/soc/intel/Makefile
create mode 100644 drivers/soc/intel/keembay-ipc.c
create mode 100644 include/linux/soc/intel/keembay-ipc.h
diff --git a/MAINTAINERS b/MAINTAINERS
index e451dcce054f..64475ba4ea9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8955,6 +8955,14 @@ M: Deepak Saxena <dsaxena@plexity.net>
S: Maintained
F: drivers/char/hw_random/ixp4xx-rng.c
+INTEL KEEM BAY IPC DRIVER
+M: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
+M: Mark Gross <mgross@linux.intel.com>
+S: Maintained
+F: Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
+F: drivers/soc/intel/keembay-ipc.c
+F: include/linux/soc/intel/keembay-ipc.h
+
INTEL MANAGEMENT ENGINE (mei)
M: Tomas Winkler <tomas.winkler@intel.com>
L: linux-kernel@vger.kernel.org
diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index 425ab6f7e375..c21a85bc3899 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -8,6 +8,7 @@ source "drivers/soc/atmel/Kconfig"
source "drivers/soc/bcm/Kconfig"
source "drivers/soc/fsl/Kconfig"
source "drivers/soc/imx/Kconfig"
+source "drivers/soc/intel/Kconfig"
source "drivers/soc/ixp4xx/Kconfig"
source "drivers/soc/mediatek/Kconfig"
source "drivers/soc/qcom/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 36452bed86ef..fbd587640e1f 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_MACH_DOVE) += dove/
obj-y += fsl/
obj-$(CONFIG_ARCH_GEMINI) += gemini/
obj-y += imx/
+obj-y += intel/
obj-$(CONFIG_ARCH_IXP4XX) += ixp4xx/
obj-$(CONFIG_SOC_XWAY) += lantiq/
obj-y += mediatek/
diff --git a/drivers/soc/intel/Kconfig b/drivers/soc/intel/Kconfig
new file mode 100644
index 000000000000..1f5c05b97649
--- /dev/null
+++ b/drivers/soc/intel/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Keem Bay SoC drivers
+#
+
+menu "Intel SoC drivers"
+
+config KEEMBAY_IPC
+ tristate "Support for Intel Keem Bay IPC"
+ depends on ARCH_KEEMBAY || (ARM64 && COMPILE_TEST)
+ help
+ Keem Bay IPC enables communication between the Keem Bay CPU
+ Sub-System (CSS) and the Keem Bay Media Sub-System (MSS).
+
+ Select this if you are compiling the Kernel for an Intel SoC that
+ includes the Intel Vision Processing Unit (VPU) such as Keem Bay.
+endmenu
diff --git a/drivers/soc/intel/Makefile b/drivers/soc/intel/Makefile
new file mode 100644
index 000000000000..ecf0246e7822
--- /dev/null
+++ b/drivers/soc/intel/Makefile
@@ -0,0 +1,4 @@
+#
+# Makefile for Keem Bay IPC Linux driver
+#
+obj-$(CONFIG_KEEMBAY_IPC) += keembay-ipc.o
diff --git a/drivers/soc/intel/keembay-ipc.c b/drivers/soc/intel/keembay-ipc.c
new file mode 100644
index 000000000000..7f1e195a0cf8
--- /dev/null
+++ b/drivers/soc/intel/keembay-ipc.c
@@ -0,0 +1,1669 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Keem Bay IPC Driver.
+ *
+ * Copyright (C) 2018-2020 Intel Corporation
+ *
+ * This driver enables Inter-Processor Communication (IPC) between the
+ * Computing Sub-System (CSS) and the Multimedia Sub-System (MSS) of Keem Bay.
+ * CSS is where Linux is running; while MSS is where the VPU firmware runs.
+ *
+ * The IPC uses the following terminology:
+ *
+ * - Node: A processing entity that can use the IPC to communicate
+ * (currently, we just have two nodes, CSS and MSS).
+ *
+ * - Link: Two nodes that can communicate over IPC form an IPC link
+ * (currently, we just have one link, the one formed by CSS and MSS).
+ *
+ * - Channel: An IPC link can provide multiple IPC channels. IPC channels allow
+ * communication multiplexing, i.e., the same IPC link can be used
+ * by different applications for different communications. Each
+ * channel is identified by a channel ID, which must be unique
+ * within a single IPC link. Channels are divided in two categories,
+ * High-Speed (HS) channels and General-Purpose (GP) channels.
+ * HS channels have higher priority over GP channels.
+ *
+ * Keem Bay IPC mechanism is based on shared memory and hardware FIFOs. Both
+ * CSS and MSS have their own hardware FIFO. When CSS wants to send an IPC
+ * message to MSS, it writes to the MSS FIFO; similarly, when MSS wants to send
+ * an IPC message to CSS, it writes to the CSS FIFO.
+ *
+ * A FIFO entry is simply a pointer to an IPC buffer (aka IPC header) stored in
+ * a portion of memory shared between CSS and MSS. Specifically, the FIFO entry
+ * contains the (VPU) physical address of the IPC buffer being transferred.
+ *
+ * In turn, the IPC buffer contains the (VPU) physical address of the payload
+ * (which must be located in shared memory too) as well as other information
+ * (payload size, IPC channel ID, etc.).
+ *
+ * Each IPC node instantiates a pool of IPC buffers from its own IPC buffer
+ * memory region. When instantiated, IPC buffers are marked as free. When the
+ * node needs to send an IPC message, it gets the first free buffer it finds
+ * (from its own pool), marks it as allocated (used), and puts its physical
+ * address into the IPC FIFO of the destination node. The destination node
+ * (which is notified by an interrupt when there are entries pending in its
+ * FIFO) extract the physical address from the FIFO and process the IPC buffer,
+ * marking it as free once done (so that the sender can reuse the buffer).
+ *
+ * Note: Keem Bay IPC is not based on RPMsg, since MSS HW/FW does not support
+ * Virtio and Virtqueues.
+ */
+
+#include <linux/circ_buf.h>
+#include <linux/completion.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+
+#include <linux/soc/intel/keembay-ipc.h>
+
+#define DRV_NAME "keembay-ipc"
+
+/* The alignment to be used for IPC Buffers and IPC Data. */
+#define KMB_IPC_ALIGNMENT 64
+
+/* Maximum number of channels per link. */
+#define KMB_IPC_MAX_CHANNELS 1024
+
+/* The number of high-speed channels per link. */
+#define KMB_IPC_NUM_HS_CHANNELS 10
+
+/*
+ * This is used as index for retrieving reserved memory from the device
+ * tree.
+ */
+#define LOCAL_IPC_BUFFER_IDX 0
+#define REMOTE_IPC_BUFFER_IDX 1
+
+/*
+ * The length of the RX software FIFO.
+ *
+ * Note: This must be a power of 2 as required by the circular buffer API
+ * defined in "linux/circ_buf.h".
+ */
+#define RX_SW_FIFO_LEN 256
+
+/*
+ * The IPC FIFO registers (offsets to the base address defined in device tree).
+ */
+
+/* Read/Write a word from/to FIFO. */
+#define IPC_FIFO 0x00
+
+/* Read from FIFO using ATM mode (currently not used by this driver). */
+#define IPC_FIFO_ATM 0x04
+
+/* FIFO Status. */
+#define IPC_FIFO_STAT 0x0C
+
+/* IPC FIFO overflow status, IDs 63-0 */
+#define IPC_FIFO_OF_FLAG0 0x10
+#define IPC_FIFO_OF_FLAG1 0x14
+
+/* The possible states of an IPC buffer. */
+enum {
+ /*
+ * KMB_IPC_BUF_FREE must be set to 0 to ensure that buffers can be
+ * initialized with memset(&buf, 0, sizeof(buf)).
+ */
+ KMB_IPC_BUF_FREE = 0,
+ KMB_IPC_BUF_ALLOCATED,
+};
+
+/**
+ * struct kmb_ipc_buf - The IPC buffer structure.
+ * @data_addr: The address where the IPC payload is located; NOTE: this is a
+ * VPU address (not a CPU one).
+ * @data_size: The size of the payload.
+ * @channel: The channel used.
+ * @src_node: The Node ID of the sender.
+ * @dst_node: The Node ID of the intended receiver.
+ * @status: Either free or allocated.
+ */
+struct kmb_ipc_buf {
+ u32 data_addr;
+ u32 data_size;
+ u16 channel;
+ u8 src_node;
+ u8 dst_node;
+ u8 status;
+} __packed __aligned(KMB_IPC_ALIGNMENT);
+
+/**
+ * struct ipc_buf_mem - IPC Buffer Memory Region.
+ * @dev: Child device managing the memory region.
+ * @vaddr: The virtual address of the memory region.
+ * @dma_handle: The VPU address of the memory region.
+ * @size: The size of the memory region.
+ */
+struct ipc_buf_mem {
+ struct device *dev;
+ void *vaddr;
+ dma_addr_t dma_handle;
+ size_t size;
+};
+
+/**
+ * struct ipc_buf_pool - IPC buffer pool.
+ * @buffers: Pointer to the array of buffers.
+ * @buf_cnt: Pool size (i.e., number of buffers).
+ * @idx: Current index.
+ */
+struct ipc_buf_pool {
+ struct kmb_ipc_buf *buffers;
+ size_t buf_cnt;
+ size_t idx;
+};
+
+/**
+ * struct ipc_chan - IPC channel.
+ * @rx_data_list: The list of incoming messages.
+ * @rx_lock: The lock for modifying the rx_data_list.
+ * @rx_wait_queue: The wait queue for RX Data (recv() waits on it).
+ * @closing: Closing flag, set when the channel is being closed.
+ */
+struct ipc_chan {
+ struct list_head rx_data_list;
+ spinlock_t rx_lock; /* Protects 'rx_data_list'. */
+ wait_queue_head_t rx_wait_queue;
+ bool closing;
+};
+
+/**
+ * struct tx_data - Element of a TX queue.
+ * @vpu_addr: The VPU address of the data to be transferred.
+ * @size: The size of the data to be transferred.
+ * @chan_id: The channel to be used for the transfer.
+ * @dst_node: The destination node.
+ * @retv: The result of the transfer.
+ * @list: The list head used to concatenate TX data elements.
+ * @tx_done: The completion struct used by the sender to wait for the
+ * transfer to complete.
+ */
+struct tx_data {
+ u32 vpu_addr;
+ u32 size;
+ u16 chan_id;
+ u8 dst_node;
+ int retv;
+ struct list_head list;
+ struct completion tx_done;
+};
+
+/**
+ * struct tx_queue - The TX queue structure.
+ * @tx_data_list: The list of pending TX data on this queue.
+ * @lock: The lock protecting the TX data list.
+ */
+struct tx_queue {
+ struct list_head tx_data_list;
+ spinlock_t lock; /* Protects tx_data_list. */
+};
+
+/**
+ * struct ipc_link - An IPC link.
+ * @channels: The channels associated with this link (the pointers to the
+ * channels are RCU-protected).
+ * @chan_lock: The lock for modifying the channels array.
+ * @srcu_sp: The Sleepable RCU structs associated with the link's channels.
+ * @tx_queues: The TX queues for this link (1 queue for each high-speed
+ * channels + 1 shared among all the general-purpose channels).
+ * @tx_qidx: The index of the next tx_queue to be check for outgoing data.
+ * @tx_queued: The TX completion used to signal when new TX data is pending.
+ * @tx_thread: The TX thread.
+ * @tx_stopping: Flag signaling that the IPC Link is being closed.
+ */
+struct ipc_link {
+ struct ipc_chan __rcu *channels[KMB_IPC_MAX_CHANNELS];
+ spinlock_t chan_lock; /* Protects modifications to 'channels'. */
+ struct srcu_struct srcu_sp[KMB_IPC_MAX_CHANNELS];
+ struct tx_queue tx_queues[KMB_IPC_NUM_HS_CHANNELS + 1];
+ int tx_qidx;
+ struct completion tx_queued;
+ struct task_struct *tx_thread;
+ bool tx_stopping;
+};
+
+/*
+ * RX Circular Buffer struct.
+ *
+ * The producer inserts elements to the head and the consumers extracts
+ * elements from the tail. See Documentation/circular-buffers.txt for usage
+ * details.
+ */
+struct rx_circ_buf {
+ u32 buf[RX_SW_FIFO_LEN];
+ int head;
+ int tail;
+};
+
+/**
+ * struct keembay_ipc_dev - IPC private data.
+ *
+ * @plat_dev: Platform device for this driver.
+ * @local_ipc_mem: Local IPC Buffer memory region.
+ * @remote_ipc_mem: Remove IPC Buffer memory region.
+ * @ipc_buf_pool: The pool of IPC buffers.
+ * @leon_mss_link: The ARM_CSS-Leon_MSS link.
+ * @local_fifo_irq: The IRQ line used by the local hardware FIFO.
+ * @local_fifo_irq_enabled: Whether or not the local hardware FIFO IRQ is
+ * enabled.
+ * @local_fifo_irq_lock: The spinlock to protect the enabled status of the
+ * local FIFO IRQ.
+ * @local_fifo_reg: The base address of the local HW FIFO.
+ * @remote_fifo_reg: The base address of the remote HW FIFO (i.e., the
+ * Leon MSS FIFO).
+ * @rx_tasklet: The RX tasklet.
+ * @rx_sw_fifo: The RX SW FIFO. The RX ISR writes to this FIFO and
+ * the RX tasklet reads from it.
+ */
+struct keembay_ipc_dev {
+ struct platform_device *plat_dev;
+ struct ipc_buf_mem local_ipc_mem;
+ struct ipc_buf_mem remote_ipc_mem;
+ struct ipc_buf_pool ipc_buf_pool;
+ struct ipc_link leon_mss_link;
+ int local_fifo_irq;
+ int local_fifo_irq_enabled;
+ spinlock_t local_fifo_irq_lock; /* Protects status of FIFO IRQ. */
+ void __iomem *local_fifo_reg;
+ void __iomem *remote_fifo_reg;
+ struct tasklet_struct rx_tasklet;
+ struct rx_circ_buf rx_sw_fifo;
+};
+
+/*
+ * RX Data Descriptor.
+ *
+ * Instances of this struct are meant to be inserted in the RX Data queue
+ * (list) associated with each channel.
+ */
+struct rx_data {
+ /* List head for creating a list of rx_data elements. */
+ struct list_head list;
+ /* The VPU address of the received data. */
+ u32 data_vpu_addr;
+ /* The size of the received data. */
+ u32 data_size;
+};
+
+/* Forward declaration of TX thread function. */
+static int tx_thread_fn(void *ptr);
+
+/* Forward declaration of ISR function. */
+static irqreturn_t local_fifo_irq_handler(int irq, void *ptr);
+
+/* Forward declaration of RX tasklet function. */
+static void rx_tasklet_func(unsigned long);
+
+/*
+ * Functions related to reserved-memory sub-devices.
+ */
+
+/*
+ * init_ipc_rsvd_mem() - Init the specified IPC reserved memory.
+ * @dev: The IPC device for which the memory will be initialized.
+ * @mem: Where to stored information about the initialized memory.
+ * @mem_name: The name of this IPC memory.
+ * @mem_idx: The index of the memory to initialize.
+ *
+ * Create a child device for 'dev' and use it to initialize the reserved
+ * memory region specified in the device tree at index 'mem_idx'.
+ * Also allocate DMA memory from the initialized memory region.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int init_ipc_rsvd_mem(struct device *dev, struct ipc_buf_mem *mem,
+ const char *mem_name, unsigned int mem_idx)
+{
+ struct device *mem_dev;
+ struct device_node *np;
+ struct resource mem_res;
+ dma_addr_t dma_handle;
+ size_t mem_size;
+ void *vaddr;
+ int rc;
+
+ /* Create a child device (of dev) to own the reserved memory. */
+ mem_dev = devm_kzalloc(dev, sizeof(struct device), GFP_KERNEL);
+ if (!mem_dev)
+ return -ENOMEM;
+
+ device_initialize(mem_dev);
+ dev_set_name(mem_dev, "%s:%s", dev_name(dev), mem_name);
+ mem_dev->parent = dev;
+ mem_dev->dma_mask = dev->dma_mask;
+ mem_dev->coherent_dma_mask = dev->coherent_dma_mask;
+ mem_dev->release = of_reserved_mem_device_release;
+
+ /* Set up DMA configuration using information from parent's DT node. */
+ rc = of_dma_configure(mem_dev, dev->of_node, true);
+ if (rc)
+ goto err_add;
+
+ rc = device_add(mem_dev);
+ if (rc)
+ goto err_add;
+
+ /* Initialized the device reserved memory region. */
+ rc = of_reserved_mem_device_init_by_idx(mem_dev, dev->of_node, mem_idx);
+ if (rc) {
+ dev_err(dev, "Couldn't get reserved memory with idx = %d, %d\n",
+ mem_idx, rc);
+ goto err_post_add;
+ }
+
+ /* Find out the size of the memory region. */
+ np = of_parse_phandle(dev->of_node, "memory-region", mem_idx);
+ if (!np) {
+ dev_err(dev, "Couldn't find memory-region %d\n", mem_idx);
+ rc = -EINVAL;
+ goto err_post_add;
+ }
+ rc = of_address_to_resource(np, 0, &mem_res);
+ if (rc) {
+ dev_err(dev, "Couldn't map address to resource %d\n", mem_idx);
+ goto err_post_add;
+ }
+ mem_size = resource_size(&mem_res);
+
+ /* Allocate memory from the reserved memory regions */
+ vaddr = dmam_alloc_coherent(mem_dev, mem_size, &dma_handle, GFP_KERNEL);
+ if (!vaddr) {
+ dev_err(mem_dev, "Failed to allocate from reserved memory.\n");
+ rc = -ENOMEM;
+ goto err_post_add;
+ }
+
+ mem->dev = mem_dev;
+ mem->vaddr = vaddr;
+ mem->dma_handle = dma_handle;
+ mem->size = mem_size;
+
+ return 0;
+
+err_post_add:
+ device_del(mem_dev);
+err_add:
+ put_device(mem_dev);
+ return rc;
+}
+
+/*
+ * Keem Bay IPC HW FIFO API.
+ *
+ * We use two HW FIFOs: the local one and the remote one (Leon MSS FIFO).
+ *
+ * The local FIFO holds incoming IPC FIFO entries and therefore is used by RX
+ * code.
+ *
+ * The remote FIFO is where we put outgoing FIFO entries and therefore is used
+ * by TX code.
+ */
+
+/* Get number of entries in the FIFO. */
+static int local_fifo_cnt(struct keembay_ipc_dev *ipc_dev)
+{
+ /*
+ * TIM_IPC_FIFO_STAT
+ *
+ * Bits 31:24: Reserved
+ * Bits 23:16: IPC FIFO fill level
+ * Bits 15:8: IPC FIFO write pointer
+ * Bits 7:0: IPC FIFO read pointer
+ */
+ u32 val = ioread32(ipc_dev->local_fifo_reg + IPC_FIFO_STAT);
+
+ return (val >> 16) & 0xFF;
+}
+
+/* Extract an entry from the FIFO. */
+static u32 local_fifo_get(struct keembay_ipc_dev *ipc_dev)
+{
+ /*
+ * TIM_IPC_FIFO_ATM
+ *
+ * Read a value from IPC HW FIFO.
+ *
+ * If FIFO is empty return 0xFFFFFFFF, otherwise return value from FIFO
+ * with 6 least-significant bits set to 0.
+ * Increment FIFO read pointer and decrement fill level.
+ */
+ return ioread32(ipc_dev->local_fifo_reg + IPC_FIFO_ATM);
+}
+
+/* Disable local FIFO IRQ. */
+static void local_fifo_irq_disable(struct keembay_ipc_dev *ipc_dev)
+{
+ unsigned long flags;
+
+ /* Lock shared between IRQ and Tasklet, use spin_lock_irqsave(). */
+ spin_lock_irqsave(&ipc_dev->local_fifo_irq_lock, flags);
+ if (ipc_dev->local_fifo_irq_enabled) {
+ disable_irq_nosync(ipc_dev->local_fifo_irq);
+ ipc_dev->local_fifo_irq_enabled = false;
+ }
+ spin_unlock_irqrestore(&ipc_dev->local_fifo_irq_lock, flags);
+}
+
+/* Enable local FIFO IRQ. */
+static void local_fifo_irq_enable(struct keembay_ipc_dev *ipc_dev)
+{
+ unsigned long flags;
+
+ /* Lock shared between IRQ and Tasklet, use spin_lock_irqsave(). */
+ spin_lock_irqsave(&ipc_dev->local_fifo_irq_lock, flags);
+ if (!ipc_dev->local_fifo_irq_enabled) {
+ enable_irq(ipc_dev->local_fifo_irq);
+ ipc_dev->local_fifo_irq_enabled = true;
+ }
+ spin_unlock_irqrestore(&ipc_dev->local_fifo_irq_lock, flags);
+}
+
+/* Add an entry to a remote FIFO. */
+static void remote_fifo_put(struct keembay_ipc_dev *ipc_dev, u32 entry)
+{
+ /*
+ * TIM_IPC_FIFO.
+ *
+ * Write a value to IPC FIFO if not full.
+ *
+ * Increment FIFO write pointer and fill level.
+ *
+ * In normal usage the 6 least-significant bits are reserved for the
+ * writing processor to include its processor ID, 0 <= x <= 62, so it
+ * can determine if the word was written correctly by checking the
+ * appropriate bit of register TIM_IPC_FIFO_OF_FLAG[n].
+ */
+ iowrite32(entry, ipc_dev->remote_fifo_reg + IPC_FIFO);
+}
+
+/*
+ * Check if we have overflowed the remote FIFO.
+ *
+ * Check the overflow register of the remote FIFO to see if we have overflowed
+ * it. If so clear our overflow.
+ *
+ * @return True if we have overflowed the FIFO, false otherwise.
+ *
+ */
+static bool remote_fifo_overflow(struct keembay_ipc_dev *ipc_dev)
+{
+ /*
+ * TIM_IPC_FIFO_OF_FLAG[n]
+ *
+ * Read:
+ *
+ * A processor with ID = x can check that its writes to the IPC FIFO
+ * were successful by reading 0 from bit x of TIM_IPC_FIFO_OF_FLAG0 or
+ * TIM_IPC_FIFO_OF_FLAG1.
+ *
+ * Bit x, 0 <= x <= 31, of TIM_IPC_FIFO_OF_FLAG0 is set high if a write
+ * to TIM_IPC_FIFO by processor ID x failed because the FIFO was full.
+ *
+ * Bit x, 0 <= x <= 30, of TIM_IPC_FIFO_OF_FLAG1 is set high if a write
+ * to TIM_IPC_FIFO by processor ID x+32 failed because the FIFO was
+ * full.
+ *
+ * Processors are identified by the 6 least-significant bits of words
+ * written to TIM_IPC_FIFO, i.e. x = TIM_IPC_FIFO[5:0].
+ * Processor ID = 0x3F is reserved to indicate a read of an empty FIFO
+ * has occurred.
+ *
+ * Write:
+ *
+ * Writing 1 to bit position x of TIM_IPC_FIFO_OF_FLAG0 clears the
+ * overflow flag corresponding to processor ID x. Writing 1 to bit
+ * position x of TIM_IPC_FIFO_OF_FLAG1 clears the overflow flag
+ * corresponding to processor ID x+32.
+ *
+ * Writing 0 to any bit position has not effect.
+ */
+ /* KMB_IPC_NODE_ARM_CSS = 0, so we use IPC_FIFO_OF_FLAG0. */
+ const bool rc = ioread32(ipc_dev->remote_fifo_reg + IPC_FIFO_OF_FLAG0) &
+ BIT(KMB_IPC_NODE_ARM_CSS);
+
+ /* Clear our overflow bit, if we overflowed. */
+ if (rc)
+ iowrite32(BIT(KMB_IPC_NODE_ARM_CSS),
+ ipc_dev->remote_fifo_reg + IPC_FIFO_OF_FLAG0);
+
+ return rc;
+}
+
+/*
+ * IPC internal functions.
+ */
+
+/**
+ * channel_close() - Close a channel and return whether it was open or not.
+ * @link: The link the channel belongs to.
+ * @chan_id: The channel ID of the channel to close.
+ *
+ * Return: 0 if the channel was already closed, 1 otherwise.
+ */
+static int channel_close(struct ipc_link *link, u16 chan_id)
+{
+ struct ipc_chan *chan;
+ struct rx_data *pos, *nxt;
+
+ /* Get channel from channel array. */
+ spin_lock(&link->chan_lock);
+ chan = rcu_dereference_protected(link->channels[chan_id],
+ lockdep_is_held(&link->chan_lock));
+
+ /* If channel is already NULL, we are done. */
+ if (!chan) {
+ spin_unlock(&link->chan_lock);
+ return 0;
+ }
+
+ /* Otherwise remove it from the 'channels' array. */
+ RCU_INIT_POINTER(link->channels[chan_id], NULL);
+ spin_unlock(&link->chan_lock);
+
+ /* Set closing flag and wake up user threads waiting on recv(). */
+ chan->closing = true;
+ wake_up_all(&chan->rx_wait_queue);
+
+ /* Wait for channel users to drop the reference to the old channel. */
+ synchronize_srcu(&link->srcu_sp[chan_id]);
+
+ /* Free channel memory (rx_data queue and channel struct). */
+ /*
+ * No need to get chan->rx_lock as we know that nobody is using the
+ * channel at this point.
+ */
+ list_for_each_entry_safe(pos, nxt, &chan->rx_data_list, list) {
+ list_del(&pos->list);
+ kfree(pos);
+ }
+ kfree(chan);
+
+ return 1;
+}
+
+/**
+ * ipc_buf_tx_alloc() - Allocate an IPC buffer to be used for TX.
+ * @pool: The IPC buffer pool from which the IPC buffer will be allocated.
+ *
+ * Return: The pointer to the allocated buffer, or NULL if allocation fails.
+ */
+static struct kmb_ipc_buf *ipc_buf_tx_alloc(struct ipc_buf_pool *pool)
+{
+ struct kmb_ipc_buf *buf;
+ int i;
+
+ /*
+ * Look for a free buffer starting from the last index (pointing to the
+ * next buffer after the last allocated one) and potentially going
+ * through all the buffers in the pool.
+ */
+ for (i = 0; i < pool->buf_cnt; i++) {
+ /*
+ * Get reference to current buffer and increment index (for
+ * next iteration or function call).
+ */
+ buf = &pool->buffers[pool->idx++];
+ if (pool->idx == pool->buf_cnt)
+ pool->idx = 0;
+
+ /* Use current buffer if free. */
+ if (buf->status == KMB_IPC_BUF_FREE) {
+ buf->status = KMB_IPC_BUF_ALLOCATED;
+ return buf;
+ }
+ }
+
+ /* We went through all the buffers and found none free. */
+ return NULL;
+}
+
+/**
+ * init_ipc_buf_pool() - Init the IPC Buffer Pool.
+ * @ipc_dev: The IPC device the pool belongs to.
+ *
+ * Set up the IPC Buffer Pool to be used for allocating TX buffers.
+ *
+ * The pool uses the local IPC Buffer memory previously allocated.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int init_ipc_buf_pool(struct keembay_ipc_dev *ipc_dev)
+{
+ struct ipc_buf_mem *mem = &ipc_dev->local_ipc_mem;
+
+ ipc_dev->ipc_buf_pool.buf_cnt = mem->size / sizeof(struct kmb_ipc_buf);
+
+ /* Fail if we end up having a pool of 0 buffers. */
+ if (ipc_dev->ipc_buf_pool.buf_cnt == 0)
+ return -ENOMEM;
+ /*
+ * Set reserved memory to 0 to initialize the IPC Buffer array
+ * (ipc_buf_pool.buffers); this works because the value of
+ * KMB_IPC_BUF_FREE is 0.
+ */
+ memset(mem->vaddr, 0, mem->size);
+ ipc_dev->ipc_buf_pool.buffers = mem->vaddr;
+ ipc_dev->ipc_buf_pool.idx = 0;
+
+ return 0;
+}
+
+/*
+ * ipc_link_init() - Initialize CPU/VPU IPC link.
+ * @ipc_dev: The IPC device the link belongs to.
+ *
+ * This function is expected to be called during probing.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int ipc_link_init(struct keembay_ipc_dev *ipc_dev)
+{
+ struct ipc_link *link = &ipc_dev->leon_mss_link;
+ struct tx_queue *queue;
+ int i;
+
+ spin_lock_init(&link->chan_lock);
+ for (i = 0; i < ARRAY_SIZE(link->srcu_sp); i++)
+ init_srcu_struct(&link->srcu_sp[i]);
+ memset(link->channels, 0, sizeof(link->channels));
+
+ /* Init TX queues. */
+ for (i = 0; i < ARRAY_SIZE(link->tx_queues); i++) {
+ queue = &link->tx_queues[i];
+ INIT_LIST_HEAD(&queue->tx_data_list);
+ spin_lock_init(&queue->lock);
+ }
+ link->tx_qidx = 0;
+ link->tx_stopping = false;
+ init_completion(&link->tx_queued);
+
+ /* Start TX thread. */
+ link->tx_thread = kthread_run(tx_thread_fn, ipc_dev,
+ "kmb_ipc_tx_thread");
+ return PTR_ERR_OR_ZERO(link->tx_thread);
+}
+
+/**
+ * ipc_link_deinit() - De-initialize CPU/VPU IPC link.
+ * @ipc_dev: The IPC device the link belongs to.
+ */
+static void ipc_link_deinit(struct keembay_ipc_dev *ipc_dev)
+{
+ struct ipc_link *link = &ipc_dev->leon_mss_link;
+ struct tx_queue *queue;
+ struct tx_data *pos, *nxt;
+ int i;
+
+ /* Close all channels. */
+ for (i = 0; i < ARRAY_SIZE(link->channels); i++)
+ channel_close(link, i);
+
+ /* Stop TX Thread. */
+ link->tx_stopping = true;
+ complete(&link->tx_queued);
+ kthread_stop(link->tx_thread);
+
+ /* Flush all TX queue. */
+ for (i = 0; i < ARRAY_SIZE(link->tx_queues); i++) {
+ queue = &link->tx_queues[i];
+ list_for_each_entry_safe(pos, nxt, &queue->tx_data_list, list) {
+ list_del(&pos->list);
+ pos->retv = -EPIPE;
+ complete(&pos->tx_done);
+ }
+ }
+}
+
+static int get_pdev_res_and_ioremap(struct platform_device *pdev,
+ const char *reg_name,
+ void __iomem **target_reg)
+{
+ struct resource *res;
+ void __iomem *reg;
+ struct device *dev = &pdev->dev;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, reg_name);
+ if (!res) {
+ dev_err(dev, "Couldn't get resource for %s\n", reg_name);
+ return -EINVAL;
+ }
+
+ reg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(reg)) {
+ dev_err(dev, "Couldn't ioremap resource for %s\n", reg_name);
+ return PTR_ERR(reg);
+ }
+
+ *target_reg = reg;
+
+ return 0;
+}
+
+/**
+ * ipc_hw_init() - Init IPC-related hardware.
+ * @ipc_dev: The IPC device to initialize.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int ipc_hw_init(struct keembay_ipc_dev *ipc_dev)
+{
+ struct platform_device *pdev = ipc_dev->plat_dev;
+ struct device *dev = &pdev->dev;
+ int rc, irq;
+
+ rc = get_pdev_res_and_ioremap(pdev, "css_fifo",
+ &ipc_dev->local_fifo_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get local FIFO registers\n");
+ return rc;
+ }
+
+ rc = get_pdev_res_and_ioremap(pdev, "mss_fifo",
+ &ipc_dev->remote_fifo_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get remote FIFO registers\n");
+ return rc;
+ }
+
+ /*
+ * Register interrupt handler and initialize IRQ-related fields in IPC
+ * device struct.
+ */
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ ipc_dev->local_fifo_irq = irq;
+ ipc_dev->local_fifo_irq_enabled = true;
+
+ tasklet_init(&ipc_dev->rx_tasklet, rx_tasklet_func, (uintptr_t)ipc_dev);
+
+ spin_lock_init(&ipc_dev->local_fifo_irq_lock);
+
+ return devm_request_irq(&pdev->dev, irq, local_fifo_irq_handler, 0,
+ dev_name(&pdev->dev), ipc_dev);
+}
+
+/**
+ * ipc_hw_deinit() - De-initialize IPC-related hardware and memory.
+ * @ipc_dev: The IPC device to de-initialize.
+ */
+static void ipc_hw_deinit(struct keembay_ipc_dev *ipc_dev)
+{
+ /*
+ * Just kill the tasklet as iomem and irq have been requested with
+ * devm_* functions and, therefore, are freed automatically.
+ */
+ tasklet_kill(&ipc_dev->rx_tasklet);
+}
+
+/* Driver probing. */
+static int kmb_ipc_probe(struct platform_device *pdev)
+{
+ int rc;
+ struct keembay_ipc_dev *ipc_dev;
+ struct device *dev = &pdev->dev;
+
+ ipc_dev = devm_kzalloc(dev, sizeof(*ipc_dev), GFP_KERNEL);
+ if (!ipc_dev)
+ return -ENOMEM;
+
+ ipc_dev->plat_dev = pdev;
+
+ /* Init the memory used for local IPC buffers. */
+ rc = init_ipc_rsvd_mem(dev, &ipc_dev->local_ipc_mem,
+ "local_rsvd_mem", LOCAL_IPC_BUFFER_IDX);
+ if (rc) {
+ dev_err(dev, "Failed to set up local reserved memory.\n");
+ return rc;
+ }
+
+ /* Init the memory used for remote IPC buffers. */
+ rc = init_ipc_rsvd_mem(dev, &ipc_dev->remote_ipc_mem,
+ "remote_rsvd_mem", REMOTE_IPC_BUFFER_IDX);
+ if (rc) {
+ dev_err(dev, "Failed to set up remote reserved memory.\n");
+ device_unregister(ipc_dev->local_ipc_mem.dev);
+ return rc;
+ }
+
+ /* Init the pool of IPC Buffers to be used to TX. */
+ rc = init_ipc_buf_pool(ipc_dev);
+ if (rc)
+ goto err_post_rsvd_mem;
+
+ /* Init the only link we have (ARM CSS -> LEON MSS). */
+ rc = ipc_link_init(ipc_dev);
+ if (rc)
+ goto err_post_rsvd_mem;
+
+ /* Init IPC HW FIFO. */
+ ipc_hw_init(ipc_dev);
+
+ platform_set_drvdata(pdev, ipc_dev);
+
+ return 0;
+
+err_post_rsvd_mem:
+ device_unregister(ipc_dev->local_ipc_mem.dev);
+ device_unregister(ipc_dev->remote_ipc_mem.dev);
+
+ return rc;
+}
+
+/* Driver removal. */
+static int kmb_ipc_remove(struct platform_device *pdev)
+{
+ struct keembay_ipc_dev *ipc_dev = platform_get_drvdata(pdev);
+
+ ipc_hw_deinit(ipc_dev);
+
+ ipc_link_deinit(ipc_dev);
+
+ /*
+ * No need to de-alloc IPC memory (local_ipc_mem and remote_ipc_mem)
+ * since it was allocated with dmam_alloc_coherent().
+ */
+
+ device_unregister(ipc_dev->local_ipc_mem.dev);
+ device_unregister(ipc_dev->remote_ipc_mem.dev);
+
+ return 0;
+}
+
+/*
+ * ipc_vpu_to_virt() - Convert a VPU address to a CPU virtual address.
+ *
+ * @ipc_mem: The IPC memory region where the VPU address is expected to be
+ * mapped to.
+ * @vpu_addr: The VPU address to be converted to a virtual one.
+ *
+ * The VPU can map the DDR differently than the CPU. This functions converts
+ * VPU addresses to CPU virtual addresses.
+ *
+ * Return: The corresponding CPU virtual address, or NULL if the VPU address
+ * is not in the expected memory range.
+ */
+static void *ipc_vpu_to_virt(const struct ipc_buf_mem *ipc_mem,
+ u32 vpu_addr)
+{
+ if (unlikely(vpu_addr < ipc_mem->dma_handle ||
+ vpu_addr >= (ipc_mem->dma_handle + ipc_mem->size)))
+ return NULL;
+
+ /* Cast to (u8 *) needed since void pointer arithmetic is undefined. */
+ return (u8 *)ipc_mem->vaddr + (vpu_addr - ipc_mem->dma_handle);
+}
+
+/*
+ * ipc_virt_to_vpu() - Convert an CPU virtual address to a VPU address.
+ * @ipc_mem: [in] The IPC memory region where the VPU address is expected to
+ * be mapped to.
+ * @vaddr: [in] The CPU virtual address to be converted to a VPU one.
+ * @vpu_addr: [out] Where to store the computed VPU address.
+ *
+ * The VPU can map the DDR differently than the CPU. This functions converts
+ * CPU virtual addresses to VPU virtual addresses.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int ipc_virt_to_vpu(struct ipc_buf_mem *ipc_mem, void *vaddr,
+ u32 *vpu_addr)
+{
+ if (unlikely((ipc_mem->dma_handle + ipc_mem->size) > 0xFFFFFFFF))
+ return -EINVAL;
+
+ /* Cast to (u8 *) needed since void pointer arithmetic is undefined. */
+ if (unlikely(vaddr < ipc_mem->vaddr ||
+ (u8 *)vaddr >= ((u8 *)ipc_mem->vaddr + ipc_mem->size)))
+ return -EINVAL;
+
+ *vpu_addr = ipc_mem->dma_handle + (vaddr - ipc_mem->vaddr);
+
+ return 0;
+}
+
+/**
+ * process_rx_fifo_entry() - Process a FIFO entry.
+ * @entry: The FIFO entry to process.
+ * @ipc_dev: The IPC device to use.
+ *
+ * This function performs the following tasks:
+ * - Check the source node id.
+ * - Process the IPC buffer (locate it, validate it, read data info, release
+ * buffer).
+ * - Add an RX Data descriptor (data ptr and data size) to the channel RX queue.
+ */
+static void process_rx_fifo_entry(u32 entry,
+ struct keembay_ipc_dev *ipc_dev)
+{
+ struct device *dev = &ipc_dev->plat_dev->dev;
+ struct ipc_link *link = &ipc_dev->leon_mss_link;
+ struct kmb_ipc_buf *ipc_buf;
+ struct rx_data *rx_data;
+ struct ipc_chan *chan;
+ int idx;
+
+ /* Get IPC buffer. */
+ ipc_buf = ipc_vpu_to_virt(&ipc_dev->remote_ipc_mem, entry);
+ if (unlikely(!ipc_buf)) {
+ dev_warn(dev, "RX: Message out of expected memory range: %x\n",
+ entry);
+
+ /* Return immediately (cannot mark the IPC buffer as free). */
+ return;
+ }
+ if (unlikely(ipc_buf->src_node != KMB_IPC_NODE_LEON_MSS)) {
+ dev_warn(dev, "RX: Message from unexpected source: %d\n",
+ ipc_buf->src_node);
+ goto exit;
+ }
+
+ /* Check destination node. */
+ if (unlikely(ipc_buf->dst_node != KMB_IPC_NODE_ARM_CSS)) {
+ dev_warn(dev, "RX: Message is not for this node\n");
+ goto exit;
+ }
+
+ /* Preliminary channel check. */
+ if (unlikely(ipc_buf->channel >= KMB_IPC_MAX_CHANNELS)) {
+ dev_warn(dev, "RX: Message for invalid channel\n");
+ goto exit;
+ }
+
+ /* Access internal channel struct (this is protected by an SRCU). */
+ idx = srcu_read_lock(&link->srcu_sp[ipc_buf->channel]);
+ chan = srcu_dereference(link->channels[ipc_buf->channel],
+ &link->srcu_sp[ipc_buf->channel]);
+ if (unlikely(!chan)) {
+ srcu_read_unlock(&link->srcu_sp[ipc_buf->channel], idx);
+ dev_warn(dev, "RX: Message for closed channel.\n");
+ goto exit;
+ }
+ rx_data = kmalloc(sizeof(*rx_data), GFP_ATOMIC);
+ if (unlikely(!rx_data)) {
+ /* If kmalloc fails, we are forced to discard the message. */
+ srcu_read_unlock(&link->srcu_sp[ipc_buf->channel], idx);
+ dev_err(dev, "RX: Message dropped: Cannot allocate RX Data.\n");
+ goto exit;
+ }
+ /* Read data info. */
+ rx_data->data_vpu_addr = ipc_buf->data_addr;
+ rx_data->data_size = ipc_buf->data_size;
+ /*
+ * Put data info in rx channel queue.
+ *
+ * Note: rx_lock is shared with user context only (and this function is
+ * run in tasklet context), so spin_lock() is enough.
+ */
+ spin_lock(&chan->rx_lock);
+ list_add_tail(&rx_data->list, &chan->rx_data_list);
+ spin_unlock(&chan->rx_lock);
+
+ /* Wake up threads waiting on recv(). */
+ wake_up_interruptible(&chan->rx_wait_queue);
+
+ /* Exit SRCU region protecting chan struct. */
+ srcu_read_unlock(&link->srcu_sp[ipc_buf->channel], idx);
+
+exit:
+ barrier(); /* Ensure IPC buffer is fully processed before release. */
+ ipc_buf->status = KMB_IPC_BUF_FREE;
+}
+
+/*
+ * The function implementing the RX tasklet.
+ *
+ * Go through the RX SW FIFO (filled by the RX ISR) and process every entry.
+ */
+static void rx_tasklet_func(unsigned long ipc_dev_ptr)
+{
+ struct keembay_ipc_dev *ipc_dev = (struct keembay_ipc_dev *)ipc_dev_ptr;
+ struct rx_circ_buf *rx_sw_fifo = &ipc_dev->rx_sw_fifo;
+ u32 entry;
+
+ /*
+ * Memory barrier: make sure that we get buffer head before any other
+ * operation on the circular buffer.
+ */
+ const unsigned int head = smp_load_acquire(&rx_sw_fifo->head);
+ unsigned long tail = rx_sw_fifo->tail;
+ const size_t size = ARRAY_SIZE(rx_sw_fifo->buf);
+
+ while (CIRC_CNT(head, tail, size)) {
+ /* Extract one item from the buffer. */
+
+ entry = rx_sw_fifo->buf[tail];
+ /* Consume the item. */
+ process_rx_fifo_entry(entry, ipc_dev);
+
+ /* Update tail index (wrapping it if needed) */
+ tail = (tail + 1) & (size - 1);
+ }
+ /* Memory barrier ensuring tail is updated only at the end. */
+ smp_store_release(&rx_sw_fifo->tail, tail);
+
+ /* Enable Local FIFO interrupt. */
+ local_fifo_irq_enable(ipc_dev);
+}
+
+/*
+ * Local FIFO ISR.
+ *
+ * The HW FIFO produces interrupts when not empty. This ISR is called to handle
+ * such interrupts.
+ *
+ * The ISR moves FIFO entries from the HW FIFO to an internal SW FIFO
+ * (implemented as a circular buffer) and then activates a tasklet to handle
+ * the entries in the SW FIFO.
+ *
+ * The ISR tries to empty the HW FIFO, thus making it stop generating
+ * interrupts. If it is not possible to empty the HW FIFO because the SW FIFO
+ * is full, the ISR disables the HW FIFO interrupts, thus preventing ISR
+ * re-activation and giving time to the tasklet to consume entries in the SW
+ * FIFO. When the tasklet it is done, it re-enables the interrupts.
+ */
+static irqreturn_t local_fifo_irq_handler(int irq, void *ptr)
+{
+ struct keembay_ipc_dev *ipc_dev = ptr;
+ struct rx_circ_buf *rx_sw_fifo = &ipc_dev->rx_sw_fifo;
+ unsigned int head = rx_sw_fifo->head;
+ const unsigned int tail = READ_ONCE(rx_sw_fifo->tail);
+ const size_t size = ARRAY_SIZE(rx_sw_fifo->buf);
+
+ /* Copy entries to internal SW FIFO. */
+ while (likely(local_fifo_cnt(ipc_dev))) {
+ /* If SW FIFO is full, disable HW FIFO interrupt. */
+ if (unlikely(!CIRC_SPACE(head, tail, size))) {
+ local_fifo_irq_disable(ipc_dev);
+ goto exit;
+ }
+ rx_sw_fifo->buf[head] = local_fifo_get(ipc_dev);
+
+ /* Update head index (wrapping it if needed). */
+ head = (head + 1) & (size - 1);
+ }
+
+exit:
+ /*
+ * Memory barrier: make sure that updating the buffer head is the last
+ * operation done on the circular buffer.
+ */
+ smp_store_release(&rx_sw_fifo->head, head);
+ tasklet_schedule(&ipc_dev->rx_tasklet);
+ return IRQ_HANDLED;
+}
+
+/**
+ * tx_data_send() - Send a TX data element.
+ * @ipc_dev: The IPC device to use.
+ * @tx_data: The TX data element to send.
+ */
+static int tx_data_send(struct keembay_ipc_dev *ipc_dev,
+ struct tx_data *tx_data)
+{
+ struct device *dev = &ipc_dev->plat_dev->dev;
+ struct kmb_ipc_buf *ipc_buf;
+ u32 entry;
+ int rc;
+
+ /* Allocate and set IPC buffer. */
+ ipc_buf = ipc_buf_tx_alloc(&ipc_dev->ipc_buf_pool);
+ if (unlikely(!ipc_buf)) {
+ rc = -ENOMEM;
+ goto exit;
+ }
+
+ /* Prepare IPC buffer. */
+ ipc_buf->channel = tx_data->chan_id;
+ ipc_buf->src_node = KMB_IPC_NODE_ARM_CSS;
+ ipc_buf->dst_node = tx_data->dst_node;
+ ipc_buf->data_addr = tx_data->vpu_addr;
+ ipc_buf->data_size = tx_data->size;
+
+ /* Ensure changes to IPC Buffer are performed before entry is sent. */
+ wmb();
+
+ /* Initialize entry to ipc_buf VPU address. */
+ rc = ipc_virt_to_vpu(&ipc_dev->local_ipc_mem, ipc_buf, &entry);
+
+ /*
+ * Check validity of IPC buffer VPU address (this error should never
+ * occur if IPC buffer region is defined properly in Device Tree).
+ */
+ if (unlikely(rc)) {
+ dev_err(dev, "Cannot convert IPC buf vaddr to vpu_addr: %p\n",
+ ipc_buf);
+ rc = -ENXIO;
+ goto exit;
+ }
+ if (unlikely(!IS_ALIGNED(entry, KMB_IPC_ALIGNMENT))) {
+ dev_err(dev, "Allocated IPC buf is not 64-byte aligned: %p\n",
+ ipc_buf);
+ rc = -EFAULT;
+ goto exit;
+ }
+
+ /* Set Processor ID in entry. */
+ entry |= (KMB_IPC_NODE_ARM_CSS & 0x3F);
+
+ /* Send entry and check for overflow. */
+ remote_fifo_put(ipc_dev, entry);
+ if (remote_fifo_overflow(ipc_dev))
+ rc = -EBUSY;
+
+exit:
+ /* If an error occurred and a buffer was allocated, free it. */
+ if (rc && ipc_buf)
+ ipc_buf->status = KMB_IPC_BUF_FREE;
+ return rc;
+}
+
+/**
+ * tx_data_dequeue() - Dequeue the next TX data waiting for transfer.
+ * @link: The link from which we dequeue the TX Data.
+ *
+ * The de-queue policy is round robin between each high speed channel queue and
+ * the one queue for all low speed channels.
+ *
+ * Return: The next TX data waiting to be transferred, or NULL if no TX is
+ * pending.
+ */
+static struct tx_data *tx_data_dequeue(struct ipc_link *link)
+{
+ struct tx_data *tx_data;
+ struct tx_queue *queue;
+ int i;
+
+ /*
+ * TX queues are logically organized in a circular array.
+ * We go through such an array until we find a non-empty queue.
+ * We start from where we left since last function invocation.
+ * If all queues are empty we return NULL.
+ */
+ for (i = 0; i < ARRAY_SIZE(link->tx_queues); i++) {
+ queue = &link->tx_queues[link->tx_qidx];
+ link->tx_qidx++;
+ if (link->tx_qidx == ARRAY_SIZE(link->tx_queues))
+ link->tx_qidx = 0;
+
+ spin_lock(&queue->lock);
+ tx_data = list_first_entry_or_null(&queue->tx_data_list,
+ struct tx_data, list);
+ /* If no data in the queue, process the next queue. */
+ if (!tx_data) {
+ spin_unlock(&queue->lock);
+ continue;
+ }
+ /* Otherwise remove rx_data from queue and return. */
+ list_del(&tx_data->list);
+ spin_unlock(&queue->lock);
+
+ return tx_data;
+ }
+
+ return NULL;
+}
+
+/**
+ * tx_data_enqueue() - Enqueue TX data for transfer into the specified link.
+ * @link: The link the data is enqueued to.
+ * @tx_data: The TX data to enqueue.
+ */
+static void tx_data_enqueue(struct ipc_link *link, struct tx_data *tx_data)
+{
+ struct tx_queue *queue;
+ int qid;
+
+ /*
+ * Find the right queue where to put TX data:
+ * - Each high-speed channel has a dedicated queue, whose index is the
+ * same as the channel id (e.g., Channel 1 uses tx_queues[1]).
+ * - All the general-purpose channels use the same TX queue, which is
+ * the last element in the tx_queues array.
+ *
+ * Note: tx_queues[] has KMB_IPC_NUM_HS_CHANNELS+1 elements)
+ */
+ qid = tx_data->chan_id < ARRAY_SIZE(link->tx_queues) ?
+ tx_data->chan_id : (ARRAY_SIZE(link->tx_queues) - 1);
+
+ queue = &link->tx_queues[qid];
+
+ /*
+ * Lock shared between user contexts (callers of ipc_send()) and tx
+ * thread; so spin_lock() is enough.
+ */
+ spin_lock(&queue->lock);
+ list_add_tail(&tx_data->list, &queue->tx_data_list);
+ spin_unlock(&queue->lock);
+}
+
+/**
+ * tx_data_remove() - Remove TX data element from specified link.
+ * @link: The link the data is currently enqueued to.
+ * @tx_data: The TX data element to be removed.
+ *
+ * This function is called by the main send function, when the send is
+ * interrupted or has timed out.
+ */
+static void tx_data_remove(struct ipc_link *link, struct tx_data *tx_data)
+{
+ struct tx_queue *queue;
+ int qid;
+
+ /*
+ * Find the TX queue where TX data is currently located:
+ * - Each high-speed channel has a dedicated queue, whose index is the
+ * same as the channel id (e.g., Channel 1 uses tx_queues[1]).
+ * - All the general-purpose channels use the same TX queue, which is
+ * the last element in the tx_queues array.
+ *
+ * Note: tx_queues[] has KMB_IPC_NUM_HS_CHANNELS+1 elements)
+ */
+ qid = tx_data->chan_id < ARRAY_SIZE(link->tx_queues) ?
+ tx_data->chan_id : (ARRAY_SIZE(link->tx_queues) - 1);
+
+ queue = &link->tx_queues[qid];
+
+ /*
+ * Lock shared between user contexts (callers of ipc_send()) and tx
+ * thread; so spin_lock() is enough.
+ */
+ spin_lock(&queue->lock);
+ list_del(&tx_data->list);
+ spin_unlock(&queue->lock);
+}
+
+/**
+ * tx_thread_fn() - The function run by the TX thread.
+ * @ptr: A pointer to the keembay_ipc_dev struct associated with the thread.
+ *
+ * This thread continuously dequeues and send TX data elements. The TX
+ * semaphore is used to pause the loop when all the pending TX data elements
+ * have been transmitted (the send function 'ups' the semaphore every time a
+ * new TX data element is enqueued).
+ */
+static int tx_thread_fn(void *ptr)
+{
+ struct keembay_ipc_dev *ipc_dev = ptr;
+ struct ipc_link *link = &ipc_dev->leon_mss_link;
+ struct tx_data *tx_data;
+ int rc;
+
+ while (1) {
+ rc = wait_for_completion_interruptible(&link->tx_queued);
+ if (rc || link->tx_stopping)
+ break;
+ tx_data = tx_data_dequeue(link);
+ /*
+ * We can get a null tx_data if tx_data_remove() has been
+ * called. Just ignore it and continue.
+ */
+ if (!tx_data)
+ continue;
+ tx_data->retv = tx_data_send(ipc_dev, tx_data);
+ complete(&tx_data->tx_done);
+ }
+
+ /* Wait until kthread_stop() is called. */
+ set_current_state(TASK_INTERRUPTIBLE);
+ while (!kthread_should_stop()) {
+ schedule();
+ set_current_state(TASK_INTERRUPTIBLE);
+ }
+ __set_current_state(TASK_RUNNING);
+
+ return rc;
+}
+
+/* Internal send. */
+static int __ipc_send(struct keembay_ipc_dev *ipc_dev, u8 dst_node,
+ u16 chan_id, u32 vpu_addr, size_t size)
+{
+ struct ipc_link *link = &ipc_dev->leon_mss_link;
+ struct tx_data *tx_data;
+ int rc;
+
+ /* Allocate and init TX data. */
+ tx_data = kmalloc(sizeof(*tx_data), GFP_KERNEL);
+ if (!tx_data)
+ return -ENOMEM;
+ tx_data->dst_node = dst_node;
+ tx_data->chan_id = chan_id;
+ tx_data->vpu_addr = vpu_addr;
+ tx_data->size = size;
+ tx_data->retv = 1;
+ INIT_LIST_HEAD(&tx_data->list);
+ init_completion(&tx_data->tx_done);
+
+ /* Add tx_data to tx queues. */
+ tx_data_enqueue(link, tx_data);
+
+ /* Signal that we have a new pending TX. */
+ complete(&link->tx_queued);
+
+ /* Wait until data is transmitted. */
+ rc = wait_for_completion_interruptible(&tx_data->tx_done);
+ if (unlikely(rc)) {
+ tx_data_remove(link, tx_data);
+ goto exit;
+ }
+ rc = tx_data->retv;
+
+exit:
+ kfree(tx_data);
+ return rc;
+}
+
+/*
+ * Driver allocation.
+ */
+
+/* Device tree driver match. */
+static const struct of_device_id kmb_ipc_of_match[] = {
+ {
+ .compatible = "intel,keembay-ipc",
+ },
+ {}
+};
+
+/* The IPC driver is a platform device. */
+static struct platform_driver kmb_ipc_driver = {
+ .probe = kmb_ipc_probe,
+ .remove = kmb_ipc_remove,
+ .driver = {
+ .name = DRV_NAME,
+ .of_match_table = kmb_ipc_of_match,
+ },
+};
+
+module_platform_driver(kmb_ipc_driver);
+
+/*
+ * Perform basic validity check on common API arguments.
+ *
+ * Verify that the specified device is a Keem Bay IPC device and that node ID
+ * and the channel ID are within the allowed ranges.
+ */
+static int validate_api_args(struct device *dev, u8 node_id, u16 chan_id)
+{
+ if (!dev || dev->driver != &kmb_ipc_driver.driver)
+ return -EINVAL;
+ if (node_id != KMB_IPC_NODE_LEON_MSS) {
+ dev_warn(dev, "Invalid Link ID\n");
+ return -EINVAL;
+ }
+ if (chan_id >= KMB_IPC_MAX_CHANNELS) {
+ dev_warn(dev, "Invalid Channel ID\n");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/*
+ * IPC Kernel API.
+ */
+
+/**
+ * intel_keembay_ipc_open_channel() - Open an IPC channel.
+ * @dev: The IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be opened.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_ipc_open_channel(struct device *dev, u8 node_id, u16 chan_id)
+{
+ struct ipc_chan *chan, *cur_chan;
+ struct keembay_ipc_dev *ipc_dev;
+ struct ipc_link *link;
+ int rc;
+
+ rc = validate_api_args(dev, node_id, chan_id);
+ if (rc)
+ return rc;
+
+ ipc_dev = dev_get_drvdata(dev);
+ link = &ipc_dev->leon_mss_link;
+
+ /* Create channel before getting lock. */
+ chan = kzalloc(sizeof(*chan), GFP_KERNEL);
+ if (!chan)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&chan->rx_data_list);
+ spin_lock_init(&chan->rx_lock);
+ init_waitqueue_head(&chan->rx_wait_queue);
+
+ /* Add channel to the channel array (if not already present). */
+ spin_lock(&link->chan_lock);
+ cur_chan = rcu_dereference_protected(link->channels[chan_id],
+ lockdep_is_held(&link->chan_lock));
+ if (cur_chan) {
+ spin_unlock(&link->chan_lock);
+ kfree(chan);
+ return -EEXIST;
+ }
+ rcu_assign_pointer(link->channels[chan_id], chan);
+ spin_unlock(&link->chan_lock);
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_ipc_open_channel);
+
+/**
+ * intel_keembay_ipc_close_channel() - Close an IPC channel.
+ * @dev: The IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be closed.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_ipc_close_channel(struct device *dev, u8 node_id, u16 chan_id)
+{
+ struct keembay_ipc_dev *ipc_dev;
+ struct ipc_link *link;
+ int rc;
+
+ rc = validate_api_args(dev, node_id, chan_id);
+ if (rc)
+ return rc;
+
+ ipc_dev = dev_get_drvdata(dev);
+ link = &ipc_dev->leon_mss_link;
+
+ rc = channel_close(link, chan_id);
+ if (!rc)
+ dev_info(dev, "Channel was already closed\n");
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_ipc_close_channel);
+
+/**
+ * intel_keembay_ipc_send() - Send data via IPC.
+ * @dev: The IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to be used to send the message.
+ * @vpu_addr: The VPU address of the data to be transferred.
+ * @size: The size of the data to be transferred.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 vpu_addr, size_t size)
+{
+ struct keembay_ipc_dev *ipc_dev;
+ struct ipc_link *link;
+ struct ipc_chan *chan;
+ int idx, rc;
+
+ rc = validate_api_args(dev, node_id, chan_id);
+ if (rc)
+ return rc;
+
+ ipc_dev = dev_get_drvdata(dev);
+ link = &ipc_dev->leon_mss_link;
+ /*
+ * Start Sleepable RCU critical section (this prevents close() from
+ * destroying the channels struct while we are sending data)
+ */
+ idx = srcu_read_lock(&link->srcu_sp[chan_id]);
+
+ /* Get channel. */
+ chan = srcu_dereference(link->channels[chan_id],
+ &link->srcu_sp[chan_id]);
+ if (unlikely(!chan)) {
+ /* The channel is closed. */
+ rc = -ENOENT;
+ goto exit;
+ }
+
+ rc = __ipc_send(ipc_dev, node_id, chan_id, vpu_addr, size);
+
+exit:
+ /* End sleepable RCU critical section. */
+ srcu_read_unlock(&link->srcu_sp[chan_id], idx);
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_ipc_send);
+
+/**
+ * intel_keembay_ipc_recv() - Read data via IPC
+ * @dev: The IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to read from.
+ * @vpu_addr: [out] The VPU address of the received data.
+ * @size: [out] Where to store the size of the received data.
+ * @timeout: How long (in ms) the function will block waiting for an IPC
+ * message; if UINT32_MAX it will block indefinitely; if 0 it
+ * will not block.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *vpu_addr, size_t *size, u32 timeout)
+{
+ struct keembay_ipc_dev *ipc_dev;
+ struct rx_data *rx_entry;
+ struct ipc_link *link;
+ struct ipc_chan *chan;
+ int idx, rc;
+
+ rc = validate_api_args(dev, node_id, chan_id);
+ if (rc)
+ return rc;
+
+ if (!vpu_addr || !size)
+ return -EINVAL;
+
+ ipc_dev = dev_get_drvdata(dev);
+ link = &ipc_dev->leon_mss_link;
+
+ /*
+ * Start Sleepable RCU critical section (this prevents close() from
+ * destroying the channels struct while we are using it)
+ */
+ idx = srcu_read_lock(&link->srcu_sp[chan_id]);
+
+ /* Get channel. */
+ chan = srcu_dereference(link->channels[chan_id],
+ &link->srcu_sp[chan_id]);
+ if (unlikely(!chan)) {
+ rc = -ENOENT;
+ goto err;
+ }
+ /*
+ * Get the lock protecting rx_data_list; the lock will be released by
+ * wait_event_*_lock_irq() before going to sleep and automatically
+ * reacquired after wake up.
+ *
+ * NOTE: lock_irq() is needed because rx_lock is also used by the RX
+ * tasklet; also lock_bh() is not used because there is no
+ * wait_event_interruptible_lock_bh().
+ */
+ spin_lock_irq(&chan->rx_lock);
+ /*
+ * Wait for RX data.
+ *
+ * Note: wait_event_interruptible_lock_irq_timeout() has different
+ * return values than wait_event_interruptible_lock_irq().
+ *
+ * The following if/then branch ensures that return values are
+ * consistent for the both cases, that is:
+ * - rc == 0 only if the wait was successfully (i.e., we were notified
+ * of a message or of a channel closure)
+ * - rc < 0 if an error occurred (we got interrupted or the timeout
+ * expired).
+ */
+ if (timeout == U32_MAX) {
+ rc = wait_event_interruptible_lock_irq(chan->rx_wait_queue,
+ !list_empty(&chan->rx_data_list) ||
+ chan->closing,
+ chan->rx_lock);
+ } else {
+ rc = wait_event_interruptible_lock_irq_timeout(chan->rx_wait_queue,
+ !list_empty(&chan->rx_data_list) ||
+ chan->closing,
+ chan->rx_lock,
+ msecs_to_jiffies(timeout));
+ if (!rc)
+ rc = -ETIME;
+ if (rc > 0)
+ rc = 0;
+ }
+
+ /* Check if the channel was closed while waiting. */
+ if (chan->closing)
+ rc = -EPIPE;
+ if (rc) {
+ spin_unlock_irq(&chan->rx_lock);
+ goto err;
+ }
+
+ /* Extract RX entry. */
+ rx_entry = list_first_entry(&chan->rx_data_list, struct rx_data, list);
+ list_del(&rx_entry->list);
+ spin_unlock_irq(&chan->rx_lock);
+
+ /* Set output parameters. */
+ *vpu_addr = rx_entry->data_vpu_addr;
+ *size = rx_entry->data_size;
+
+ /* Free RX entry. */
+ kfree(rx_entry);
+
+err:
+ /* End sleepable RCU critical section. */
+ srcu_read_unlock(&link->srcu_sp[chan_id], idx);
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_ipc_recv);
+
+MODULE_DESCRIPTION("Keem Bay IPC Driver");
+MODULE_AUTHOR("Daniele Alessandrelli <daniele.alessandrelli@intel.com>");
+MODULE_AUTHOR("Paul Murphy <paul.j.murphy@intel.com>");
+MODULE_LICENSE("GPL");
diff --git a/include/linux/soc/intel/keembay-ipc.h b/include/linux/soc/intel/keembay-ipc.h
new file mode 100644
index 000000000000..bbcf387ecdbd
--- /dev/null
+++ b/include/linux/soc/intel/keembay-ipc.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Keem Bay IPC Linux Kernel API
+ *
+ * Copyright (C) 2018-2020 Intel Corporation
+ */
+
+#ifndef __KEEMBAY_IPC_H
+#define __KEEMBAY_IPC_H
+
+#include <linux/types.h>
+
+/* The possible node IDs. */
+enum {
+ KMB_IPC_NODE_ARM_CSS = 0,
+ KMB_IPC_NODE_LEON_MSS,
+};
+
+int intel_keembay_ipc_open_channel(struct device *dev, u8 node_id, u16 chan_id);
+
+int intel_keembay_ipc_close_channel(struct device *dev, u8 node_id,
+ u16 chan_id);
+
+int intel_keembay_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 vpu_addr, size_t size);
+
+int intel_keembay_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *vpu_addr, size_t *size, u32 timeout);
+
+#endif /* __KEEMBAY_IPC_H */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 04/22] dt-bindings: Add bindings for Keem Bay VPU IPC driver
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (2 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 03/22] keembay-ipc: Add Keem Bay IPC module mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 05/22] keembay-vpu-ipc: Add Keem Bay VPU IPC module mgross
` (18 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Paul Murphy, Rob Herring,
devicetree, Daniele Alessandrelli
From: Paul Murphy <paul.j.murphy@intel.com>
Add DT bindings documentation for the Keem Bay VPU IPC driver.
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Paul Murphy <paul.j.murphy@intel.com>
Co-developed-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../soc/intel/intel,keembay-vpu-ipc.yaml | 151 ++++++++++++++++++
1 file changed, 151 insertions(+)
create mode 100644 Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
diff --git a/Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml b/Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
new file mode 100644
index 000000000000..60d4a028563c
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
@@ -0,0 +1,151 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright (c) Intel Corporation. All rights reserved.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/soc/intel/intel,keembay-vpu-ipc.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel Keem Bay VPU IPC
+
+maintainers:
+ - Paul Murphy <paul.j.murphy@intel.com>
+
+description:
+ The VPU IPC driver facilitates loading of firmware, control, and communication
+ with the VPU over the IPC FIFO in the Intel Keem Bay SoC.
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: intel,keembay-vpu-ipc
+
+ reg:
+ items:
+ - description: NCE WDT registers
+ - description: NCE TIM_GEN_CONFIG registers
+ - description: MSS WDT registers
+ - description: MSS TIM_GEN_CONFIG registers
+
+ reg-names:
+ items:
+ - const: nce_wdt
+ - const: nce_tim_cfg
+ - const: mss_wdt
+ - const: mss_tim_cfg
+
+ memory-region:
+ items:
+ - description: reference to the VPU reserved memory region
+ - description: reference to the X509 reserved memory region
+ - description: reference to the MSS IPC area
+
+ clocks:
+ items:
+ - description: cpu clock
+ - description: pll 0 out 0 rate
+ - description: pll 0 out 1 rate
+ - description: pll 0 out 2 rate
+ - description: pll 0 out 3 rate
+ - description: pll 1 out 0 rate
+ - description: pll 1 out 1 rate
+ - description: pll 1 out 2 rate
+ - description: pll 1 out 3 rate
+ - description: pll 2 out 0 rate
+ - description: pll 2 out 1 rate
+ - description: pll 2 out 2 rate
+ - description: pll 2 out 3 rate
+
+ clocks-names:
+ items:
+ - const: cpu_clock
+ - const: pll_0_out_0
+ - const: pll_0_out_1
+ - const: pll_0_out_2
+ - const: pll_0_out_3
+ - const: pll_1_out_0
+ - const: pll_1_out_1
+ - const: pll_1_out_2
+ - const: pll_1_out_3
+ - const: pll_2_out_0
+ - const: pll_2_out_1
+ - const: pll_2_out_2
+ - const: pll_2_out_3
+
+ interrupts:
+ items:
+ - description: number of NCE sub-system WDT timeout IRQ
+ - description: number of MSS sub-system WDT timeout IRQ
+
+ interrupt-names:
+ items:
+ - const: nce_wdt
+ - const: mss_wdt
+
+ intel,keembay-vpu-ipc-nce-wdt-redirect:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description:
+ Number to which we will request that the NCE sub-system
+ re-directs it's WDT timeout IRQ
+
+ intel,keembay-vpu-ipc-mss-wdt-redirect:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description:
+ Number to which we will request that the MSS sub-system
+ re-directs it's WDT timeout IRQ
+
+ intel,keembay-vpu-ipc-imr:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description:
+ IMR (isolated memory region) number which we will request
+ the runtime service uses to protect the VPU memory region
+ before authentication
+
+ intel,keembay-vpu-ipc-id:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description: The VPU ID to be passed to the VPU firmware.
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ vpu-ipc@3f00209c {
+ compatible = "intel,keembay-vpu-ipc";
+ reg = <0x3f00209c 0x10>,
+ <0x3f003008 0x4>,
+ <0x2082009c 0x10>,
+ <0x20821008 0x4>;
+ reg-names = "nce_wdt",
+ "nce_tim_cfg",
+ "mss_wdt",
+ "mss_tim_cfg";
+ memory-region = <&vpu_reserved>,
+ <&vpu_x509_reserved>,
+ <&mss_ipc_reserved>;
+ clocks = <&scmi_clk 0>,
+ <&scmi_clk 0>,
+ <&scmi_clk 1>,
+ <&scmi_clk 2>,
+ <&scmi_clk 3>,
+ <&scmi_clk 4>,
+ <&scmi_clk 5>,
+ <&scmi_clk 6>,
+ <&scmi_clk 7>,
+ <&scmi_clk 8>,
+ <&scmi_clk 9>,
+ <&scmi_clk 10>,
+ <&scmi_clk 11>;
+ clock-names = "cpu_clock",
+ "pll_0_out_0", "pll_0_out_1",
+ "pll_0_out_2", "pll_0_out_3",
+ "pll_1_out_0", "pll_1_out_1",
+ "pll_1_out_2", "pll_1_out_3",
+ "pll_2_out_0", "pll_2_out_1",
+ "pll_2_out_2", "pll_2_out_3";
+ interrupts = <GIC_SPI 63 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 47 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "nce_wdt", "mss_wdt";
+ intel,keembay-vpu-ipc-nce-wdt-redirect = <63>;
+ intel,keembay-vpu-ipc-mss-wdt-redirect = <47>;
+ intel,keembay-vpu-ipc-imr = <9>;
+ intel,keembay-vpu-ipc-id = <0>;
+ };
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 05/22] keembay-vpu-ipc: Add Keem Bay VPU IPC module
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (3 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 04/22] dt-bindings: Add bindings for Keem Bay VPU IPC driver mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 06/22] misc: xlink-pcie: Add documentation for XLink PCIe driver mgross
` (17 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Paul Murphy, Daniele Alessandrelli
From: Paul Murphy <paul.j.murphy@intel.com>
Intel Keem Bay SoC contains a Vision Processing Unit (VPU) to enable
machine vision and other applications.
Enable Linux to control the VPU processor and provides an interface to
the Keem Bay IPC for communicating with the VPU firmware.
Specifically the driver provides the following functionality to other
kernel components:
- Starting (including loading the VPU firmware) / Stopping / Rebooting
the VPU.
- Getting notifications of VPU events (e.g., WDT events).
- Communicating with the VPU via the Keem Bay IPC mechanism.
In addition to the above, the driver also exposes SoC information (like
stepping, device ID, etc.) to user-space via sysfs. Specifically, the
following sysfs files are provided:
- /sys/firmware/keembay-vpu-ipc/device_id
- /sys/firmware/keembay-vpu-ipc/feature_exclusion
- /sys/firmware/keembay-vpu-ipc/hardware_id
- /sys/firmware/keembay-vpu-ipc/sku
- /sys/firmware/keembay-vpu-ipc/stepping
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Paul Murphy <paul.j.murphy@intel.com>
Co-developed-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: Mark Gross <mgross@linux.intel.com>
---
MAINTAINERS | 9 +
drivers/soc/intel/Kconfig | 15 +
drivers/soc/intel/Makefile | 3 +-
drivers/soc/intel/keembay-vpu-ipc.c | 2036 +++++++++++++++++++++
include/linux/soc/intel/keembay-vpu-ipc.h | 62 +
5 files changed, 2124 insertions(+), 1 deletion(-)
create mode 100644 drivers/soc/intel/keembay-vpu-ipc.c
create mode 100644 include/linux/soc/intel/keembay-vpu-ipc.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 64475ba4ea9d..f95cef927d19 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8963,6 +8963,15 @@ F: Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
F: drivers/soc/intel/keembay-ipc.c
F: include/linux/soc/intel/keembay-ipc.h
+INTEL KEEM BAY VPU IPC DRIVER
+M: Paul J Murphy <paul.j.murphy@intel.com>
+M: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
+M: Mark Gross <mgross@linux.intel.com>
+S: Maintained
+F: Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
+F: drivers/soc/intel/keembay-vpu-ipc.c
+F: include/linux/soc/intel/keembay-vpu-ipc.h
+
INTEL MANAGEMENT ENGINE (mei)
M: Tomas Winkler <tomas.winkler@intel.com>
L: linux-kernel@vger.kernel.org
diff --git a/drivers/soc/intel/Kconfig b/drivers/soc/intel/Kconfig
index 1f5c05b97649..83ce8c4a7851 100644
--- a/drivers/soc/intel/Kconfig
+++ b/drivers/soc/intel/Kconfig
@@ -14,4 +14,19 @@ config KEEMBAY_IPC
Select this if you are compiling the Kernel for an Intel SoC that
includes the Intel Vision Processing Unit (VPU) such as Keem Bay.
+
+config KEEMBAY_VPU_IPC
+ tristate "Intel Keem Bay VPU IPC Driver"
+ depends on KEEMBAY_IPC
+ depends on HAVE_ARM_SMCCC
+ help
+ This option enables support for loading and communicating with
+ the firmware on the Vision Processing Unit (VPU) of the Keem Bay
+ SoC. The driver depends on the Keem Bay IPC driver to do
+ communication, and it depends on secure world monitor software to
+ do the control of the VPU state.
+
+ Select this if you are compiling the Kernel for an Intel SoC that
+ includes the Intel Vision Processing Unit (VPU) such as Keem Bay.
+
endmenu
diff --git a/drivers/soc/intel/Makefile b/drivers/soc/intel/Makefile
index ecf0246e7822..363a81848843 100644
--- a/drivers/soc/intel/Makefile
+++ b/drivers/soc/intel/Makefile
@@ -1,4 +1,5 @@
#
# Makefile for Keem Bay IPC Linux driver
#
-obj-$(CONFIG_KEEMBAY_IPC) += keembay-ipc.o
+obj-$(CONFIG_KEEMBAY_IPC) += keembay-ipc.o
+obj-$(CONFIG_KEEMBAY_VPU_IPC) += keembay-vpu-ipc.o
diff --git a/drivers/soc/intel/keembay-vpu-ipc.c b/drivers/soc/intel/keembay-vpu-ipc.c
new file mode 100644
index 000000000000..bcf9bb4a225a
--- /dev/null
+++ b/drivers/soc/intel/keembay-vpu-ipc.c
@@ -0,0 +1,2036 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Keem Bay VPU IPC Driver.
+ *
+ * Copyright (c) 2018-2020 Intel Corporation.
+ *
+ * The purpose of this driver is to facilitate booting, control and
+ * communication with the VPU IP on the Keem Bay SoC.
+ *
+ * Specifically the driver provides the following functionality to other kernel
+ * components:
+ * - Loading the VPU firmware into DDR for the VPU to execute.
+ * - Starting / Stopping / Rebooting the VPU.
+ * - Getting notifications of VPU events (e.g., WDT events).
+ * - Communicating with the VPU using the Keem Bay IPC mechanism.
+ *
+ * In addition to the above, the driver also exposes SoC information (like
+ * stepping, device ID, etc.) to user-space via sysfs.
+ *
+ *
+ * VPU Firmware loading
+ * --------------------
+ *
+ * The VPU Firmware consists of both the RTOS and the application code meant to
+ * be run by the VPU.
+ *
+ * The VPU Firmware is loaded into DDR using the Linux Firmware API. The
+ * firmware is loaded into a specific reserved memory region in DDR and
+ * executed by the VPU directly from there.
+ *
+ * The VPU Firmware binary is expected to have the following format:
+ *
+ * +------------------+ 0x0
+ * | Header |
+ * +------------------+ 0x1000
+ * | FW Version Area |
+ * +------------------+ 0x2000
+ * | FW Image |
+ * +------------------+ 0x2000 + FW image size
+ * | x509 Certificate |
+ * +------------------+
+ *
+ * Note: the x509 Certificate is ignored for now.
+ *
+ * As part of the firmware loading process, the driver performs the following
+ * operations:
+ * - It parses the VPU firmware binary.
+ * - It loads the FW version area to the DDR location expected by the VPU
+ * firmware and specified in the FW header.
+ * - It loads the FW image to the location specified in the FW header.
+ * - It prepares the boot parameters to be passed to the VPU firmware and loads
+ * them at the location specified in the FW header.
+ *
+ * VPU Start / Stop / Reboot
+ * -------------------------
+ *
+ * The VPU is started / stopped by the SoC firmware, not by this driver
+ * directly. This driver just calls the VPU_BOOT and VPU_STOP SMC SiP services
+ * provided by the SoC firmware.
+ *
+ * Reboot is performed by stopping and re-starting the VPU, including
+ * re-loading the VPU firmware (this is because the VPU firmware .data and .bss
+ * sections need to be re-initialized).
+ *
+ * Sysfs interface
+ * ---------------
+ *
+ * This module exposes SoC information via sysfs. The following sysfs files are
+ * created by this module:
+ * - /sys/firmware/keembay-vpu-ipc/device_id
+ * - /sys/firmware/keembay-vpu-ipc/feature_exclusion
+ * - /sys/firmware/keembay-vpu-ipc/hardware_id
+ * - /sys/firmware/keembay-vpu-ipc/sku
+ * - /sys/firmware/keembay-vpu-ipc/stepping
+ */
+
+#include <linux/arm-smccc.h>
+#include <linux/clk.h>
+#include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include <linux/soc/intel/keembay-ipc.h>
+#include <linux/soc/intel/keembay-vpu-ipc.h>
+
+/* Function ID for the SiP SMC to boot the VPU */
+#define KMB_SIP_SVC_VPU_BOOT 0xFF10
+
+/* Function ID for the SiP SMC to stop the VPU */
+#define KMB_SIP_SVC_VPU_STOP 0xFF16
+
+/* Device tree "memory-region" for VPU firmware area */
+#define VPU_IPC_FW_AREA_IDX 0
+
+/* Device tree region for VPU driver to store X509 region */
+#define VPU_IPC_X509_AREA_IDX 1
+
+/* Device tree "memory-region" for MSS IPC header area */
+#define MSS_IPC_AREA_IDX 2
+
+/*
+ * These are the parameters of the ready message to be received
+ * from the VPU when it is booted correctly.
+ */
+#define READY_MESSAGE_IPC_CHANNEL 0xA
+
+/* Ready message timeout, in ms */
+#define READY_MESSAGE_TIMEOUT_MS 2000
+
+/* Ready message 'physical data address', which is actually a command. */
+#define READY_MESSAGE_EXPECTED_PADDR 0x424f4f54
+
+/* Ready message size */
+#define READY_MESSAGE_EXPECTED_SIZE 0
+
+/* Size of version information in the header */
+#define VPU_VERSION_SIZE 32
+
+/* Version of header that this driver supports. */
+#define HEADER_VERSION_SUPPORTED 0x1
+
+/* Maximum size allowed for firmware version region */
+#define MAX_FIRMWARE_VERSION_SIZE 0x1000
+
+/* Size allowed for header region */
+#define MAX_HEADER_SIZE 0x1000
+
+/* VPU reset vector must be aligned to 4kB. */
+#define VPU_RESET_VECTOR_ALIGNMENT 0x1000
+
+/* Watchdog timer reset trigger */
+#define TIM_WATCHDOG 0x0
+
+/* Watchdog counter enable register */
+#define TIM_WDOG_EN 0x8
+
+/* Write access to protected registers */
+#define TIM_SAFE 0xC
+
+/* Watchdog timer count value */
+#define TIM_WATCHDOG_RESET_VALUE 0xFFFFFFFF
+
+/* Watchdog timer safe value */
+#define TIM_SAFE_ENABLE 0xf1d0dead
+
+/* Watchdog timeout interrupt clear bit */
+#define TIM_GEN_CONFIG_WDOG_TO_INT_CLR BIT(9)
+
+/* Magic number for the boot parameters. */
+#define BOOT_PARAMS_MAGIC_NUMBER 0x00010000
+
+/* Maximum size of string of form "pll_i_out_j" */
+#define PLL_STRING_SIZE 128
+
+/* Number of PLLs */
+#define NUM_PLLS 3
+
+/* Every PLL has this many outputs. */
+#define NUM_PLL_OUTPUTS 4
+
+/* SoC SKU length, in bytes. */
+#define SOC_INFO_SKU_BYTES 6
+
+/* SoC stepping length, in bytes. */
+#define SOC_INFO_STEPPING_BYTES 2
+
+/**
+ * struct boot_parameters - Boot parameters passed to the VPU.
+ * @magic_number: Magic number to indicate structure populated
+ * @vpu_id: ID to be passed to the VPU firmware.
+ * @reserved_0: Reserved memory for other 'header' information
+ * @cpu_frequency_hz: Frequency that the CPU is running at
+ * @pll0_out: Frequency of each of the outputs of PLL 0
+ * @pll1_out: Frequency of each of the outputs of PLL 1
+ * @pll2_out: Frequency of each of the outputs of PLL 2
+ * @reserved_1: Reserved memory for other clock frequencies
+ * @mss_ipc_header_address: Base address of MSS IPC header region
+ * @mss_ipc_header_area_size: Size of MSS IPC header region
+ * @mmio_address: MMIO region for VPU communication
+ * @mmio_area_size: Size of MMIO region for VPU communication
+ * @reserved_2: Reserved memory for other memory regions
+ * @mss_wdt_to_irq_a53_redir: MSS redirects WDT TO IRQ to this ARM IRQ number
+ * @nce_wdt_to_irq_a53_redir: NCE redirects WDT TO IRQ to this ARM IRQ number
+ * @vpu_to_host_irq: VPU to host notification IRQ
+ * @reserved_3: Reserved memory for other configurations
+ * @a53ss_version_id: SoC A53SS_VERSION_ID register value
+ * @si_stepping: Silicon stepping, 2 characters
+ * @device_id: 64 bits of device ID info from fuses
+ * @feature_exclusion: 64 bits of feature exclusion info from fuses
+ * @sku: 64-bit SKU identifier.
+ * @reserved_4: Reserved memory for other information
+ * @reserved_5: Unused/reserved memory
+ */
+struct boot_parameters {
+ /* Header: 0x0 - 0x1F */
+ u32 magic_number;
+ u32 vpu_id;
+ u32 reserved_0[6];
+ /* Clock frequencies: 0x20 - 0xFF */
+ u32 cpu_frequency_hz;
+ u32 pll0_out[NUM_PLL_OUTPUTS];
+ u32 pll1_out[NUM_PLL_OUTPUTS];
+ u32 pll2_out[NUM_PLL_OUTPUTS];
+ u32 reserved_1[43];
+ /* Memory regions: 0x100 - 0x1FF */
+ u64 mss_ipc_header_address;
+ u32 mss_ipc_header_area_size;
+ u64 mmio_address;
+ u32 mmio_area_size;
+ u32 reserved_2[58];
+ /* IRQ re-direct numbers: 0x200 - 0x2FF */
+ u32 mss_wdt_to_irq_a53_redir;
+ u32 nce_wdt_to_irq_a53_redir;
+ u32 vpu_to_host_irq;
+ u32 reserved_3[61];
+ /* Silicon information: 0x300 - 0x3FF */
+ u32 a53ss_version_id;
+ u32 si_stepping;
+ u64 device_id;
+ u64 feature_exclusion;
+ u64 sku;
+ u32 reserved_4[56];
+ /* Unused/reserved: 0x400 - 0xFFF */
+ u32 reserved_5[0x300];
+} __packed;
+
+/**
+ * struct firmware_header - Firmware header information
+ * @header_ver: This header version dictates content structure of
+ * remainder of firmware image, including the header
+ * itself.
+ * @image_format: Image format defines how the loader will handle the
+ * 'firmware image'.
+ * @image_load_addr: VPU address where the firmware image must be loaded to.
+ * @image_size: Size of the image.
+ * @entry_point: Entry point for the VPU firmware (this is a VPU
+ * address).
+ * @vpu_ver: Version of the VPU firmware.
+ * @compression_type: Type of compression used for the VPU firmware.
+ * @fw_ver_load_addr: VPU address where to load the data in the FW version
+ * area of the binary.
+ * @fw_ver_size: Size of the FW version.
+ * @config_load_addr: VPU IPC driver will populate the 4kB of configuration
+ * data to this address.
+ */
+struct firmware_header {
+ u32 header_ver;
+ u32 image_format;
+ u64 image_load_addr;
+ u32 image_size;
+ u64 entry_point;
+ u8 vpu_ver[VPU_VERSION_SIZE];
+ u32 compression_type;
+ u64 fw_ver_load_addr;
+ u32 fw_ver_size;
+ u64 config_load_addr;
+} __packed;
+
+/**
+ * struct vpu_mem - Information about reserved memory shared with VPU.
+ * @vaddr: The virtual address of the memory region.
+ * @paddr: The (CPU) physical address of the memory region.
+ * @vpu_addr: The VPU address of the memory region.
+ * @size: The size of the memory region.
+ */
+struct vpu_mem {
+ void *vaddr;
+ phys_addr_t paddr;
+ dma_addr_t vpu_addr;
+ size_t size;
+};
+
+/**
+ * struct vpu_mem - Information about reserved memory shared with ATF.
+ * @vaddr: The virtual address of the memory region.
+ * @paddr: The physical address of the memory region.
+ * @size: The size of the memory region.
+ */
+struct atf_mem {
+ void __iomem *vaddr;
+ phys_addr_t paddr;
+ size_t size;
+};
+
+/**
+ * struct vpu_ipc_dev - The VPU IPC device structure.
+ * @pdev: Platform device associated with this VPU IPC device.
+ * @state: The current state of the device's finite state machine.
+ * @reserved_mem: VPU firmware reserved memory region. The VPU firmware,
+ * VPU firmware version and the boot parameters are loaded
+ * inside this region.
+ * @x509_mem: x509 reserved memory region.
+ * @mss_ipc_mem: The reserved memory from which the VPU is expected to
+ * allocate its own IPC buffers.
+ * @boot_vec_paddr: The VPU entry point (as specified in the VPU FW binary).
+ * @boot_params: Pointer to the VPU boot parameters.
+ * @fw_res: The memory region where the VPU FW was loaded.
+ * @ready_message_task: The ktrhead instantiated to handle the reception of the
+ * VPU ready message.
+ * @lock: Spinlock protecting @state.
+ * @cpu_clock: The main clock powering the VPU IP.
+ * @pll: Array of PLL clocks.
+ * @nce_irq: IRQ number of the A53 re-direct IRQ used for receiving
+ * the NCE WDT timeout interrupt.
+ * @mss_irq: IRQ number of the A53 re-direct IRQ which will be used
+ * for receiving the MSS WDT timeout interrupt.
+ * @nce_wdt_redirect: Re-direct IRQ for NCE ICB.
+ * @mss_wdt_redirect: Re-direct IRQ for MSS ICB.
+ * @imr: Isolated Memory Region (IMR) to be used to protect the
+ * loaded VPU firmware.
+ * @vpu_id: The ID of the VPU associated with this device.
+ * @nce_wdt_reg: NCE WDT registers.
+ * @nce_tim_cfg_reg: NCE TIM registers.
+ * @mss_wdt_reg: MSS WDT registers.
+ * @mss_tim_cfg_reg: MSS TIM registers.
+ * @nce_wdt_count: Number of NCE WDT timeout event occurred since device
+ * probing.
+ * @mss_wdt_count: Number of MSS WDT timeout event occurred since device
+ * probing.
+ * @ready_queue: Wait queue for waiting on VPU to be ready.
+ * @ipc_dev: The IPC device to use for IPC communication.
+ * @firmware_name: The name of the firmware binary to be loaded.
+ * @callback: The callback executed when CONNECT / DISCONNECT events
+ * happen.
+ */
+struct vpu_ipc_dev {
+ struct platform_device *pdev;
+ enum intel_keembay_vpu_state state;
+ struct vpu_mem reserved_mem;
+ struct atf_mem x509_mem;
+ struct vpu_mem mss_ipc_mem;
+ u64 boot_vec_paddr;
+ struct boot_parameters *boot_params;
+ struct resource fw_res;
+ struct task_struct *ready_message_task;
+ spinlock_t lock; /* Protects the 'state' field above. */
+ struct clk *cpu_clock;
+ struct clk *pll[NUM_PLLS][NUM_PLL_OUTPUTS];
+ int nce_irq;
+ int mss_irq;
+ u32 nce_wdt_redirect;
+ u32 mss_wdt_redirect;
+ u32 imr;
+ u32 vpu_id;
+ void __iomem *nce_wdt_reg;
+ void __iomem *nce_tim_cfg_reg;
+ void __iomem *mss_wdt_reg;
+ void __iomem *mss_tim_cfg_reg;
+ unsigned int nce_wdt_count;
+ unsigned int mss_wdt_count;
+ wait_queue_head_t ready_queue;
+ struct device *ipc_dev;
+ char *firmware_name;
+ void (*callback)(struct device *dev, enum intel_keembay_vpu_event);
+};
+
+/**
+ * struct vpu_ipc_soc_info - VPU SKU information.
+ * @device_id: Value of device ID e-fuse.
+ * @feature_exclusion: Value of feature exclusion e-fuse.
+ * @hardware_id: Hardware identifier.
+ * @stepping: Silicon stepping.
+ * @sku: SKU identifier.
+ *
+ * SoC information read from the device-tree and exported via sysfs.
+ */
+struct vpu_ipc_soc_info {
+ u64 device_id;
+ u64 feature_exclusion;
+ u32 hardware_id;
+ u8 stepping[SOC_INFO_STEPPING_BYTES];
+ u8 sku[SOC_INFO_SKU_BYTES];
+};
+
+/**
+ * enum keembay_vpu_event - Internal events handled by the driver state machine.
+ * @KEEMBAY_VPU_EVENT_BOOT: VPU booted.
+ * @KEEMBAY_VPU_EVENT_BOOT_FAILED: VPU boot failed.
+ * @KEEMBAY_VPU_EVENT_STOP: VPU stop initiated.
+ * @KEEMBAY_VPU_EVENT_STOP_COMPLETE: VPU stop completed.
+ * @KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT: NCE watchdog triggered.
+ * @KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT: MSS watchdog triggered.
+ * @KEEMBAY_VPU_EVENT_MSS_READY_OK: VPU ready message successfully received.
+ * @KEEMBAY_VPU_EVENT_MSS_READY_FAIL: Failed to receive VPU ready message.
+ */
+enum keembay_vpu_event {
+ KEEMBAY_VPU_EVENT_BOOT = 0,
+ KEEMBAY_VPU_EVENT_BOOT_FAILED,
+ KEEMBAY_VPU_EVENT_STOP,
+ KEEMBAY_VPU_EVENT_STOP_COMPLETE,
+ KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT,
+ KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT,
+ KEEMBAY_VPU_EVENT_MSS_READY_OK,
+ KEEMBAY_VPU_EVENT_MSS_READY_FAIL
+};
+
+static struct vpu_ipc_dev *to_vpu_dev(struct device *dev);
+
+/* Variable containing SoC information. */
+static struct vpu_ipc_soc_info *vpu_ipc_soc_info;
+
+/**
+ * vpu_ipc_notify_event() - Trigger callback
+ * @vpu_dev: Private data
+ * @event: Event to notify
+ *
+ * This function is called when an event has occurred. If a callback has
+ * been registered it is called with the device and event as arguments.
+ *
+ * This function can be called from interrupt context.
+ */
+static void vpu_ipc_notify_event(struct vpu_ipc_dev *vpu_dev,
+ enum intel_keembay_vpu_event event)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+
+ if (vpu_dev->callback)
+ vpu_dev->callback(dev, event);
+}
+
+/**
+ * vpu_ipc_handle_event() - Handle events and optionally update state
+ *
+ * @vpu_dev: Private data
+ * @event: Event that has occurred
+ *
+ * This function is called in the case that an event has occurred. This
+ * function tells the calling code if the event is valid for the current state
+ * and also updates the internal state accordingly to the event.
+ *
+ * This function can be called from interrupt context.
+ *
+ * Returns: 0 for success otherwise negative error value
+ */
+static int vpu_ipc_handle_event(struct vpu_ipc_dev *vpu_dev,
+ enum keembay_vpu_event event)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ unsigned long flags;
+ int rc = -EINVAL;
+
+ /*
+ * Atomic update of state.
+ * Note: this function is called by the WDT IRQ handlers; therefore
+ * we must use the spin_lock_irqsave().
+ */
+ spin_lock_irqsave(&vpu_dev->lock, flags);
+
+ switch (vpu_dev->state) {
+ case KEEMBAY_VPU_OFF:
+ if (event == KEEMBAY_VPU_EVENT_BOOT) {
+ vpu_dev->state = KEEMBAY_VPU_BUSY;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_BUSY:
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_OK) {
+ vpu_dev->state = KEEMBAY_VPU_READY;
+ vpu_ipc_notify_event(vpu_dev,
+ KEEMBAY_VPU_NOTIFY_CONNECT);
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_FAIL ||
+ event == KEEMBAY_VPU_EVENT_BOOT_FAILED) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_READY:
+ if (event != KEEMBAY_VPU_EVENT_MSS_READY_OK)
+ vpu_ipc_notify_event(vpu_dev,
+ KEEMBAY_VPU_NOTIFY_DISCONNECT);
+
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_FAIL ||
+ event == KEEMBAY_VPU_EVENT_BOOT_FAILED) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT ||
+ event == KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ break;
+ }
+ fallthrough;
+ case KEEMBAY_VPU_ERROR:
+ if (event == KEEMBAY_VPU_EVENT_BOOT) {
+ vpu_dev->state = KEEMBAY_VPU_BUSY;
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_STOP) {
+ vpu_dev->state = KEEMBAY_VPU_STOPPING;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_STOPPING:
+ if (event == KEEMBAY_VPU_EVENT_STOP_COMPLETE) {
+ vpu_dev->state = KEEMBAY_VPU_OFF;
+ rc = 0;
+ }
+ break;
+ default:
+ break;
+ }
+
+ spin_unlock_irqrestore(&vpu_dev->lock, flags);
+
+ if (rc)
+ dev_err(dev, "Can't handle event %d in state %d\n",
+ event, vpu_dev->state);
+
+ return rc;
+}
+
+/**
+ * clear_and_disable_vpu_wdt() - Clear and disable VPU WDT.
+ * @wdt_base: Base address of the WDT register.
+ * @tim_cfg_base: Base address of the associated TIM configuration
+ * register.
+ */
+static void clear_and_disable_vpu_wdt(u8 __iomem *wdt_base,
+ u8 __iomem *tim_cfg_base)
+{
+ u32 tim_gen_config;
+
+ /* Enable writing and set non-zero WDT value */
+ iowrite32(TIM_SAFE_ENABLE, wdt_base + TIM_SAFE);
+ iowrite32(TIM_WATCHDOG_RESET_VALUE, wdt_base + TIM_WATCHDOG);
+
+ /* Enable writing and disable watchdog timer */
+ iowrite32(TIM_SAFE_ENABLE, wdt_base + TIM_SAFE);
+ iowrite32(0, wdt_base + TIM_WDOG_EN);
+
+ /* Now clear the timeout interrupt */
+ tim_gen_config = ioread32(tim_cfg_base);
+ tim_gen_config &= ~(TIM_GEN_CONFIG_WDOG_TO_INT_CLR);
+ iowrite32(tim_gen_config, tim_cfg_base);
+}
+
+static irqreturn_t nce_wdt_irq_handler(int irq, void *ptr)
+{
+ struct vpu_ipc_dev *vpu_dev = ptr;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ vpu_ipc_notify_event(vpu_dev, KEEMBAY_VPU_NOTIFY_NCE_WDT);
+ dev_dbg_ratelimited(dev, "NCE WDT IRQ occurred.\n");
+
+ clear_and_disable_vpu_wdt(vpu_dev->nce_wdt_reg,
+ vpu_dev->nce_tim_cfg_reg);
+ /* Update driver state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT);
+ if (rc < 0)
+ dev_warn_ratelimited(dev, "Unexpected NCE WDT event.\n");
+
+ vpu_dev->nce_wdt_count++;
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mss_wdt_irq_handler(int irq, void *ptr)
+{
+ struct vpu_ipc_dev *vpu_dev = ptr;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ vpu_ipc_notify_event(vpu_dev, KEEMBAY_VPU_NOTIFY_MSS_WDT);
+ dev_dbg_ratelimited(dev, "MSS WDT IRQ occurred.\n");
+
+ clear_and_disable_vpu_wdt(vpu_dev->mss_wdt_reg,
+ vpu_dev->mss_tim_cfg_reg);
+ /* Update driver state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT);
+ if (rc < 0)
+ dev_warn_ratelimited(dev, "Unexpected MSS WDT event.\n");
+
+ vpu_dev->mss_wdt_count++;
+
+ return IRQ_HANDLED;
+}
+
+static resource_size_t get_reserved_mem_size(struct device *dev,
+ unsigned int idx)
+{
+ struct resource mem;
+ struct device_node *np;
+ int rc;
+
+ np = of_parse_phandle(dev->of_node, "memory-region", idx);
+ if (!np) {
+ pr_err("Couldn't find memory-region %d\n", idx);
+ return 0;
+ }
+
+ rc = of_address_to_resource(np, 0, &mem);
+ if (rc) {
+ pr_err("Couldn't map address to resource\n");
+ return 0;
+ }
+
+ return resource_size(&mem);
+}
+
+static int setup_vpu_fw_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct vpu_mem *rsvd_mem = &vpu_dev->reserved_mem;
+ int rc;
+
+ rc = of_reserved_mem_device_init(dev);
+ if (rc) {
+ dev_err(dev, "Failed to initialise device reserved memory.\n");
+ return rc;
+ }
+
+ rsvd_mem->size = get_reserved_mem_size(dev, VPU_IPC_FW_AREA_IDX);
+ if (rsvd_mem->size == 0) {
+ dev_err(dev, "Couldn't get size of reserved memory region.\n");
+ rc = -ENODEV;
+ goto setup_vpu_fw_fail;
+ }
+
+ rsvd_mem->vaddr = dmam_alloc_coherent(dev, rsvd_mem->size,
+ &rsvd_mem->vpu_addr, GFP_KERNEL);
+ /* Get the physical address of the reserved memory region. */
+ rsvd_mem->paddr = dma_to_phys(dev, vpu_dev->reserved_mem.vpu_addr);
+
+ if (!rsvd_mem->vaddr) {
+ dev_err(dev, "Failed to allocate memory for firmware.\n");
+ rc = -ENOMEM;
+ goto setup_vpu_fw_fail;
+ }
+
+ return 0;
+
+setup_vpu_fw_fail:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+static int setup_x509_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct device_node *node;
+ struct resource res;
+ int rc;
+
+ node = of_parse_phandle(dev->of_node, "memory-region",
+ VPU_IPC_X509_AREA_IDX);
+ if (!node) {
+ dev_err(dev, "Couldn't find X509 region.\n");
+ return -EINVAL;
+ }
+
+ rc = of_address_to_resource(node, 0, &res);
+
+ /* Release node first as we will not use it anymore */
+ of_node_put(node);
+
+ if (rc) {
+ dev_err(dev, "Couldn't resolve X509 region.\n");
+ return rc;
+ }
+
+ vpu_dev->x509_mem.vaddr =
+ devm_ioremap(dev, res.start, resource_size(&res));
+ if (!vpu_dev->x509_mem.vaddr) {
+ dev_err(dev, "Couldn't ioremap x509 region.\n");
+ return -EADDRNOTAVAIL;
+ }
+
+ vpu_dev->x509_mem.paddr = res.start;
+ vpu_dev->x509_mem.size = resource_size(&res);
+
+ return 0;
+}
+
+static int setup_mss_ipc_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct device_node *node;
+ struct resource res;
+ int rc;
+
+ node = of_parse_phandle(dev->of_node, "memory-region",
+ MSS_IPC_AREA_IDX);
+ if (!node) {
+ dev_err(dev, "Didn't find MSS IPC region.\n");
+ return -EINVAL;
+ }
+
+ rc = of_address_to_resource(node, 0, &res);
+ if (rc) {
+ dev_err(dev, "Couldn't resolve MSS IPC region.\n");
+ return rc;
+ }
+ of_node_put(node);
+
+ vpu_dev->mss_ipc_mem.paddr = res.start;
+ vpu_dev->mss_ipc_mem.vpu_addr = phys_to_dma(dev, res.start);
+ vpu_dev->mss_ipc_mem.size = resource_size(&res);
+
+ return 0;
+}
+
+static int setup_reserved_memory(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ /*
+ * Find the VPU firmware area described in the device tree,
+ * and allocate it for our usage.
+ */
+ rc = setup_vpu_fw_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to init FW memory.\n");
+ return rc;
+ }
+
+ /*
+ * Find the X509 area described in the device tree,
+ * and allocate it for our usage.
+ */
+ rc = setup_x509_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to setup X509 region.\n");
+ goto res_mem_setup_fail;
+ }
+
+ /*
+ * Find the MSS IPC area in the device tree and get the location and
+ * size information
+ */
+ rc = setup_mss_ipc_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Couldn't setup MSS IPC region.\n");
+ goto res_mem_setup_fail;
+ }
+
+ return 0;
+
+res_mem_setup_fail:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+static void ipc_device_put(struct vpu_ipc_dev *vpu_dev)
+{
+ put_device(vpu_dev->ipc_dev);
+}
+
+static int ipc_device_get(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct platform_device *pdev;
+ struct device_node *np;
+
+ np = of_parse_phandle(dev->of_node, "intel,keembay-ipc", 0);
+ if (!np) {
+ dev_err(dev, "Cannot find phandle to IPC device\n");
+ return -ENODEV;
+ }
+
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ dev_info(dev, "IPC device not probed\n");
+ of_node_put(np);
+ return -EPROBE_DEFER;
+ }
+
+ vpu_dev->ipc_dev = get_device(&pdev->dev);
+ of_node_put(np);
+
+ return 0;
+}
+
+static int retrieve_clocks(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ char pll_string[PLL_STRING_SIZE];
+ struct clk *clk;
+ int pll;
+ int out;
+
+ clk = devm_clk_get(dev, "cpu_clock");
+ if (IS_ERR(clk)) {
+ dev_err(dev, "cpu_clock not found.\n");
+ return PTR_ERR(clk);
+ }
+ vpu_dev->cpu_clock = clk;
+
+ for (pll = 0; pll < NUM_PLLS; pll++) {
+ for (out = 0; out < NUM_PLL_OUTPUTS; out++) {
+ sprintf(pll_string, "pll_%d_out_%d", pll, out);
+ clk = devm_clk_get(dev, pll_string);
+ if (IS_ERR(clk)) {
+ dev_err(dev, "%s not found.\n", pll_string);
+ return PTR_ERR(clk);
+ }
+ vpu_dev->pll[pll][out] = clk;
+ }
+ }
+
+ return 0;
+}
+
+/* Get register resource from device tree and re-map as I/O memory. */
+static int get_pdev_res_and_ioremap(struct platform_device *pdev,
+ const char *reg_name,
+ void __iomem **target_reg)
+{
+ struct device *dev = &pdev->dev;
+ struct resource *res;
+ void __iomem *reg;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, reg_name);
+ if (!res) {
+ dev_err(dev, "Couldn't get resource for %s\n", reg_name);
+ return -EINVAL;
+ }
+
+ reg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(reg)) {
+ dev_err(dev, "Couldn't ioremap resource for %s\n", reg_name);
+ return PTR_ERR(reg);
+ }
+
+ *target_reg = reg;
+
+ return 0;
+}
+
+static int setup_watchdog_resources(struct vpu_ipc_dev *vpu_dev)
+{
+ struct platform_device *pdev = vpu_dev->pdev;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ /* Get registers */
+ rc = get_pdev_res_and_ioremap(pdev, "nce_wdt", &vpu_dev->nce_wdt_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get NCE WDT registers\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "nce_tim_cfg",
+ &vpu_dev->nce_tim_cfg_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get NCE TIM_GEN_CONFIG register\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "mss_wdt", &vpu_dev->mss_wdt_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get MSS WDT registers\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "mss_tim_cfg",
+ &vpu_dev->mss_tim_cfg_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get MSS TIM_GEN_CONFIG register\n");
+ return rc;
+ }
+
+ /* Request interrupts */
+ vpu_dev->nce_irq = platform_get_irq_byname(pdev, "nce_wdt");
+ if (vpu_dev->nce_irq < 0)
+ return vpu_dev->nce_irq;
+ vpu_dev->mss_irq = platform_get_irq_byname(pdev, "mss_wdt");
+ if (vpu_dev->mss_irq < 0)
+ return vpu_dev->mss_irq;
+ rc = devm_request_irq(dev, vpu_dev->nce_irq, nce_wdt_irq_handler, 0,
+ "keembay-vpu-ipc", vpu_dev);
+ if (rc) {
+ dev_err(dev, "failed to request NCE IRQ.\n");
+ return rc;
+ }
+ rc = devm_request_irq(dev, vpu_dev->mss_irq, mss_wdt_irq_handler, 0,
+ "keembay-vpu-ipc", vpu_dev);
+ if (rc) {
+ dev_err(dev, "failed to request MSS IRQ.\n");
+ return rc;
+ }
+
+ /* Request interrupt re-direct numbers */
+ rc = of_property_read_u32(dev->of_node,
+ "intel,keembay-vpu-ipc-nce-wdt-redirect",
+ &vpu_dev->nce_wdt_redirect);
+ if (rc) {
+ dev_err(dev, "failed to get NCE WDT redirect number.\n");
+ return rc;
+ }
+ rc = of_property_read_u32(dev->of_node,
+ "intel,keembay-vpu-ipc-mss-wdt-redirect",
+ &vpu_dev->mss_wdt_redirect);
+ if (rc) {
+ dev_err(dev, "failed to get MSS WDT redirect number.\n");
+ return rc;
+ }
+
+ return 0;
+}
+
+/* Populate the boot parameters to be passed to the VPU. */
+static int setup_boot_parameters(struct vpu_ipc_dev *vpu_dev)
+{
+ int i;
+
+ /* Set all values to zero. This will disable most clocks/devices */
+ memset(vpu_dev->boot_params, 0, sizeof(*vpu_dev->boot_params));
+
+ /*
+ * Set magic number, so VPU knows that the parameters are
+ * populated correctly
+ */
+ vpu_dev->boot_params->magic_number = BOOT_PARAMS_MAGIC_NUMBER;
+
+ /* Set VPU ID. */
+ vpu_dev->boot_params->vpu_id = vpu_dev->vpu_id;
+
+ /* Inform VPU of clock frequencies */
+ vpu_dev->boot_params->cpu_frequency_hz =
+ clk_get_rate(vpu_dev->cpu_clock);
+ for (i = 0; i < NUM_PLL_OUTPUTS; i++) {
+ vpu_dev->boot_params->pll0_out[i] =
+ clk_get_rate(vpu_dev->pll[0][i]);
+ vpu_dev->boot_params->pll1_out[i] =
+ clk_get_rate(vpu_dev->pll[1][i]);
+ vpu_dev->boot_params->pll2_out[i] =
+ clk_get_rate(vpu_dev->pll[2][i]);
+ }
+
+ /*
+ * Fill in IPC buffer information: the VPU needs to know where it
+ * should allocate IPC buffer from.
+ */
+ vpu_dev->boot_params->mss_ipc_header_address =
+ vpu_dev->mss_ipc_mem.vpu_addr;
+ vpu_dev->boot_params->mss_ipc_header_area_size =
+ vpu_dev->mss_ipc_mem.size;
+
+ /* Fill in IRQ re-direct request information */
+ vpu_dev->boot_params->mss_wdt_to_irq_a53_redir =
+ vpu_dev->mss_wdt_redirect;
+ vpu_dev->boot_params->nce_wdt_to_irq_a53_redir =
+ vpu_dev->nce_wdt_redirect;
+
+ /* Setup A53SS_VERSION_ID */
+ vpu_dev->boot_params->a53ss_version_id = vpu_ipc_soc_info->hardware_id;
+
+ /* Setup Silicon stepping */
+ vpu_dev->boot_params->si_stepping = vpu_ipc_soc_info->stepping[0] |
+ vpu_ipc_soc_info->stepping[1] << 8;
+
+ /* Set feature exclude and device id information. */
+ vpu_dev->boot_params->device_id = vpu_ipc_soc_info->device_id;
+ vpu_dev->boot_params->feature_exclusion =
+ vpu_ipc_soc_info->feature_exclusion;
+
+ /* Set SKU information */
+ vpu_dev->boot_params->sku = (u64)vpu_ipc_soc_info->sku[0] |
+ (u64)vpu_ipc_soc_info->sku[1] << 8 |
+ (u64)vpu_ipc_soc_info->sku[2] << 16 |
+ (u64)vpu_ipc_soc_info->sku[3] << 24 |
+ (u64)vpu_ipc_soc_info->sku[4] << 32 |
+ (u64)vpu_ipc_soc_info->sku[5] << 40;
+ return 0;
+}
+
+/* Request SoC firmware to boot the VPU. */
+static int request_vpu_boot(struct vpu_ipc_dev *vpu_dev)
+{
+ u64 function_id;
+ struct arm_smccc_res res;
+ u16 function_number = KMB_SIP_SVC_VPU_BOOT;
+
+ function_id = ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,
+ ARM_SMCCC_OWNER_SIP, function_number);
+
+ /*
+ * Arguments are as follows:
+ * 1. Reserved
+ * 2. Reserved region size
+ * 3. Firmware image physical base address
+ * 4. Firmware image size
+ * 5. Entry point for VPU
+ * 6. X509 certificate location
+ * 7. IMR driver number
+ */
+ arm_smccc_smc(function_id, 0,
+ vpu_dev->reserved_mem.size, vpu_dev->fw_res.start,
+ resource_size(&vpu_dev->fw_res), vpu_dev->boot_vec_paddr,
+ vpu_dev->x509_mem.paddr, vpu_dev->imr, &res);
+
+ if (res.a0) {
+ dev_info(&vpu_dev->pdev->dev, "Boot failed: 0x%lx.\n", res.a0);
+ return -EIO;
+ }
+
+ dev_info(&vpu_dev->pdev->dev,
+ "VPU Boot successfully requested to secure monitor.\n");
+
+ return 0;
+}
+
+/* Request SoC firmware to stop the VPU. */
+static int request_vpu_stop(struct vpu_ipc_dev *vpu_dev)
+{
+ u64 function_id;
+ struct arm_smccc_res res;
+ u16 function_number = KMB_SIP_SVC_VPU_STOP;
+
+ function_id = ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,
+ ARM_SMCCC_OWNER_SIP, function_number);
+
+ arm_smccc_smc(function_id, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (res.a0) {
+ dev_info(&vpu_dev->pdev->dev, "Stop failed: 0x%lx.\n", res.a0);
+ return -EIO;
+ }
+
+ dev_info(&vpu_dev->pdev->dev,
+ "VPU Stop successfully requested to secure monitor.\n");
+
+ return 0;
+}
+
+/*
+ * Get kernel virtual address of resource inside the VPU reserved memory
+ * region.
+ */
+static void *get_vpu_dev_vaddr(struct vpu_ipc_dev *vpu_dev,
+ struct resource *res)
+{
+ unsigned long offset;
+
+ /* Given the calculation below, this must not be true. */
+ if (res->start < vpu_dev->reserved_mem.vpu_addr)
+ return NULL;
+
+ offset = res->start - vpu_dev->reserved_mem.vpu_addr;
+
+ /* Cast to (u8 *) since void pointer arithmetic is undefined. */
+ return (u8 *)vpu_dev->reserved_mem.vaddr + offset;
+}
+
+static int parse_fw_header(struct vpu_ipc_dev *vpu_dev,
+ const struct firmware *fw)
+{
+ struct resource config_res, version_res, total_reserved_res;
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct firmware_header *fw_header;
+ void *version_region;
+ void *config_region;
+ void *fw_region;
+
+ /* Is the fw size big enough to read the header? */
+ if (fw->size < sizeof(struct firmware_header)) {
+ dev_err(dev, "Firmware was too small for header.\n");
+ return -EINVAL;
+ }
+
+ fw_header = (struct firmware_header *)fw->data;
+
+ /* Check header version */
+ if (fw_header->header_ver != HEADER_VERSION_SUPPORTED) {
+ dev_err(dev, "Header version check expected 0x%x, got 0x%x\n",
+ HEADER_VERSION_SUPPORTED, fw_header->header_ver);
+ return -EINVAL;
+ }
+
+ /* Check firmware version size is allowed */
+ if (fw_header->fw_ver_size > MAX_FIRMWARE_VERSION_SIZE) {
+ dev_err(dev, "Firmware version area larger than allowed: %d\n",
+ fw_header->fw_ver_size);
+ return -EINVAL;
+ }
+
+ /*
+ * Check the firmware binary is at least large enough for the
+ * firmware image size described in the header.
+ */
+ if (fw->size < (MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE +
+ fw_header->image_size)) {
+ dev_err(dev, "Real firmware size is not large enough.\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Make sure that the final address is aligned correctly. If not, the
+ * boot will never work.
+ */
+ if (!IS_ALIGNED(fw_header->entry_point, VPU_RESET_VECTOR_ALIGNMENT)) {
+ dev_err(dev,
+ "Entry point for firmware (0x%llX) is not correctly aligned.\n",
+ fw_header->entry_point);
+ return -EINVAL;
+ }
+
+ /*
+ * Generate the resource describing the region containing the actual
+ * firmware data.
+ */
+ vpu_dev->fw_res.start = fw_header->image_load_addr;
+ vpu_dev->fw_res.end = fw_header->image_size +
+ fw_header->image_load_addr - 1;
+ vpu_dev->fw_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region containing the
+ * configuration data for the VPU.
+ */
+ config_res.start = fw_header->config_load_addr;
+ config_res.end = sizeof(struct boot_parameters) +
+ fw_header->config_load_addr - 1;
+ config_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region containing the
+ * version information for the VPU.
+ */
+ version_res.start = fw_header->fw_ver_load_addr;
+ version_res.end = fw_header->fw_ver_size +
+ fw_header->fw_ver_load_addr - 1;
+ version_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region of memory
+ * completely dedicated to the VPU.
+ */
+ total_reserved_res.start = vpu_dev->reserved_mem.vpu_addr;
+ total_reserved_res.end = vpu_dev->reserved_mem.vpu_addr +
+ vpu_dev->reserved_mem.size - 1;
+ total_reserved_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Check all pieces to be copied reside completely in the reserved
+ * region
+ */
+ if (!resource_contains(&total_reserved_res, &vpu_dev->fw_res)) {
+ dev_err(dev, "Can't fit firmware in reserved region.\n");
+ return -EINVAL;
+ }
+ if (!resource_contains(&total_reserved_res, &version_res)) {
+ dev_err(dev,
+ "Can't fit firmware version data in reserved region.\n");
+ return -EINVAL;
+ }
+ if (!resource_contains(&total_reserved_res, &config_res)) {
+ dev_err(dev,
+ "Can't fit configuration information in reserved region.\n");
+ return -EINVAL;
+ }
+
+ /* Check for overlapping regions */
+ if (resource_overlaps(&vpu_dev->fw_res, &version_res)) {
+ dev_err(dev, "FW and version regions overlap.\n");
+ return -EINVAL;
+ }
+ if (resource_overlaps(&vpu_dev->fw_res, &config_res)) {
+ dev_err(dev, "FW and config regions overlap.\n");
+ return -EINVAL;
+ }
+ if (resource_overlaps(&config_res, &version_res)) {
+ dev_err(dev, "Version and config regions overlap.\n");
+ return -EINVAL;
+ }
+
+ /* Setup boot parameter region */
+ config_region = get_vpu_dev_vaddr(vpu_dev, &config_res);
+ if (!config_region) {
+ dev_err(dev,
+ "Couldn't map boot configuration area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+ version_region = get_vpu_dev_vaddr(vpu_dev, &version_res);
+ if (!version_region) {
+ dev_err(dev,
+ "Couldn't map version area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+ fw_region = get_vpu_dev_vaddr(vpu_dev, &vpu_dev->fw_res);
+ if (!fw_region) {
+ dev_err(dev,
+ "Couldn't map firmware area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Copy version region: the region is located in the file @ offset of
+ * MAX_HEADER_SIZE, size was specified in the header and has been
+ * checked to not be larger than that allowed.
+ */
+ memcpy(version_region, &fw->data[MAX_HEADER_SIZE],
+ fw_header->fw_ver_size);
+
+ /*
+ * Copy firmware region: the region is located in the file @ offset of
+ * MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE, size was specified in
+ * the header and has been checked to not be larger than that allowed.
+ */
+ memcpy(fw_region,
+ &fw->data[MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE],
+ fw_header->image_size);
+
+ /* Save off boot parameters region vaddr */
+ vpu_dev->boot_params = config_region;
+
+ /* Save off boot vector physical address */
+ vpu_dev->boot_vec_paddr = fw_header->entry_point;
+
+ return 0;
+}
+
+static int ready_message_wait_thread(void *arg)
+{
+ struct vpu_ipc_dev *vpu_dev = arg;
+ struct device *dev = &vpu_dev->pdev->dev;
+ size_t size = 0;
+ u32 paddr = 0;
+ int close_rc;
+ int rc;
+
+ /*
+ * We will wait a few seconds for the message. We will complete earlier
+ * if the message is received earlier.
+ * NOTE: this is not a busy wait, we sleep until message is received.
+ */
+ rc = intel_keembay_ipc_recv(vpu_dev->ipc_dev, KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL, &paddr, &size,
+ READY_MESSAGE_TIMEOUT_MS);
+ /*
+ * IPC channel must be closed regardless of 'rc' value, so close the
+ * channel now and then process 'rc' value.
+ */
+ close_rc = intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (close_rc < 0) {
+ dev_warn(dev, "Couldn't close IPC channel.\n");
+ /* Continue, as this is not a critical issue. */
+ }
+
+ /* Now process recv() return code. */
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to receive ready message within %d ms: %d.\n",
+ READY_MESSAGE_TIMEOUT_MS, rc);
+ goto ready_message_thread_failure;
+ }
+
+ if (paddr != READY_MESSAGE_EXPECTED_PADDR ||
+ size != READY_MESSAGE_EXPECTED_SIZE) {
+ dev_err(dev, "Bad ready message: (paddr, size) = (0x%x, %zu)\n",
+ paddr, size);
+ goto ready_message_thread_failure;
+ }
+
+ dev_info(dev, "VPU ready message received successfully!\n");
+
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_READY_OK);
+ if (rc < 0)
+ dev_err(dev, "Fatal error: failed to set state (ready ok).\n");
+
+ /* Wake up anyone waiting for READY. */
+ wake_up_all(&vpu_dev->ready_queue);
+
+ return 0;
+
+ready_message_thread_failure:
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_READY_FAIL);
+ if (rc < 0)
+ dev_err(dev,
+ "Fatal error: failed to set state (ready timeout).\n");
+
+ return 0;
+}
+
+static int create_ready_message_thread(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct task_struct *task;
+
+ task = kthread_run(&ready_message_wait_thread, (void *)vpu_dev,
+ "keembay-vpu-ipc-ready");
+ if (IS_ERR(task)) {
+ dev_err(dev, "Couldn't start thread to receive message.\n");
+ return -EIO;
+ }
+
+ vpu_dev->ready_message_task = task;
+
+ return 0;
+}
+
+static int kickoff_vpu_sequence(struct vpu_ipc_dev *vpu_dev,
+ int (*boot_fn)(struct vpu_ipc_dev *))
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ int err_rc;
+ int rc;
+
+ /*
+ * Open the IPC channel. If we don't do it before booting
+ * the VPU, we may miss the message, as the IPC driver will
+ * discard messages for unopened channels.
+ */
+ rc = intel_keembay_ipc_open_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (rc < 0) {
+ dev_err(dev,
+ "Couldn't open IPC channel to receive ready message.\n");
+ goto kickoff_failed;
+ }
+
+ /* Request boot */
+ rc = boot_fn(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev, "Failed to do request to boot.\n");
+ goto close_and_kickoff_failed;
+ }
+
+ /*
+ * Start thread waiting for message, and update state
+ * if the request was successful.
+ */
+ rc = create_ready_message_thread(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to create thread to wait for ready message.\n");
+ goto close_and_kickoff_failed;
+ }
+
+ return 0;
+
+close_and_kickoff_failed:
+ /* Close the channel. */
+ err_rc = intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (err_rc < 0) {
+ dev_err(dev, "Couldn't close IPC channel: %d\n", err_rc);
+ /*
+ * We have had a more serious failure - don't update the
+ * original 'rc' and continue.
+ */
+ }
+
+kickoff_failed:
+ return rc;
+}
+
+/*
+ * Try to boot the VPU using the firmware name stored in
+ * vpu_dev->firmware_name (which when this function is called is expected to be
+ * not NULL).
+ */
+static int do_boot_sequence(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ const struct firmware *fw;
+ int event_rc;
+ int rc;
+
+ /* Update state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_BOOT);
+ if (rc < 0) {
+ dev_err(dev, "Can't start in this state.\n");
+ return rc;
+ }
+
+ /* Stop the VPU running */
+ rc = request_vpu_stop(vpu_dev);
+ if (rc < 0)
+ dev_err(dev, "Failed stop - continue sequence anyway.\n");
+
+ dev_info(dev, "Keem Bay VPU IPC start with %s.\n",
+ vpu_dev->firmware_name);
+
+ /* Request firmware and wait for it. */
+ rc = request_firmware(&fw, vpu_dev->firmware_name, &vpu_dev->pdev->dev);
+ if (rc < 0) {
+ dev_err(dev, "Couldn't find firmware: %d\n", rc);
+ goto boot_failed_no_fw;
+ }
+
+ /* Do checks on the firmware header. */
+ rc = parse_fw_header(vpu_dev, fw);
+ if (rc < 0) {
+ dev_err(dev, "Firmware checks failed.\n");
+ goto boot_failed;
+ }
+
+ /* Write configuration data. */
+ rc = setup_boot_parameters(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev, "Failed to set up boot parameters.\n");
+ goto boot_failed;
+ }
+
+ /* Try 'boot' sequence */
+ rc = kickoff_vpu_sequence(vpu_dev, request_vpu_boot);
+ if (rc < 0) {
+ dev_err(dev, "Failed to boot VPU.\n");
+ goto boot_failed;
+ }
+
+ release_firmware(fw);
+ return 0;
+
+boot_failed:
+ release_firmware(fw);
+
+boot_failed_no_fw:
+ /* Update state machine after failure. */
+ event_rc = vpu_ipc_handle_event(vpu_dev,
+ KEEMBAY_VPU_EVENT_BOOT_FAILED);
+ if (event_rc < 0) {
+ dev_err(dev,
+ "Unexpected error: failed to handle fail event: %d.\n",
+ event_rc);
+ /* Continue: prefer original 'rc' to 'event_rc'. */
+ }
+
+ return rc;
+}
+
+/**
+ * intel_keembay_vpu_ipc_open_channel() - Open an IPC channel.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be opened.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_vpu_ipc_open_channel(struct device *dev, u8 node_id,
+ u16 chan_id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_open_channel(vpu_dev->ipc_dev, node_id,
+ chan_id);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_open_channel);
+
+/**
+ * intel_keembay_vpu_ipc_close_channel() - Close an IPC channel.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be closed.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+
+int intel_keembay_vpu_ipc_close_channel(struct device *dev, u8 node_id,
+ u16 chan_id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ node_id, chan_id);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_close_channel);
+
+/**
+ * intel_keembay_vpu_ipc_send() - Send data via IPC.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to be used to send the message.
+ * @vpu_addr: The VPU address of the data to be transferred.
+ * @size: The size of the data to be transferred.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_vpu_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 vpu_addr, size_t size)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_send(vpu_dev->ipc_dev, node_id, chan_id,
+ vpu_addr, size);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_send);
+
+/**
+ * intel_keembay_vpu_ipc_recv() - Read data via IPC
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to read from.
+ * @vpu_addr: [out] The VPU address of the received data.
+ * @size: [out] Where to store the size of the received data.
+ * @timeout: How long (in ms) the function will block waiting for an IPC
+ * message; if UINT32_MAX it will block indefinitely; if 0 it
+ * will not block.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *vpu_addr, size_t *size, u32 timeout)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_recv(vpu_dev->ipc_dev, node_id, chan_id,
+ vpu_addr, size, timeout);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_recv);
+
+/**
+ * intel_keembay_vpu_startup() - Boot the VPU
+ * @dev: The VPU device to boot.
+ * @firmware_name: Name of firmware file
+ *
+ * This API is only valid while the VPU is OFF.
+ *
+ * The firmware called "firmware_name" will be searched for using the
+ * kernel firmware API. The firmware header will then be parsed. This driver
+ * will load requested information to the reserved memory region, including
+ * initialisation data. Lastly, we will request the secure world to do the
+ * boot sequence. If the boot sequence is successful, the
+ * VPU state will become BUSY. The caller should then wait for the status to
+ * become READY before starting to communicate with the VPU. If the boot
+ * sequence failed, this function will fail and the caller may try again,
+ * the VPU status will still be OFF.
+ *
+ * If we fail to get to READY, because the VPU did not send us the 'ready'
+ * message, the VPU state will go to ERROR.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_startup(struct device *dev, const char *firmware_name)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ if (!firmware_name)
+ return -EINVAL;
+
+ /* Free old vpu_dev->firmware_name value (if any). */
+ kfree(vpu_dev->firmware_name);
+
+ /* Set new value. */
+ vpu_dev->firmware_name = kstrdup(firmware_name, GFP_KERNEL);
+
+ return do_boot_sequence(vpu_dev);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_startup);
+
+/**
+ * intel_keembay_vpu_reset() - Reset the VPU
+ * @dev: The VPU device to reset.
+ *
+ * Resets the VPU. Only valid when the VPU is in the READY or ERROR state.
+ * The state of the VPU will become BUSY.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_reset(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ /*
+ * If vpu_dev->firmware_name == NULL then the VPU is not running
+ * (either it was never booted or vpu_stop() was called). So, calling
+ * reset is not allowed.
+ */
+ if (!vpu_dev->firmware_name)
+ return -EINVAL;
+
+ return do_boot_sequence(vpu_dev);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_reset);
+
+/**
+ * intel_keembay_vpu_stop() - Stop the VPU
+ * @dev: The VPU device to stop.
+ *
+ * Stops the VPU and restores to the OFF state. Only valid when the VPU is in
+ * the READY or ERROR state.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_stop(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int event_rc;
+ int rc;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_STOP);
+ if (rc < 0) {
+ dev_err(dev, "Can't stop in this state.\n");
+ return rc;
+ }
+
+ dev_info(dev, "Keem Bay VPU IPC stop.\n");
+
+ /* Request stop */
+ rc = request_vpu_stop(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to do request to stop - resetting state to OFF anyway.\n");
+ }
+
+ /* Remove any saved-off name */
+ kfree(vpu_dev->firmware_name);
+ vpu_dev->firmware_name = NULL;
+
+ event_rc = vpu_ipc_handle_event(vpu_dev,
+ KEEMBAY_VPU_EVENT_STOP_COMPLETE);
+ if (event_rc < 0) {
+ dev_err(dev,
+ "Failed to handle 'stop complete' event, probably fatal.\n");
+ return event_rc;
+ }
+
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_stop);
+
+/**
+ * intel_keembay_vpu_status() - Get the VPU state.
+ * @dev: The VPU device to retrieve the status for.
+ *
+ * Returns the state of the VPU as tracked by this driver.
+ *
+ * Return: Relevant value of enum intel_keembay_vpu_state
+ */
+enum intel_keembay_vpu_state intel_keembay_vpu_status(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return vpu_dev->state;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_status);
+
+/**
+ * intel_keembay_vpu_get_wdt_count() - Get the WDT count
+ * @dev: The VPU device to get the WDT count for.
+ * @id: ID of WDT events we wish to get
+ *
+ * Returns: Number of WDT timeout occurrences for given ID, or negative
+ * error value for invalid ID.
+ */
+int intel_keembay_vpu_get_wdt_count(struct device *dev,
+ enum intel_keembay_wdt_cpu_id id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int rc = -EINVAL;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ switch (id) {
+ case KEEMBAY_VPU_NCE:
+ rc = vpu_dev->nce_wdt_count;
+ break;
+ case KEEMBAY_VPU_MSS:
+ rc = vpu_dev->mss_wdt_count;
+ break;
+ default:
+ break;
+ }
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_get_wdt_count);
+
+/**
+ * intel_keembay_vpu_wait_for_ready() - Sleep until VPU is READY
+ * @dev: The VPU device for which we are waiting the ready message.
+ * @timeout: How long (in ms) the function will block waiting for the VPU
+ * to become ready.
+ *
+ * The caller may ask the VPU IPC driver to notify it when the VPU
+ * is READY. The driver performs no checks on the current state, so it
+ * is up to the caller to confirm that the state is correct before starting
+ * the wait.
+ *
+ * Returns: 0 on success negative error code otherwise
+ */
+int intel_keembay_vpu_wait_for_ready(struct device *dev, u32 timeout)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int rc;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ /*
+ * If we are in ERROR state, we will not get to READY
+ * state without some other transitions, so return
+ * error immediately for caller to handle.
+ */
+ if (vpu_dev->state == KEEMBAY_VPU_ERROR)
+ return -EIO;
+
+ rc = wait_event_interruptible_timeout(vpu_dev->ready_queue,
+ vpu_dev->state == KEEMBAY_VPU_READY,
+ msecs_to_jiffies(timeout));
+
+ /* Condition was false after timeout elapsed */
+ if (!rc)
+ rc = -ETIME;
+
+ /* Condition was true, so rc == 1 */
+ if (rc > 0)
+ rc = 0;
+
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_wait_for_ready);
+
+/**
+ * intel_keembay_vpu_register_for_events() - Register callback for event notification
+ * @dev: The VPU device.
+ * @callback: Callback function pointer
+ *
+ * Only a single callback can be registered at a time.
+ *
+ * Callback can be triggered from any context, so needs to be able to be run
+ * from IRQ context.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_register_for_events(struct device *dev,
+ void (*callback)(struct device *dev,
+ enum intel_keembay_vpu_event))
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ if (vpu_dev->callback)
+ return -EEXIST;
+
+ vpu_dev->callback = callback;
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_register_for_events);
+
+/**
+ * intel_keembay_vpu_unregister_for_events() - Unregister callback for event notification
+ * @dev: The VPU device.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_unregister_for_events(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ vpu_dev->callback = NULL;
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_unregister_for_events);
+
+/* Probe() function for the VPU IPC platform driver. */
+static int keembay_vpu_ipc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct vpu_ipc_dev *vpu_dev;
+ int rc;
+
+ vpu_dev = devm_kzalloc(dev, sizeof(*vpu_dev), GFP_KERNEL);
+ if (!vpu_dev)
+ return -ENOMEM;
+
+ vpu_dev->pdev = pdev;
+ vpu_dev->state = KEEMBAY_VPU_OFF;
+ vpu_dev->ready_message_task = NULL;
+ vpu_dev->firmware_name = NULL;
+ vpu_dev->nce_wdt_count = 0;
+ vpu_dev->mss_wdt_count = 0;
+ spin_lock_init(&vpu_dev->lock);
+ init_waitqueue_head(&vpu_dev->ready_queue);
+
+ /* Retrieve clocks */
+ rc = retrieve_clocks(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to retrieve clocks %d\n", rc);
+ return rc;
+ }
+
+ /* Retrieve memory regions, allocate memory */
+ rc = setup_reserved_memory(vpu_dev);
+ if (rc) {
+ dev_err(dev,
+ "Failed to set up reserved memory regions: %d\n", rc);
+ return rc;
+ }
+
+ /* Request watchdog timer resources */
+ rc = setup_watchdog_resources(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to setup watchdog resources %d\n", rc);
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Request the IMR number to be used */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-imr",
+ &vpu_dev->imr);
+ if (rc) {
+ dev_err(dev, "failed to get IMR number.\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Get VPU ID. */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-id",
+ &vpu_dev->vpu_id);
+ if (rc) {
+ /* Only warn for now; we will enforce this in the future. */
+ dev_err(dev, "VPU ID not defined in Device Tree\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Get IPC device to be used for IPC communication. */
+ rc = ipc_device_get(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to get IPC device\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Set platform data reference. */
+ platform_set_drvdata(pdev, vpu_dev);
+
+ return rc;
+
+probe_fail_post_resmem_setup:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+/* Remove() function for the VPU IPC platform driver. */
+static int keembay_vpu_ipc_remove(struct platform_device *pdev)
+{
+ struct vpu_ipc_dev *vpu_dev = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+
+ if (vpu_dev->ready_message_task) {
+ kthread_stop(vpu_dev->ready_message_task);
+ vpu_dev->ready_message_task = NULL;
+ }
+
+ of_reserved_mem_device_release(dev);
+
+ ipc_device_put(vpu_dev);
+
+ return 0;
+}
+
+/* Compatible string for the VPU/IPC driver. */
+static const struct of_device_id keembay_vpu_ipc_of_match[] = {
+ {
+ .compatible = "intel,keembay-vpu-ipc",
+ },
+ {}
+};
+
+/* The VPU IPC platform driver. */
+static struct platform_driver keem_bay_vpu_ipc_driver = {
+ .driver = {
+ .name = "keembay-vpu-ipc",
+ .of_match_table = keembay_vpu_ipc_of_match,
+ },
+ .probe = keembay_vpu_ipc_probe,
+ .remove = keembay_vpu_ipc_remove,
+};
+
+/* Helper function to get a vpu_dev struct from a generic device pointer. */
+static struct vpu_ipc_dev *to_vpu_dev(struct device *dev)
+{
+ struct platform_device *pdev;
+
+ if (!dev || dev->driver != &keem_bay_vpu_ipc_driver.driver)
+ return ERR_PTR(-EINVAL);
+ pdev = to_platform_device(dev);
+
+ return platform_get_drvdata(pdev);
+}
+
+/*
+ * Retrieve SoC information from the '/soc/version-info' device tree node and
+ * store it into 'vpu_ipc_soc_info' global variable.
+ */
+static int retrieve_dt_soc_information(void)
+{
+ struct device_node *soc_info_dn;
+ int ret;
+
+ soc_info_dn = of_find_node_by_path("/soc/version-info");
+ if (!soc_info_dn)
+ return -ENOENT;
+
+ ret = of_property_read_u64(soc_info_dn, "feature-exclusion",
+ &vpu_ipc_soc_info->feature_exclusion);
+ if (ret) {
+ pr_err("Property 'feature-exclusion' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u64(soc_info_dn, "device-id",
+ &vpu_ipc_soc_info->device_id);
+ if (ret) {
+ pr_err("Property 'device-id' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u32(soc_info_dn, "hardware-id",
+ &vpu_ipc_soc_info->hardware_id);
+ if (ret) {
+ pr_err("Property 'hardware-id' can't be read.\n");
+ return ret;
+ }
+ /*
+ * Note: the SKU and stepping information from the device tree is
+ * not a string, but an array of u8/chars. Therefore, we cannot
+ * parse it as a string.
+ */
+ ret = of_property_read_u8_array(soc_info_dn, "sku",
+ vpu_ipc_soc_info->sku,
+ sizeof(vpu_ipc_soc_info->sku));
+ if (ret) {
+ pr_err("Property 'sku' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u8_array(soc_info_dn, "stepping",
+ vpu_ipc_soc_info->stepping,
+ sizeof(vpu_ipc_soc_info->stepping));
+ if (ret) {
+ pr_err("Property 'stepping' can't be read.\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Init VPU IPC module:
+ * - Retrieve SoC information from device tree.
+ * - Register the VPU IPC platform driver.
+ */
+static int __init vpu_ipc_init(void)
+{
+ int rc;
+
+ vpu_ipc_soc_info = kzalloc(sizeof(*vpu_ipc_soc_info), GFP_KERNEL);
+ if (!vpu_ipc_soc_info)
+ return -ENOMEM;
+
+ rc = retrieve_dt_soc_information();
+ if (rc < 0)
+ pr_warn("VPU IPC failed to find SoC info, using defaults.\n");
+
+ rc = platform_driver_register(&keem_bay_vpu_ipc_driver);
+ if (rc < 0) {
+ pr_err("Failed to register platform driver for VPU IPC.\n");
+ goto cleanup_soc_info;
+ }
+
+ return 0;
+
+cleanup_soc_info:
+ kfree(vpu_ipc_soc_info);
+
+ return rc;
+}
+
+/*
+ * Remove VPU IPC module.
+ * - Un-register the VPU IPC platform driver.
+ * - Remove sysfs exposing SoC information.
+ * - Free allocated memory.
+ */
+static void __exit vpu_ipc_exit(void)
+{
+ platform_driver_unregister(&keem_bay_vpu_ipc_driver);
+ kfree(vpu_ipc_soc_info);
+}
+
+module_init(vpu_ipc_init);
+module_exit(vpu_ipc_exit);
+
+MODULE_DESCRIPTION("Keem Bay VPU IPC Driver");
+MODULE_AUTHOR("Paul Murphy <paul.j.murphy@intel.com>");
+MODULE_AUTHOR("Daniele Alessandrelli <daniele.alessandrelli@intel.com>");
+MODULE_LICENSE("GPL");
diff --git a/include/linux/soc/intel/keembay-vpu-ipc.h b/include/linux/soc/intel/keembay-vpu-ipc.h
new file mode 100644
index 000000000000..81d132186482
--- /dev/null
+++ b/include/linux/soc/intel/keembay-vpu-ipc.h
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Keem Bay VPU IPC Linux Kernel API
+ *
+ * Copyright (c) 2018-2020 Intel Corporation.
+ */
+
+#ifndef __KEEMBAY_VPU_IPC_H
+#define __KEEMBAY_VPU_IPC_H
+
+#include "linux/types.h"
+
+/* The possible node IDs. */
+enum {
+ KMB_VPU_IPC_NODE_ARM_CSS = 0,
+ KMB_VPU_IPC_NODE_LEON_MSS,
+};
+
+/* Possible states of VPU. */
+enum intel_keembay_vpu_state {
+ KEEMBAY_VPU_OFF = 0,
+ KEEMBAY_VPU_BUSY,
+ KEEMBAY_VPU_READY,
+ KEEMBAY_VPU_ERROR,
+ KEEMBAY_VPU_STOPPING
+};
+
+/* Possible CPU IDs for which we receive WDT timeout events. */
+enum intel_keembay_wdt_cpu_id {
+ KEEMBAY_VPU_MSS = 0,
+ KEEMBAY_VPU_NCE
+};
+
+/* Events that can be notified via callback, when registered. */
+enum intel_keembay_vpu_event {
+ KEEMBAY_VPU_NOTIFY_DISCONNECT = 0,
+ KEEMBAY_VPU_NOTIFY_CONNECT,
+ KEEMBAY_VPU_NOTIFY_MSS_WDT,
+ KEEMBAY_VPU_NOTIFY_NCE_WDT,
+};
+
+int intel_keembay_vpu_ipc_open_channel(struct device *dev, u8 node_id,
+ u16 chan_id);
+int intel_keembay_vpu_ipc_close_channel(struct device *dev, u8 node_id,
+ u16 chan_id);
+int intel_keembay_vpu_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 paddr, size_t size);
+int intel_keembay_vpu_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *paddr, size_t *size, u32 timeout);
+int intel_keembay_vpu_startup(struct device *dev, const char *firmware_name);
+int intel_keembay_vpu_reset(struct device *dev);
+int intel_keembay_vpu_stop(struct device *dev);
+enum intel_keembay_vpu_state intel_keembay_vpu_status(struct device *dev);
+int intel_keembay_vpu_get_wdt_count(struct device *dev,
+ enum intel_keembay_wdt_cpu_id id);
+int intel_keembay_vpu_wait_for_ready(struct device *dev, u32 timeout);
+int intel_keembay_vpu_register_for_events(struct device *dev,
+ void (*callback)(struct device *dev,
+ enum intel_keembay_vpu_event event));
+int intel_keembay_vpu_unregister_for_events(struct device *dev);
+
+#endif /* __KEEMBAY_VPU_IPC_H */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 06/22] misc: xlink-pcie: Add documentation for XLink PCIe driver
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (4 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 05/22] keembay-vpu-ipc: Add Keem Bay VPU IPC module mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host mgross
` (16 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Jonathan Corbet, linux-doc
From: Srikanth Thokala <srikanth.thokala@intel.com>
Provide overview of XLink PCIe driver implementation
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
Documentation/vpu/index.rst | 1 +
Documentation/vpu/xlink-pcie.rst | 91 ++++++++++++++++++++++++++++++++
2 files changed, 92 insertions(+)
create mode 100644 Documentation/vpu/xlink-pcie.rst
diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
index 7e290e048910..661cc700ee45 100644
--- a/Documentation/vpu/index.rst
+++ b/Documentation/vpu/index.rst
@@ -14,3 +14,4 @@ This documentation contains information for the Intel VPU stack.
:maxdepth: 2
vpu-stack-overview
+ xlink-pcie
diff --git a/Documentation/vpu/xlink-pcie.rst b/Documentation/vpu/xlink-pcie.rst
new file mode 100644
index 000000000000..bc64b566989d
--- /dev/null
+++ b/Documentation/vpu/xlink-pcie.rst
@@ -0,0 +1,91 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+Kernel driver: xlink-pcie driver
+================================
+Supported chips:
+ * Intel Edge.AI Computer Vision platforms: Keem Bay
+ Suffix: Bay
+ Slave address: 6240
+ Datasheet: Publicly available at Intel
+
+Author: Srikanth Thokala Srikanth.Thokala@intel.com
+
+-------------
+Introduction:
+-------------
+The xlink-pcie driver in linux-5.4 provides transport layer implementation for
+the data transfers to support xlink protocol subsystem communication with the
+peer device. i.e, between remote host system and the local Keem Bay device.
+
+The Keem Bay device is an ARM based SOC that includes a vision processing
+unit (VPU) and deep learning, neural network core in the hardware.
+The xlink-pcie driver exports a functional device endpoint to the Keem Bay device
+and supports two-way communication with peer device.
+
+------------------------
+High-level architecture:
+------------------------
+Remote Host: IA CPU
+Local Host: ARM CPU (Keem Bay)::
+
+ +------------------------------------------------------------------------+
+ | Remote Host IA CPU | | Local Host ARM CPU (Keem Bay) | |
+ +==================================+=+===============================+===+
+ | User App | | User App | |
+ +----------------------------------+-+-------------------------------+---+
+ | XLink UAPI | | XLink UAPI | |
+ +----------------------------------+-+-------------------------------+---+
+ | XLink Core | | XLink Core | |
+ +----------------------------------+-+-------------------------------+---+
+ | XLink PCIe | | XLink PCIe | |
+ +----------------------------------+-+-------------------------------+---+
+ | XLink-PCIe Remote Host driver | | XLink-PCIe Local Host driver | |
+ +----------------------------------+-+-------------------------------+---+
+ |-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:|:|:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:|
+ +----------------------------------+-+-------------------------------+---+
+ | PCIe Host Controller | | PCIe Device Controller | HW|
+ +----------------------------------+-+-------------------------------+---+
+ ^ ^
+ | |
+ |------------- PCIe x2 Link -----------------|
+
+This XLink PCIe driver comprises of two variants:
+* Local Host driver
+
+ * Intended for ARM CPU
+ * It is based on PCI Endpoint Framework
+ * Driver path: {tree}/drivers/misc/xlink-pcie/local_host
+
+* Remote Host driver
+
+ * Intended for IA CPU
+ * It is a PCIe endpoint driver
+ * Driver path: {tree}/drivers/misc/xlink-pcie/remote_host
+
+XLink PCIe communication between local host and remote host is achieved through
+ring buffer management and MSI/Doorbell interrupts.
+
+The xlink-pcie driver subsystem registers Keem Bay device as an endpoint driver
+and provides standard linux pcie sysfs interface, # /sys/bus/pci/devices/xxxx:xx:xx.0/
+
+
+-------------------------
+XLink protocol subsystem:
+-------------------------
+xlink is an abstracted control and communication subsystem based on channel
+identification. It is intended to support VPU technology both at SoC level as
+well as at IP level, over multiple interfaces.
+
+- The xlink subsystem abstracts several types of communication channels
+ underneath, allowing the usage of different interfaces with the
+ same function call interface.
+- The Communication channels are full-duplex protocol channels allowing
+ concurrent bidirectional communication.
+- The xlink subsystem also supports control operations to VPU either
+ from standalone local system or from remote system based on communication
+ interface underneath.
+- The xlink subsystem supports following communication interfaces:
+ * USB CDC
+ * Gigabit Ethernet
+ * PCIe
+ * IPC
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (5 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 06/22] misc: xlink-pcie: Add documentation for XLink PCIe driver mgross
@ 2020-11-30 23:06 ` mgross
2020-12-01 10:13 ` Greg Kroah-Hartman
2020-12-01 10:17 ` Greg Kroah-Hartman
2020-11-30 23:06 ` [PATCH 08/22] misc: xlink-pcie: lh: Add PCIe EP DMA functionality mgross
` (15 subsequent siblings)
22 siblings, 2 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Derek Kiernan, Dragan Cvetic, Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add PCIe EPF driver for local host (lh) to configure BAR's and other
HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
Cc: Derek Kiernan <derek.kiernan@xilinx.com>
Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
MAINTAINERS | 6 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/xlink-pcie/Kconfig | 9 +
drivers/misc/xlink-pcie/Makefile | 1 +
drivers/misc/xlink-pcie/local_host/Makefile | 2 +
drivers/misc/xlink-pcie/local_host/epf.c | 414 ++++++++++++++++++++
drivers/misc/xlink-pcie/local_host/epf.h | 39 ++
drivers/misc/xlink-pcie/local_host/xpcie.h | 54 +++
9 files changed, 527 insertions(+)
create mode 100644 drivers/misc/xlink-pcie/Kconfig
create mode 100644 drivers/misc/xlink-pcie/Makefile
create mode 100644 drivers/misc/xlink-pcie/local_host/Makefile
create mode 100644 drivers/misc/xlink-pcie/local_host/epf.c
create mode 100644 drivers/misc/xlink-pcie/local_host/epf.h
create mode 100644 drivers/misc/xlink-pcie/local_host/xpcie.h
diff --git a/MAINTAINERS b/MAINTAINERS
index f95cef927d19..9af8685967ac 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1952,6 +1952,12 @@ F: Documentation/devicetree/bindings/arm/intel,keembay.yaml
F: arch/arm64/boot/dts/intel/keembay-evm.dts
F: arch/arm64/boot/dts/intel/keembay-soc.dtsi
+ARM KEEMBAY XLINK PCIE SUPPORT
+M: Srikanth Thokala <srikanth.thokala@intel.com>
+M: Mark Gross <mgross@linux.intel.com>
+S: Maintained
+F: drivers/misc/xlink-pcie/
+
ARM/INTEL RESEARCH IMOTE/STARGATE 2 MACHINE SUPPORT
M: Jonathan Cameron <jic23@cam.ac.uk>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index fafa8b0d8099..dfb98e444c6e 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -481,4 +481,5 @@ source "drivers/misc/ocxl/Kconfig"
source "drivers/misc/cardreader/Kconfig"
source "drivers/misc/habanalabs/Kconfig"
source "drivers/misc/uacce/Kconfig"
+source "drivers/misc/xlink-pcie/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index d23231e73330..d17621fc43d5 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -57,3 +57,4 @@ obj-$(CONFIG_HABANA_AI) += habanalabs/
obj-$(CONFIG_UACCE) += uacce/
obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
+obj-y += xlink-pcie/
diff --git a/drivers/misc/xlink-pcie/Kconfig b/drivers/misc/xlink-pcie/Kconfig
new file mode 100644
index 000000000000..46aa401d79b7
--- /dev/null
+++ b/drivers/misc/xlink-pcie/Kconfig
@@ -0,0 +1,9 @@
+config XLINK_PCIE_LH_DRIVER
+ tristate "XLink PCIe Local Host driver"
+ depends on PCI_ENDPOINT && ARCH_KEEMBAY
+ help
+ This option enables XLink PCIe Local Host driver.
+
+ Choose M here to compile this driver as a module, name is mxlk_ep.
+ This driver is used for XLink communication over PCIe and is to be
+ loaded on the Intel Keem Bay platform.
diff --git a/drivers/misc/xlink-pcie/Makefile b/drivers/misc/xlink-pcie/Makefile
new file mode 100644
index 000000000000..d693d382e9c6
--- /dev/null
+++ b/drivers/misc/xlink-pcie/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += local_host/
diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
new file mode 100644
index 000000000000..514d3f0c91bc
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
+mxlk_ep-objs := epf.o
diff --git a/drivers/misc/xlink-pcie/local_host/epf.c b/drivers/misc/xlink-pcie/local_host/epf.c
new file mode 100644
index 000000000000..ae07b364734d
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/epf.c
@@ -0,0 +1,414 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include <linux/of.h>
+#include <linux/platform_device.h>
+
+#include "epf.h"
+
+#define BAR2_MIN_SIZE SZ_16K
+#define BAR4_MIN_SIZE SZ_16K
+
+#define PCIE_REGS_PCIE_INTR_ENABLE 0x18
+#define PCIE_REGS_PCIE_INTR_FLAGS 0x1C
+#define LBC_CII_EVENT_FLAG BIT(18)
+#define PCIE_REGS_PCIE_ERR_INTR_FLAGS 0x24
+#define LINK_REQ_RST_FLG BIT(15)
+
+static struct pci_epf_header xpcie_header = {
+ .vendorid = PCI_VENDOR_ID_INTEL,
+ .deviceid = PCI_DEVICE_ID_INTEL_KEEMBAY,
+ .baseclass_code = PCI_BASE_CLASS_MULTIMEDIA,
+ .subclass_code = 0x0,
+ .subsys_vendor_id = 0x0,
+ .subsys_id = 0x0,
+};
+
+static const struct pci_epf_device_id xpcie_epf_ids[] = {
+ {
+ .name = "mxlk_pcie_epf",
+ },
+ {},
+};
+
+static irqreturn_t intel_xpcie_err_interrupt(int irq, void *args)
+{
+ struct xpcie_epf *xpcie_epf;
+ struct xpcie *xpcie = args;
+ u32 val;
+
+ xpcie_epf = container_of(xpcie, struct xpcie_epf, xpcie);
+ val = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_ERR_INTR_FLAGS);
+
+ iowrite32(val, xpcie_epf->apb_base + PCIE_REGS_PCIE_ERR_INTR_FLAGS);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t intel_xpcie_host_interrupt(int irq, void *args)
+{
+ struct xpcie_epf *xpcie_epf;
+ struct xpcie *xpcie = args;
+ u32 val;
+
+ xpcie_epf = container_of(xpcie, struct xpcie_epf, xpcie);
+ val = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_INTR_FLAGS);
+ if (val & LBC_CII_EVENT_FLAG) {
+ iowrite32(LBC_CII_EVENT_FLAG,
+ xpcie_epf->apb_base + PCIE_REGS_PCIE_INTR_FLAGS);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int intel_xpcie_check_bar(struct pci_epf *epf,
+ struct pci_epf_bar *epf_bar,
+ enum pci_barno barno,
+ size_t size, u8 reserved_bar)
+{
+ if (reserved_bar & (1 << barno)) {
+ dev_err(&epf->dev, "BAR%d is already reserved\n", barno);
+ return -EFAULT;
+ }
+
+ if (epf_bar->size != 0 && epf_bar->size < size) {
+ dev_err(&epf->dev, "BAR%d fixed size is not enough\n", barno);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int intel_xpcie_configure_bar(struct pci_epf *epf,
+ const struct pci_epc_features
+ *epc_features)
+{
+ struct pci_epf_bar *epf_bar;
+ bool bar_fixed_64bit;
+ int ret, i;
+
+ for (i = BAR_0; i <= BAR_5; i++) {
+ epf_bar = &epf->bar[i];
+ bar_fixed_64bit = !!(epc_features->bar_fixed_64bit & (1 << i));
+ if (bar_fixed_64bit)
+ epf_bar->flags |= PCI_BASE_ADDRESS_MEM_TYPE_64;
+ if (epc_features->bar_fixed_size[i])
+ epf_bar->size = epc_features->bar_fixed_size[i];
+
+ if (i == BAR_2) {
+ ret = intel_xpcie_check_bar(epf, epf_bar, BAR_2,
+ BAR2_MIN_SIZE,
+ epc_features->reserved_bar);
+ if (ret)
+ return ret;
+ }
+
+ if (i == BAR_4) {
+ ret = intel_xpcie_check_bar(epf, epf_bar, BAR_4,
+ BAR4_MIN_SIZE,
+ epc_features->reserved_bar);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void intel_xpcie_cleanup_bar(struct pci_epf *epf, enum pci_barno barno)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ struct pci_epc *epc = epf->epc;
+
+ if (xpcie_epf->vaddr[barno]) {
+ pci_epc_clear_bar(epc, epf->func_no, &epf->bar[barno]);
+ pci_epf_free_space(epf, xpcie_epf->vaddr[barno], barno);
+ xpcie_epf->vaddr[barno] = NULL;
+ }
+}
+
+static void intel_xpcie_cleanup_bars(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+
+ intel_xpcie_cleanup_bar(epf, BAR_2);
+ intel_xpcie_cleanup_bar(epf, BAR_4);
+ xpcie_epf->xpcie.mmio = NULL;
+ xpcie_epf->xpcie.bar4 = NULL;
+}
+
+static int intel_xpcie_setup_bar(struct pci_epf *epf, enum pci_barno barno,
+ size_t min_size, size_t align)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ struct pci_epf_bar *bar = &epf->bar[barno];
+ struct pci_epc *epc = epf->epc;
+ void *vaddr;
+ int ret;
+
+ bar->flags |= PCI_BASE_ADDRESS_MEM_TYPE_64;
+ if (!bar->size)
+ bar->size = min_size;
+
+ if (barno == BAR_4)
+ bar->flags |= PCI_BASE_ADDRESS_MEM_PREFETCH;
+
+ vaddr = pci_epf_alloc_space(epf, bar->size, barno, align);
+ if (!vaddr) {
+ dev_err(&epf->dev, "Failed to map BAR%d\n", barno);
+ return -ENOMEM;
+ }
+
+ ret = pci_epc_set_bar(epc, epf->func_no, bar);
+ if (ret) {
+ pci_epf_free_space(epf, vaddr, barno);
+ dev_err(&epf->dev, "Failed to set BAR%d\n", barno);
+ return ret;
+ }
+
+ xpcie_epf->vaddr[barno] = vaddr;
+
+ return 0;
+}
+
+static int intel_xpcie_setup_bars(struct pci_epf *epf, size_t align)
+{
+ int ret;
+
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+
+ ret = intel_xpcie_setup_bar(epf, BAR_2, BAR2_MIN_SIZE, align);
+ if (ret)
+ return ret;
+
+ ret = intel_xpcie_setup_bar(epf, BAR_4, BAR4_MIN_SIZE, align);
+ if (ret) {
+ intel_xpcie_cleanup_bar(epf, BAR_2);
+ return ret;
+ }
+
+ xpcie_epf->comm_bar = BAR_2;
+ xpcie_epf->xpcie.mmio = (void *)xpcie_epf->vaddr[BAR_2] +
+ XPCIE_MMIO_OFFSET;
+
+ xpcie_epf->bar4 = BAR_4;
+ xpcie_epf->xpcie.bar4 = xpcie_epf->vaddr[BAR_4];
+
+ return 0;
+}
+
+static int intel_xpcie_epf_get_platform_data(struct device *dev,
+ struct xpcie_epf *xpcie_epf)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct device_node *soc_node, *version_node;
+ struct resource *res;
+ const char *prop;
+ int prop_size;
+
+ xpcie_epf->irq_dma = platform_get_irq_byname(pdev, "intr");
+ if (xpcie_epf->irq_dma < 0) {
+ dev_err(&xpcie_epf->epf->dev, "failed to get IRQ: %d\n",
+ xpcie_epf->irq_dma);
+ return -EINVAL;
+ }
+
+ xpcie_epf->irq_err = platform_get_irq_byname(pdev, "err_intr");
+ if (xpcie_epf->irq_err < 0) {
+ dev_err(&xpcie_epf->epf->dev, "failed to get erroe IRQ: %d\n",
+ xpcie_epf->irq_err);
+ return -EINVAL;
+ }
+
+ xpcie_epf->irq = platform_get_irq_byname(pdev, "ev_intr");
+ if (xpcie_epf->irq < 0) {
+ dev_err(&xpcie_epf->epf->dev, "failed to get event IRQ: %d\n",
+ xpcie_epf->irq);
+ return -EINVAL;
+ }
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb");
+ xpcie_epf->apb_base =
+ devm_ioremap(dev, res->start, resource_size(res));
+ if (IS_ERR(xpcie_epf->apb_base))
+ return PTR_ERR(xpcie_epf->apb_base);
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+ xpcie_epf->dbi_base =
+ devm_ioremap(dev, res->start, resource_size(res));
+ if (IS_ERR(xpcie_epf->dbi_base))
+ return PTR_ERR(xpcie_epf->dbi_base);
+
+ memcpy(xpcie_epf->stepping, "B0", 2);
+ soc_node = of_get_parent(pdev->dev.of_node);
+ if (soc_node) {
+ version_node = of_get_child_by_name(soc_node, "version-info");
+ if (version_node) {
+ prop = of_get_property(version_node, "stepping",
+ &prop_size);
+ if (prop && prop_size <= KEEMBAY_XPCIE_STEPPING_MAXLEN)
+ memcpy(xpcie_epf->stepping, prop, prop_size);
+ of_node_put(version_node);
+ }
+ of_node_put(soc_node);
+ }
+
+ return 0;
+}
+
+static int intel_xpcie_epf_bind(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ const struct pci_epc_features *features;
+ struct pci_epc *epc = epf->epc;
+ struct device *dev;
+ size_t align = SZ_16K;
+ int ret;
+
+ if (WARN_ON_ONCE(!epc))
+ return -EINVAL;
+
+ dev = epc->dev.parent;
+ features = pci_epc_get_features(epc, epf->func_no);
+ xpcie_epf->epc_features = features;
+ if (features) {
+ align = features->align;
+ ret = intel_xpcie_configure_bar(epf, features);
+ if (ret)
+ return ret;
+ }
+
+ ret = intel_xpcie_setup_bars(epf, align);
+ if (ret) {
+ dev_err(&epf->dev, "BAR initialization failed\n");
+ return ret;
+ }
+
+ ret = intel_xpcie_epf_get_platform_data(dev, xpcie_epf);
+ if (ret) {
+ dev_err(&epf->dev, "Unable to get platform data\n");
+ return -EINVAL;
+ }
+
+ if (!strcmp(xpcie_epf->stepping, "A0")) {
+ xpcie_epf->xpcie.legacy_a0 = true;
+ iowrite32(1, (void __iomem *)xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
+ } else {
+ xpcie_epf->xpcie.legacy_a0 = false;
+ iowrite32(0, (void __iomem *)xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
+ }
+
+ /* Enable interrupt */
+ writel(LBC_CII_EVENT_FLAG,
+ xpcie_epf->apb_base + PCIE_REGS_PCIE_INTR_ENABLE);
+ ret = devm_request_irq(&epf->dev, xpcie_epf->irq,
+ &intel_xpcie_host_interrupt, 0,
+ XPCIE_DRIVER_NAME, &xpcie_epf->xpcie);
+ if (ret) {
+ dev_err(&epf->dev, "failed to request irq\n");
+ goto err_cleanup_bars;
+ }
+
+ ret = devm_request_irq(&epf->dev, xpcie_epf->irq_err,
+ &intel_xpcie_err_interrupt, 0,
+ XPCIE_DRIVER_NAME, &xpcie_epf->xpcie);
+ if (ret) {
+ dev_err(&epf->dev, "failed to request error irq\n");
+ goto err_cleanup_bars;
+ }
+
+ return 0;
+
+err_cleanup_bars:
+ intel_xpcie_cleanup_bars(epf);
+
+ return ret;
+}
+
+static void intel_xpcie_epf_unbind(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ struct pci_epc *epc = epf->epc;
+
+ free_irq(xpcie_epf->irq, &xpcie_epf->xpcie);
+ free_irq(xpcie_epf->irq_err, &xpcie_epf->xpcie);
+
+ pci_epc_stop(epc);
+
+ intel_xpcie_cleanup_bars(epf);
+}
+
+static int intel_xpcie_epf_probe(struct pci_epf *epf)
+{
+ struct device *dev = &epf->dev;
+ struct xpcie_epf *xpcie_epf;
+
+ xpcie_epf = devm_kzalloc(dev, sizeof(*xpcie_epf), GFP_KERNEL);
+ if (!xpcie_epf)
+ return -ENOMEM;
+
+ epf->header = &xpcie_header;
+ xpcie_epf->epf = epf;
+ epf_set_drvdata(epf, xpcie_epf);
+
+ return 0;
+}
+
+static void intel_xpcie_epf_shutdown(struct device *dev)
+{
+ struct pci_epf *epf = to_pci_epf(dev);
+ struct xpcie_epf *xpcie_epf;
+
+ xpcie_epf = epf_get_drvdata(epf);
+
+ /* Notify host in case PCIe hot plug not supported */
+ if (xpcie_epf)
+ pci_epc_raise_irq(epf->epc, epf->func_no, PCI_EPC_IRQ_MSI, 1);
+}
+
+static struct pci_epf_ops ops = {
+ .bind = intel_xpcie_epf_bind,
+ .unbind = intel_xpcie_epf_unbind,
+};
+
+static struct pci_epf_driver xpcie_epf_driver = {
+ .driver.name = "mxlk_pcie_epf",
+ .driver.shutdown = intel_xpcie_epf_shutdown,
+ .probe = intel_xpcie_epf_probe,
+ .id_table = xpcie_epf_ids,
+ .ops = &ops,
+ .owner = THIS_MODULE,
+};
+
+static int __init intel_xpcie_epf_init(void)
+{
+ int ret;
+
+ ret = pci_epf_register_driver(&xpcie_epf_driver);
+ if (ret) {
+ pr_err("Failed to register xlink pcie epf driver: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+module_init(intel_xpcie_epf_init);
+
+static void __exit intel_xpcie_epf_exit(void)
+{
+ pci_epf_unregister_driver(&xpcie_epf_driver);
+}
+module_exit(intel_xpcie_epf_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION(XPCIE_DRIVER_DESC);
+MODULE_VERSION(XPCIE_DRIVER_VERSION);
diff --git a/drivers/misc/xlink-pcie/local_host/epf.h b/drivers/misc/xlink-pcie/local_host/epf.h
new file mode 100644
index 000000000000..2b38c87b3701
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/epf.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_EPF_HEADER_
+#define XPCIE_EPF_HEADER_
+
+#include <linux/pci-epc.h>
+#include <linux/pci-epf.h>
+
+#include "xpcie.h"
+
+#define XPCIE_DRIVER_NAME "mxlk_pcie_epf"
+#define XPCIE_DRIVER_DESC "Intel(R) xLink PCIe endpoint function driver"
+
+#define KEEMBAY_XPCIE_STEPPING_MAXLEN 8
+
+struct xpcie_epf {
+ struct pci_epf *epf;
+ void *vaddr[BAR_5 + 1];
+ enum pci_barno comm_bar;
+ enum pci_barno bar4;
+ const struct pci_epc_features *epc_features;
+ struct xpcie xpcie;
+ int irq;
+ int irq_dma;
+ int irq_err;
+ void __iomem *apb_base;
+ void __iomem *dma_base;
+ void __iomem *dbi_base;
+ char stepping[KEEMBAY_XPCIE_STEPPING_MAXLEN];
+};
+
+#endif /* XPCIE_EPF_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/xpcie.h b/drivers/misc/xlink-pcie/local_host/xpcie.h
new file mode 100644
index 000000000000..932ec5d5c718
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/xpcie.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_HEADER_
+#define XPCIE_HEADER_
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci_ids.h>
+
+#ifndef PCI_DEVICE_ID_INTEL_KEEMBAY
+#define PCI_DEVICE_ID_INTEL_KEEMBAY 0x6240
+#endif
+
+#define XPCIE_IO_COMM_SIZE SZ_16K
+#define XPCIE_MMIO_OFFSET SZ_4K
+
+#define XPCIE_VERSION_MAJOR 0
+#define XPCIE_VERSION_MINOR 5
+#define XPCIE_VERSION_BUILD 0
+#define _TOSTR(X) #X
+#define _VERSION(A, B, C) _TOSTR(A) "." _TOSTR(B) "." _TOSTR(C)
+#define XPCIE_DRIVER_VERSION \
+ _VERSION(XPCIE_VERSION_MAJOR, XPCIE_VERSION_MINOR, XPCIE_VERSION_BUILD)
+
+struct xpcie_version {
+ u8 major;
+ u8 minor;
+ u16 build;
+} __packed;
+
+/* MMIO layout and offsets shared between device and host */
+struct xpcie_mmio {
+ struct xpcie_version version;
+ u8 legacy_a0;
+} __packed;
+
+#define XPCIE_MMIO_VERSION (offsetof(struct xpcie_mmio, version))
+#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+
+struct xpcie {
+ u32 status;
+ bool legacy_a0;
+ void *mmio;
+ void *bar4;
+};
+
+#endif /* XPCIE_HEADER_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-11-30 23:06 ` [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host mgross
@ 2020-12-01 10:13 ` Greg Kroah-Hartman
2020-12-01 17:45 ` mark gross
2020-12-01 10:17 ` Greg Kroah-Hartman
1 sibling, 1 reply; 34+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-01 10:13 UTC (permalink / raw)
To: mgross
Cc: linux-kernel, markgross, adam.r.gretzinger, Srikanth Thokala,
Derek Kiernan, Dragan Cvetic, Arnd Bergmann
On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> From: Srikanth Thokala <srikanth.thokala@intel.com>
>
> Add PCIe EPF driver for local host (lh) to configure BAR's and other
> HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
>
> Cc: Derek Kiernan <derek.kiernan@xilinx.com>
> Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Reviewed-by: Mark Gross <mgross@linux.intel.com>
> Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
<snip>
Again, you sent this twice? And it never got to lore.kernel.org nor the
mailing lists?
And I can't see a 00/XX email anywhere explaining this, and I didn't get
the whole series?
Something is really wrong on your end with your email client
configuration, please fix that up and send this so that we can actually
see it all, and know what the heck we are supposed to be reviewing...
thanks,
greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-12-01 10:13 ` Greg Kroah-Hartman
@ 2020-12-01 17:45 ` mark gross
2020-12-01 17:54 ` Greg Kroah-Hartman
2020-12-01 18:29 ` Dragan Cvetic
0 siblings, 2 replies; 34+ messages in thread
From: mark gross @ 2020-12-01 17:45 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: mgross, linux-kernel, markgross, adam.r.gretzinger,
Srikanth Thokala, Derek Kiernan, Dragan Cvetic, Arnd Bergmann
On Tue, Dec 01, 2020 at 11:13:05AM +0100, Greg Kroah-Hartman wrote:
> On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> > From: Srikanth Thokala <srikanth.thokala@intel.com>
> >
> > Add PCIe EPF driver for local host (lh) to configure BAR's and other
> > HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
> >
> > Cc: Derek Kiernan <derek.kiernan@xilinx.com>
> > Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
> > Cc: Arnd Bergmann <arnd@arndb.de>
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Reviewed-by: Mark Gross <mgross@linux.intel.com>
> > Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
>
> <snip>
>
> Again, you sent this twice? And it never got to lore.kernel.org nor the
> mailing lists?
In my excitement of sorting out my MTA misconfiguration work around I actually
hit entry without including the magic fix
"--envelope-sender=mgross@linux.intel.com" in my git send-email command line.
>
> And I can't see a 00/XX email anywhere explaining this, and I didn't get
> the whole series?
https://lore.kernel.org/lkml/20201130230707.46351-1-mgross@linux.intel.com/
Did I mess up on who all should get the 00/xx email?
>
> Something is really wrong on your end with your email client
> configuration, please fix that up and send this so that we can actually
> see it all, and know what the heck we are supposed to be reviewing...
Yes, I think I have it fixed, or worked around now.
sorry,
--mark
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-12-01 17:45 ` mark gross
@ 2020-12-01 17:54 ` Greg Kroah-Hartman
2020-12-01 22:00 ` mark gross
2020-12-01 18:29 ` Dragan Cvetic
1 sibling, 1 reply; 34+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-01 17:54 UTC (permalink / raw)
To: mark gross
Cc: linux-kernel, markgross, adam.r.gretzinger, Srikanth Thokala,
Derek Kiernan, Dragan Cvetic, Arnd Bergmann
On Tue, Dec 01, 2020 at 09:45:43AM -0800, mark gross wrote:
> On Tue, Dec 01, 2020 at 11:13:05AM +0100, Greg Kroah-Hartman wrote:
> > On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> > > From: Srikanth Thokala <srikanth.thokala@intel.com>
> > >
> > > Add PCIe EPF driver for local host (lh) to configure BAR's and other
> > > HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
> > >
> > > Cc: Derek Kiernan <derek.kiernan@xilinx.com>
> > > Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
> > > Cc: Arnd Bergmann <arnd@arndb.de>
> > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > Reviewed-by: Mark Gross <mgross@linux.intel.com>
> > > Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
> >
> > <snip>
> >
> > Again, you sent this twice? And it never got to lore.kernel.org nor the
> > mailing lists?
> In my excitement of sorting out my MTA misconfiguration work around I actually
> hit entry without including the magic fix
> "--envelope-sender=mgross@linux.intel.com" in my git send-email command line.
> >
> > And I can't see a 00/XX email anywhere explaining this, and I didn't get
> > the whole series?
> https://lore.kernel.org/lkml/20201130230707.46351-1-mgross@linux.intel.com/
>
> Did I mess up on who all should get the 00/xx email?
Yes, you need to include the people that you want to review the
individual patches, otherwise we have no idea :(
thanks,
greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-12-01 17:54 ` Greg Kroah-Hartman
@ 2020-12-01 22:00 ` mark gross
0 siblings, 0 replies; 34+ messages in thread
From: mark gross @ 2020-12-01 22:00 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: mark gross, linux-kernel, markgross, adam.r.gretzinger,
Srikanth Thokala, Derek Kiernan, Dragan Cvetic, Arnd Bergmann
On Tue, Dec 01, 2020 at 06:54:28PM +0100, Greg Kroah-Hartman wrote:
> On Tue, Dec 01, 2020 at 09:45:43AM -0800, mark gross wrote:
> > On Tue, Dec 01, 2020 at 11:13:05AM +0100, Greg Kroah-Hartman wrote:
> > > On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> > > > From: Srikanth Thokala <srikanth.thokala@intel.com>
> > > >
> > > > Add PCIe EPF driver for local host (lh) to configure BAR's and other
> > > > HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
> > > >
> > > > Cc: Derek Kiernan <derek.kiernan@xilinx.com>
> > > > Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
> > > > Cc: Arnd Bergmann <arnd@arndb.de>
> > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > > Reviewed-by: Mark Gross <mgross@linux.intel.com>
> > > > Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
> > >
> > > <snip>
> > >
> > > Again, you sent this twice? And it never got to lore.kernel.org nor the
> > > mailing lists?
> > In my excitement of sorting out my MTA misconfiguration work around I actually
> > hit entry without including the magic fix
> > "--envelope-sender=mgross@linux.intel.com" in my git send-email command line.
> > >
> > > And I can't see a 00/XX email anywhere explaining this, and I didn't get
> > > the whole series?
> > https://lore.kernel.org/lkml/20201130230707.46351-1-mgross@linux.intel.com/
> >
> > Did I mess up on who all should get the 00/xx email?
>
> Yes, you need to include the people that you want to review the
> individual patches, otherwise we have no idea :(
>
Ok, I will reset and try again. I appoligize for this false start.
I'll resend after I trim the CC list based on Dragon's feedback.
--mark
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-12-01 17:45 ` mark gross
2020-12-01 17:54 ` Greg Kroah-Hartman
@ 2020-12-01 18:29 ` Dragan Cvetic
1 sibling, 0 replies; 34+ messages in thread
From: Dragan Cvetic @ 2020-12-01 18:29 UTC (permalink / raw)
To: mgross, Greg Kroah-Hartman
Cc: linux-kernel, markgross, adam.r.gretzinger, Srikanth Thokala,
Derek Kiernan, Arnd Bergmann
Hi Mark,
You should remove me and Derek Kiernan from CC list, we don't belong to this list.
Dragan
> -----Original Message-----
> From: mark gross <mgross@linux.intel.com>
> Sent: Tuesday 1 December 2020 17:46
> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: mgross@linux.intel.com; linux-kernel@vger.kernel.org; markgross@kernel.org; adam.r.gretzinger@intel.com; Srikanth Thokala
> <srikanth.thokala@intel.com>; Derek Kiernan <dkiernan@xilinx.com>; Dragan Cvetic <draganc@xilinx.com>; Arnd Bergmann
> <arnd@arndb.de>
> Subject: Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
>
> On Tue, Dec 01, 2020 at 11:13:05AM +0100, Greg Kroah-Hartman wrote:
> > On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> > > From: Srikanth Thokala <srikanth.thokala@intel.com>
> > >
> > > Add PCIe EPF driver for local host (lh) to configure BAR's and other
> > > HW resources. Underlying PCIe HW controller is a Synopsys DWC PCIe core.
> > >
> > > Cc: Derek Kiernan <derek.kiernan@xilinx.com>
> > > Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
> > > Cc: Arnd Bergmann <arnd@arndb.de>
> > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > Reviewed-by: Mark Gross <mgross@linux.intel.com>
> > > Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
> >
> > <snip>
> >
> > Again, you sent this twice? And it never got to lore.kernel.org nor the
> > mailing lists?
> In my excitement of sorting out my MTA misconfiguration work around I actually
> hit entry without including the magic fix
> "--envelope-sender=mgross@linux.intel.com" in my git send-email command line.
> >
> > And I can't see a 00/XX email anywhere explaining this, and I didn't get
> > the whole series?
> https://lore.kernel.org/lkml/20201130230707.46351-1-mgross@linux.intel.com/
>
> Did I mess up on who all should get the 00/xx email?
>
> >
> > Something is really wrong on your end with your email client
> > configuration, please fix that up and send this so that we can actually
> > see it all, and know what the heck we are supposed to be reviewing...
>
> Yes, I think I have it fixed, or worked around now.
>
> sorry,
>
> --mark
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-11-30 23:06 ` [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host mgross
2020-12-01 10:13 ` Greg Kroah-Hartman
@ 2020-12-01 10:17 ` Greg Kroah-Hartman
2020-12-02 16:48 ` Thokala, Srikanth
1 sibling, 1 reply; 34+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-01 10:17 UTC (permalink / raw)
To: mgross
Cc: linux-kernel, markgross, adam.r.gretzinger, Srikanth Thokala,
Derek Kiernan, Dragan Cvetic, Arnd Bergmann
On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> +#define XPCIE_VERSION_MAJOR 0
> +#define XPCIE_VERSION_MINOR 5
> +#define XPCIE_VERSION_BUILD 0
> +#define _TOSTR(X) #X
> +#define _VERSION(A, B, C) _TOSTR(A) "." _TOSTR(B) "." _TOSTR(C)
> +#define XPCIE_DRIVER_VERSION \
> + _VERSION(XPCIE_VERSION_MAJOR, XPCIE_VERSION_MINOR, XPCIE_VERSION_BUILD)
> +
> +struct xpcie_version {
> + u8 major;
> + u8 minor;
> + u16 build;
> +} __packed;
Now that the driver will be in the tree, there is no need for this
structure or these fields, right? It will just be the version of the
kernel. Otherwise something like this will never be correct, given
stable kernel releases and enterprise kernel releases, and loads of
other messy backported trees. Just stick to the kernel release number
please, that's the only sane thing.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread* RE: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
2020-12-01 10:17 ` Greg Kroah-Hartman
@ 2020-12-02 16:48 ` Thokala, Srikanth
0 siblings, 0 replies; 34+ messages in thread
From: Thokala, Srikanth @ 2020-12-02 16:48 UTC (permalink / raw)
To: Greg Kroah-Hartman, mgross
Cc: linux-kernel, markgross, Gretzinger, Adam R, Derek Kiernan,
Dragan Cvetic, Arnd Bergmann
Hi Greg,
> -----Original Message-----
> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Sent: Tuesday, December 1, 2020 3:48 PM
> To: mgross@linux.intel.com
> Cc: linux-kernel@vger.kernel.org; markgross@kernel.org; Gretzinger, Adam R
> <adam.r.gretzinger@intel.com>; Thokala, Srikanth
> <srikanth.thokala@intel.com>; Derek Kiernan <derek.kiernan@xilinx.com>;
> Dragan Cvetic <dragan.cvetic@xilinx.com>; Arnd Bergmann <arnd@arndb.de>
> Subject: Re: [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for
> Local Host
>
> On Mon, Nov 30, 2020 at 03:06:52PM -0800, mgross@linux.intel.com wrote:
> > +#define XPCIE_VERSION_MAJOR 0
> > +#define XPCIE_VERSION_MINOR 5
> > +#define XPCIE_VERSION_BUILD 0
> > +#define _TOSTR(X) #X
> > +#define _VERSION(A, B, C) _TOSTR(A) "." _TOSTR(B) "." _TOSTR(C)
> > +#define XPCIE_DRIVER_VERSION \
> > + _VERSION(XPCIE_VERSION_MAJOR, XPCIE_VERSION_MINOR,
> XPCIE_VERSION_BUILD)
> > +
> > +struct xpcie_version {
> > + u8 major;
> > + u8 minor;
> > + u16 build;
> > +} __packed;
>
> Now that the driver will be in the tree, there is no need for this
> structure or these fields, right? It will just be the version of the
> kernel. Otherwise something like this will never be correct, given
> stable kernel releases and enterprise kernel releases, and loads of
> other messy backported trees. Just stick to the kernel release number
> please, that's the only sane thing.
>
Sure, I agree. I will fix it in my v2.
thanks!
Srikanth
> thanks,
>
> greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 08/22] misc: xlink-pcie: lh: Add PCIe EP DMA functionality
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (6 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 07/22] misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic mgross
` (14 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add Synopsys PCIe DWC core embedded-DMA functionality for local host
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/local_host/Makefile | 1 +
drivers/misc/xlink-pcie/local_host/dma.c | 577 ++++++++++++++++++++
drivers/misc/xlink-pcie/local_host/epf.c | 15 +-
drivers/misc/xlink-pcie/local_host/epf.h | 41 ++
4 files changed, 631 insertions(+), 3 deletions(-)
create mode 100644 drivers/misc/xlink-pcie/local_host/dma.c
diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
index 514d3f0c91bc..54fc118e2dd1 100644
--- a/drivers/misc/xlink-pcie/local_host/Makefile
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -1,2 +1,3 @@
obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
mxlk_ep-objs := epf.o
+mxlk_ep-objs += dma.o
diff --git a/drivers/misc/xlink-pcie/local_host/dma.c b/drivers/misc/xlink-pcie/local_host/dma.c
new file mode 100644
index 000000000000..811e5eebb7ab
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/dma.c
@@ -0,0 +1,577 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/wait.h>
+
+#include "epf.h"
+
+#define DMA_DBI_OFFSET (0x380000)
+
+/* PCIe DMA control 1 register definitions. */
+#define DMA_CH_CONTROL1_CB_SHIFT (0)
+#define DMA_CH_CONTROL1_TCB_SHIFT (1)
+#define DMA_CH_CONTROL1_LLP_SHIFT (2)
+#define DMA_CH_CONTROL1_LIE_SHIFT (3)
+#define DMA_CH_CONTROL1_CS_SHIFT (5)
+#define DMA_CH_CONTROL1_CCS_SHIFT (8)
+#define DMA_CH_CONTROL1_LLE_SHIFT (9)
+#define DMA_CH_CONTROL1_CB_MASK (BIT(DMA_CH_CONTROL1_CB_SHIFT))
+#define DMA_CH_CONTROL1_TCB_MASK (BIT(DMA_CH_CONTROL1_TCB_SHIFT))
+#define DMA_CH_CONTROL1_LLP_MASK (BIT(DMA_CH_CONTROL1_LLP_SHIFT))
+#define DMA_CH_CONTROL1_LIE_MASK (BIT(DMA_CH_CONTROL1_LIE_SHIFT))
+#define DMA_CH_CONTROL1_CS_MASK (0x3 << DMA_CH_CONTROL1_CS_SHIFT)
+#define DMA_CH_CONTROL1_CCS_MASK (BIT(DMA_CH_CONTROL1_CCS_SHIFT))
+#define DMA_CH_CONTROL1_LLE_MASK (BIT(DMA_CH_CONTROL1_LLE_SHIFT))
+
+/* DMA control 1 register Channel Status */
+#define DMA_CH_CONTROL1_CS_RUNNING (0x1 << DMA_CH_CONTROL1_CS_SHIFT)
+#define DMA_CH_CONTROL1_CS_HALTED (0x2 << DMA_CH_CONTROL1_CS_SHIFT)
+#define DMA_CH_CONTROL1_CS_STOPPED (0x3 << DMA_CH_CONTROL1_CS_SHIFT)
+
+/* PCIe DMA Engine enable register definitions. */
+#define DMA_ENGINE_EN_SHIFT (0)
+#define DMA_ENGINE_EN_MASK (BIT(DMA_ENGINE_EN_SHIFT))
+
+/* PCIe DMA interrupt registers definitions. */
+#define DMA_ABORT_INTERRUPT_SHIFT (16)
+#define DMA_ABORT_INTERRUPT_MASK (0xFF << DMA_ABORT_INTERRUPT_SHIFT)
+#define DMA_ABORT_INTERRUPT_CH_MASK(_c) (BIT(_c) << DMA_ABORT_INTERRUPT_SHIFT)
+#define DMA_DONE_INTERRUPT_MASK (0xFF)
+#define DMA_DONE_INTERRUPT_CH_MASK(_c) (BIT(_c))
+#define DMA_ALL_INTERRUPT_MASK \
+ (DMA_ABORT_INTERRUPT_MASK | DMA_DONE_INTERRUPT_MASK)
+
+#define DMA_LL_ERROR_SHIFT (16)
+#define DMA_CPL_ABORT_SHIFT (8)
+#define DMA_CPL_TIMEOUT_SHIFT (16)
+#define DMA_DATA_POI_SHIFT (24)
+#define DMA_AR_ERROR_CH_MASK(_c) (BIT(_c))
+#define DMA_LL_ERROR_CH_MASK(_c) (BIT(_c) << DMA_LL_ERROR_SHIFT)
+#define DMA_UNREQ_ERROR_CH_MASK(_c) (BIT(_c))
+#define DMA_CPL_ABORT_ERROR_CH_MASK(_c) (BIT(_c) << DMA_CPL_ABORT_SHIFT)
+#define DMA_CPL_TIMEOUT_ERROR_CH_MASK(_c) (BIT(_c) << DMA_CPL_TIMEOUT_SHIFT)
+#define DMA_DATA_POI_ERROR_CH_MASK(_c) (BIT(_c) << DMA_DATA_POI_SHIFT)
+
+#define DMA_LLLAIE_SHIFT (16)
+#define DMA_LLLAIE_MASK (0xF << DMA_LLLAIE_SHIFT)
+
+#define DMA_CHAN_WRITE_MAX_WEIGHT (0x7)
+#define DMA_CHAN_READ_MAX_WEIGHT (0x3)
+#define DMA_CHAN0_WEIGHT_OFFSET (0)
+#define DMA_CHAN1_WEIGHT_OFFSET (5)
+#define DMA_CHAN2_WEIGHT_OFFSET (10)
+#define DMA_CHAN3_WEIGHT_OFFSET (15)
+#define DMA_CHAN_WRITE_ALL_MAX_WEIGHT \
+ ((DMA_CHAN_WRITE_MAX_WEIGHT << DMA_CHAN0_WEIGHT_OFFSET) | \
+ (DMA_CHAN_WRITE_MAX_WEIGHT << DMA_CHAN1_WEIGHT_OFFSET) | \
+ (DMA_CHAN_WRITE_MAX_WEIGHT << DMA_CHAN2_WEIGHT_OFFSET) | \
+ (DMA_CHAN_WRITE_MAX_WEIGHT << DMA_CHAN3_WEIGHT_OFFSET))
+#define DMA_CHAN_READ_ALL_MAX_WEIGHT \
+ ((DMA_CHAN_READ_MAX_WEIGHT << DMA_CHAN0_WEIGHT_OFFSET) | \
+ (DMA_CHAN_READ_MAX_WEIGHT << DMA_CHAN1_WEIGHT_OFFSET) | \
+ (DMA_CHAN_READ_MAX_WEIGHT << DMA_CHAN2_WEIGHT_OFFSET) | \
+ (DMA_CHAN_READ_MAX_WEIGHT << DMA_CHAN3_WEIGHT_OFFSET))
+
+#define PCIE_REGS_PCIE_APP_CNTRL 0x8
+#define APP_XFER_PENDING BIT(6)
+#define PCIE_REGS_PCIE_SII_PM_STATE_1 0xb4
+#define PM_LINKST_IN_L1 BIT(10)
+
+#define DMA_POLLING_TIMEOUT 1000000
+#define DMA_ENABLE_TIMEOUT 1000
+#define DMA_PCIE_PM_L1_TIMEOUT 20
+
+struct __packed pcie_dma_reg {
+ u32 dma_ctrl_data_arb_prior;
+ u32 reserved1;
+ u32 dma_ctrl;
+ u32 dma_write_engine_en;
+ u32 dma_write_doorbell;
+ u32 reserved2;
+ u32 dma_write_channel_arb_weight_low;
+ u32 dma_write_channel_arb_weight_high;
+ u32 reserved3[3];
+ u32 dma_read_engine_en;
+ u32 dma_read_doorbell;
+ u32 reserved4;
+ u32 dma_read_channel_arb_weight_low;
+ u32 dma_read_channel_arb_weight_high;
+ u32 reserved5[3];
+ u32 dma_write_int_status;
+ u32 reserved6;
+ u32 dma_write_int_mask;
+ u32 dma_write_int_clear;
+ u32 dma_write_err_status;
+ u32 dma_write_done_imwr_low;
+ u32 dma_write_done_imwr_high;
+ u32 dma_write_abort_imwr_low;
+ u32 dma_write_abort_imwr_high;
+ u16 dma_write_ch_imwr_data[8];
+ u32 reserved7[4];
+ u32 dma_write_linked_list_err_en;
+ u32 reserved8[3];
+ u32 dma_read_int_status;
+ u32 reserved9;
+ u32 dma_read_int_mask;
+ u32 dma_read_int_clear;
+ u32 reserved10;
+ u32 dma_read_err_status_low;
+ u32 dma_rd_err_sts_h;
+ u32 reserved11[2];
+ u32 dma_read_linked_list_err_en;
+ u32 reserved12;
+ u32 dma_read_done_imwr_low;
+ u32 dma_read_done_imwr_high;
+ u32 dma_read_abort_imwr_low;
+ u32 dma_read_abort_imwr_high;
+ u16 dma_read_ch_imwr_data[8];
+};
+
+struct __packed pcie_dma_chan {
+ u32 dma_ch_control1;
+ u32 reserved1;
+ u32 dma_transfer_size;
+ u32 dma_sar_low;
+ u32 dma_sar_high;
+ u32 dma_dar_low;
+ u32 dma_dar_high;
+ u32 dma_llp_low;
+ u32 dma_llp_high;
+};
+
+enum xpcie_ep_engine_type {
+ WRITE_ENGINE,
+ READ_ENGINE
+};
+
+static u32 dma_chan_offset[2][DMA_CHAN_NUM] = {
+ { 0x200, 0x400, 0x600, 0x800 },
+ { 0x300, 0x500, 0x700, 0x900 }
+};
+
+static void __iomem *intel_xpcie_ep_get_dma_base(struct pci_epf *epf)
+{
+ struct device *dev = &epf->dev;
+ struct xpcie_epf *xpcie_epf = (struct xpcie_epf *)dev->driver_data;
+
+ return xpcie_epf->dbi_base + DMA_DBI_OFFSET;
+}
+
+static int intel_xpcie_ep_dma_disable(void __iomem *dma_base,
+ enum xpcie_ep_engine_type rw)
+{
+ struct __iomem pcie_dma_reg * dma_reg =
+ (struct __iomem pcie_dma_reg *)dma_base;
+ void __iomem *int_mask, *int_clear;
+ void __iomem *engine_en, *ll_err;
+ int i;
+
+ if (rw == WRITE_ENGINE) {
+ engine_en = (void __iomem *)&dma_reg->dma_write_engine_en;
+ int_mask = (void __iomem *)&dma_reg->dma_write_int_mask;
+ int_clear = (void __iomem *)&dma_reg->dma_write_int_clear;
+ ll_err = (void __iomem *)&dma_reg->dma_write_linked_list_err_en;
+ } else {
+ engine_en = (void __iomem *)&dma_reg->dma_read_engine_en;
+ int_mask = (void __iomem *)&dma_reg->dma_read_int_mask;
+ int_clear = (void __iomem *)&dma_reg->dma_read_int_clear;
+ ll_err = (void __iomem *)&dma_reg->dma_read_linked_list_err_en;
+ }
+
+ iowrite32(0x0, engine_en);
+
+ /* Mask all interrupts. */
+ iowrite32(DMA_ALL_INTERRUPT_MASK, int_mask);
+
+ /* Clear all interrupts. */
+ iowrite32(DMA_ALL_INTERRUPT_MASK, int_clear);
+
+ /* Disable LL abort interrupt (LLLAIE). */
+ iowrite32(0, ll_err);
+
+ /* Wait until the engine is disabled. */
+ for (i = 0; i < DMA_ENABLE_TIMEOUT; i++) {
+ if (!(ioread32(engine_en) & DMA_ENGINE_EN_MASK))
+ return 0;
+ msleep(20);
+ }
+
+ return -EBUSY;
+}
+
+static void intel_xpcie_ep_dma_enable(void __iomem *dma_base,
+ enum xpcie_ep_engine_type rw)
+{
+ struct __iomem pcie_dma_reg * dma_reg =
+ (struct __iomem pcie_dma_reg *)(dma_base);
+ void __iomem *engine_en, *ll_err, *arb_weight;
+ struct __iomem pcie_dma_chan * dma_chan;
+ void __iomem *int_mask, *int_clear;
+ u32 offset, weight;
+ int i;
+
+ if (rw == WRITE_ENGINE) {
+ engine_en = (void __iomem *)&dma_reg->dma_write_engine_en;
+ int_mask = (void __iomem *)&dma_reg->dma_write_int_mask;
+ int_clear = (void __iomem *)&dma_reg->dma_write_int_clear;
+ ll_err = (void __iomem *)&dma_reg->dma_write_linked_list_err_en;
+ arb_weight = (void __iomem *)
+ &dma_reg->dma_write_channel_arb_weight_low;
+ weight = DMA_CHAN_WRITE_ALL_MAX_WEIGHT;
+ } else {
+ engine_en = (void __iomem *)&dma_reg->dma_read_engine_en;
+ int_mask = (void __iomem *)&dma_reg->dma_read_int_mask;
+ int_clear = (void __iomem *)&dma_reg->dma_read_int_clear;
+ ll_err = (void __iomem *)&dma_reg->dma_read_linked_list_err_en;
+ arb_weight = (void __iomem *)
+ &dma_reg->dma_read_channel_arb_weight_low;
+ weight = DMA_CHAN_READ_ALL_MAX_WEIGHT;
+ }
+
+ iowrite32(DMA_ENGINE_EN_MASK, engine_en);
+
+ /* Unmask all interrupts, so that the interrupt line gets asserted. */
+ iowrite32(~(u32)DMA_ALL_INTERRUPT_MASK, int_mask);
+
+ /* Clear all interrupts. */
+ iowrite32(DMA_ALL_INTERRUPT_MASK, int_clear);
+
+ /* Set channel round robin weight. */
+ iowrite32(weight, arb_weight);
+
+ /* Enable LL abort interrupt (LLLAIE). */
+ iowrite32(DMA_LLLAIE_MASK, ll_err);
+
+ /* Enable linked list mode. */
+ for (i = 0; i < DMA_CHAN_NUM; i++) {
+ offset = dma_chan_offset[rw][i];
+ dma_chan = (struct __iomem pcie_dma_chan *)(dma_base + offset);
+ iowrite32(DMA_CH_CONTROL1_LLE_MASK,
+ (void __iomem *)&dma_chan->dma_ch_control1);
+ }
+}
+
+/*
+ * Make sure EP is not in L1 state when DMA doorbell.
+ * The DMA controller may start the wrong channel if doorbell occurs at the
+ * same time as controller is transitioning to L1.
+ */
+static int intel_xpcie_ep_dma_doorbell(struct xpcie_epf *xpcie_epf, int chan,
+ void __iomem *doorbell)
+{
+ int i = DMA_PCIE_PM_L1_TIMEOUT, rc = 0;
+ u32 val, pm_val;
+
+ val = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_APP_CNTRL);
+ iowrite32(val | APP_XFER_PENDING,
+ xpcie_epf->apb_base + PCIE_REGS_PCIE_APP_CNTRL);
+ pm_val = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_SII_PM_STATE_1);
+ while (pm_val & PM_LINKST_IN_L1) {
+ if (i-- < 0) {
+ rc = -ETIME;
+ break;
+ }
+ udelay(5);
+ pm_val = ioread32(xpcie_epf->apb_base +
+ PCIE_REGS_PCIE_SII_PM_STATE_1);
+ }
+
+ iowrite32((u32)chan, doorbell);
+
+ iowrite32(val & ~APP_XFER_PENDING,
+ xpcie_epf->apb_base + PCIE_REGS_PCIE_APP_CNTRL);
+
+ return rc;
+}
+
+static int intel_xpcie_ep_dma_err_status(void __iomem *err_status, int chan)
+{
+ if (ioread32(err_status) &
+ (DMA_AR_ERROR_CH_MASK(chan) | DMA_LL_ERROR_CH_MASK(chan)))
+ return -EIO;
+
+ return 0;
+}
+
+static int intel_xpcie_ep_dma_rd_err_sts_h(void __iomem *err_status,
+ int chan)
+{
+ if (ioread32(err_status) &
+ (DMA_UNREQ_ERROR_CH_MASK(chan) |
+ DMA_CPL_ABORT_ERROR_CH_MASK(chan) |
+ DMA_CPL_TIMEOUT_ERROR_CH_MASK(chan) |
+ DMA_DATA_POI_ERROR_CH_MASK(chan)))
+ return -EIO;
+
+ return 0;
+}
+
+static void
+intel_xpcie_ep_dma_setup_ll_descs(struct __iomem pcie_dma_chan * dma_chan,
+ struct xpcie_dma_ll_desc_buf *desc_buf,
+ int descs_num)
+{
+ struct xpcie_dma_ll_desc *descs = desc_buf->virt;
+ int i;
+
+ /* Setup linked list descriptors */
+ for (i = 0; i < descs_num - 1; i++)
+ descs[i].dma_ch_control1 = DMA_CH_CONTROL1_CB_MASK;
+ descs[descs_num - 1].dma_ch_control1 = DMA_CH_CONTROL1_LIE_MASK |
+ DMA_CH_CONTROL1_CB_MASK;
+ descs[descs_num].dma_ch_control1 = DMA_CH_CONTROL1_LLP_MASK |
+ DMA_CH_CONTROL1_TCB_MASK;
+ descs[descs_num].src_addr = (phys_addr_t)desc_buf->phys;
+
+ /* Setup linked list settings */
+ iowrite32(DMA_CH_CONTROL1_LLE_MASK | DMA_CH_CONTROL1_CCS_MASK,
+ (void __iomem *)&dma_chan->dma_ch_control1);
+ iowrite32((u32)desc_buf->phys, (void __iomem *)&dma_chan->dma_llp_low);
+ iowrite32((u64)desc_buf->phys >> 32,
+ (void __iomem *)&dma_chan->dma_llp_high);
+}
+
+int intel_xpcie_ep_dma_write_ll(struct pci_epf *epf, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ void __iomem *dma_base = xpcie_epf->dma_base;
+ struct __iomem pcie_dma_chan * dma_chan;
+ struct xpcie_dma_ll_desc_buf *desc_buf;
+ struct __iomem pcie_dma_reg * dma_reg =
+ (struct __iomem pcie_dma_reg *)(dma_base);
+ int i, rc;
+
+ if (descs_num <= 0 || descs_num > XPCIE_NUM_TX_DESCS)
+ return -EINVAL;
+
+ if (chan < 0 || chan >= DMA_CHAN_NUM)
+ return -EINVAL;
+
+ dma_chan = (struct __iomem pcie_dma_chan *)
+ (dma_base + dma_chan_offset[WRITE_ENGINE][chan]);
+
+ desc_buf = &xpcie_epf->tx_desc_buf[chan];
+
+ intel_xpcie_ep_dma_setup_ll_descs(dma_chan, desc_buf, descs_num);
+
+ /* Start DMA transfer. */
+ rc = intel_xpcie_ep_dma_doorbell(xpcie_epf, chan,
+ (void __iomem *)
+ &dma_reg->dma_write_doorbell);
+ if (rc)
+ return rc;
+
+ /* Wait for DMA transfer to complete. */
+ for (i = 0; i < DMA_POLLING_TIMEOUT; i++) {
+ usleep_range(5, 10);
+ if (ioread32((void __iomem *)&dma_reg->dma_write_int_status) &
+ (DMA_DONE_INTERRUPT_CH_MASK(chan) |
+ DMA_ABORT_INTERRUPT_CH_MASK(chan)))
+ break;
+ }
+ if (i == DMA_POLLING_TIMEOUT) {
+ dev_err(&xpcie_epf->epf->dev, "DMA Wr timeout\n");
+ rc = -ETIME;
+ goto cleanup;
+ }
+
+ rc = intel_xpcie_ep_dma_err_status((void __iomem *)
+ &dma_reg->dma_write_err_status,
+ chan);
+
+cleanup:
+ /* Clear the done/abort interrupt. */
+ iowrite32((DMA_DONE_INTERRUPT_CH_MASK(chan) |
+ DMA_ABORT_INTERRUPT_CH_MASK(chan)),
+ (void __iomem *)&dma_reg->dma_write_int_clear);
+
+ if (rc) {
+ if (intel_xpcie_ep_dma_disable(dma_base, WRITE_ENGINE)) {
+ dev_err(&xpcie_epf->epf->dev,
+ "failed to disable WR DMA\n");
+ return rc;
+ }
+ intel_xpcie_ep_dma_enable(dma_base, WRITE_ENGINE);
+ }
+
+ return rc;
+}
+
+int intel_xpcie_ep_dma_read_ll(struct pci_epf *epf, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ void __iomem *dma_base = xpcie_epf->dma_base;
+ struct xpcie_dma_ll_desc_buf *desc_buf;
+ struct __iomem pcie_dma_reg * dma_reg =
+ (struct __iomem pcie_dma_reg *)(dma_base);
+ struct __iomem pcie_dma_chan * dma_chan;
+ int i, rc;
+
+ if (descs_num <= 0 || descs_num > XPCIE_NUM_RX_DESCS)
+ return -EINVAL;
+
+ if (chan < 0 || chan >= DMA_CHAN_NUM)
+ return -EINVAL;
+
+ dma_chan = (struct __iomem pcie_dma_chan *)
+ (dma_base + dma_chan_offset[READ_ENGINE][chan]);
+
+ desc_buf = &xpcie_epf->rx_desc_buf[chan];
+
+ intel_xpcie_ep_dma_setup_ll_descs(dma_chan, desc_buf, descs_num);
+
+ /* Start DMA transfer. */
+ rc = intel_xpcie_ep_dma_doorbell(xpcie_epf, chan,
+ (void __iomem *)
+ &dma_reg->dma_read_doorbell);
+ if (rc)
+ return rc;
+
+ /* Wait for DMA transfer to complete. */
+ for (i = 0; i < DMA_POLLING_TIMEOUT; i++) {
+ usleep_range(5, 10);
+ if (ioread32((void __iomem *)&dma_reg->dma_read_int_status) &
+ (DMA_DONE_INTERRUPT_CH_MASK(chan) |
+ DMA_ABORT_INTERRUPT_CH_MASK(chan)))
+ break;
+ }
+ if (i == DMA_POLLING_TIMEOUT) {
+ dev_err(&xpcie_epf->epf->dev, "DMA Rd timeout\n");
+ rc = -ETIME;
+ goto cleanup;
+ }
+
+ rc = intel_xpcie_ep_dma_err_status((void __iomem *)
+ &dma_reg->dma_read_err_status_low,
+ chan);
+ if (!rc) {
+ rc =
+ intel_xpcie_ep_dma_rd_err_sts_h((void __iomem *)
+ &dma_reg->dma_rd_err_sts_h,
+ chan);
+ }
+cleanup:
+ /* Clear the done/abort interrupt. */
+ iowrite32((DMA_DONE_INTERRUPT_CH_MASK(chan) |
+ DMA_ABORT_INTERRUPT_CH_MASK(chan)),
+ (void __iomem *)&dma_reg->dma_read_int_clear);
+
+ if (rc) {
+ if (intel_xpcie_ep_dma_disable(dma_base, READ_ENGINE)) {
+ dev_err(&xpcie_epf->epf->dev,
+ "failed to disable RD DMA\n");
+ return rc;
+ }
+ intel_xpcie_ep_dma_enable(dma_base, READ_ENGINE);
+ }
+
+ return rc;
+}
+
+static void intel_xpcie_ep_dma_free_ll_descs_mem(struct xpcie_epf *xpcie_epf)
+{
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ int i;
+
+ for (i = 0; i < DMA_CHAN_NUM; i++) {
+ if (xpcie_epf->tx_desc_buf[i].virt) {
+ dma_free_coherent(dma_dev,
+ xpcie_epf->tx_desc_buf[i].size,
+ xpcie_epf->tx_desc_buf[i].virt,
+ xpcie_epf->tx_desc_buf[i].phys);
+ }
+ if (xpcie_epf->rx_desc_buf[i].virt) {
+ dma_free_coherent(dma_dev,
+ xpcie_epf->rx_desc_buf[i].size,
+ xpcie_epf->rx_desc_buf[i].virt,
+ xpcie_epf->rx_desc_buf[i].phys);
+ }
+
+ memset(&xpcie_epf->tx_desc_buf[i], 0,
+ sizeof(struct xpcie_dma_ll_desc_buf));
+ memset(&xpcie_epf->rx_desc_buf[i], 0,
+ sizeof(struct xpcie_dma_ll_desc_buf));
+ }
+}
+
+static int intel_xpcie_ep_dma_alloc_ll_descs_mem(struct xpcie_epf *xpcie_epf)
+{
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ int tx_num = XPCIE_NUM_TX_DESCS + 1;
+ int rx_num = XPCIE_NUM_RX_DESCS + 1;
+ size_t tx_size, rx_size;
+ int i;
+
+ tx_size = tx_num * sizeof(struct xpcie_dma_ll_desc);
+ rx_size = rx_num * sizeof(struct xpcie_dma_ll_desc);
+
+ for (i = 0; i < DMA_CHAN_NUM; i++) {
+ xpcie_epf->tx_desc_buf[i].virt =
+ dma_alloc_coherent(dma_dev, tx_size,
+ &xpcie_epf->tx_desc_buf[i].phys,
+ GFP_KERNEL);
+ xpcie_epf->rx_desc_buf[i].virt =
+ dma_alloc_coherent(dma_dev, rx_size,
+ &xpcie_epf->rx_desc_buf[i].phys,
+ GFP_KERNEL);
+
+ if (!xpcie_epf->tx_desc_buf[i].virt ||
+ !xpcie_epf->rx_desc_buf[i].virt) {
+ intel_xpcie_ep_dma_free_ll_descs_mem(xpcie_epf);
+ return -ENOMEM;
+ }
+
+ xpcie_epf->tx_desc_buf[i].size = tx_size;
+ xpcie_epf->rx_desc_buf[i].size = rx_size;
+ }
+ return 0;
+}
+
+int intel_xpcie_ep_dma_reset(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+
+ /* Disable the DMA read/write engine. */
+ if (intel_xpcie_ep_dma_disable(xpcie_epf->dma_base, WRITE_ENGINE) ||
+ intel_xpcie_ep_dma_disable(xpcie_epf->dma_base, READ_ENGINE))
+ return -EBUSY;
+
+ intel_xpcie_ep_dma_enable(xpcie_epf->dma_base, WRITE_ENGINE);
+ intel_xpcie_ep_dma_enable(xpcie_epf->dma_base, READ_ENGINE);
+
+ return 0;
+}
+
+int intel_xpcie_ep_dma_uninit(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+
+ if (intel_xpcie_ep_dma_disable(xpcie_epf->dma_base, WRITE_ENGINE) ||
+ intel_xpcie_ep_dma_disable(xpcie_epf->dma_base, READ_ENGINE))
+ return -EBUSY;
+
+ intel_xpcie_ep_dma_free_ll_descs_mem(xpcie_epf);
+
+ return 0;
+}
+
+int intel_xpcie_ep_dma_init(struct pci_epf *epf)
+{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
+ int rc;
+
+ xpcie_epf->dma_base = intel_xpcie_ep_get_dma_base(epf);
+
+ rc = intel_xpcie_ep_dma_alloc_ll_descs_mem(xpcie_epf);
+ if (rc)
+ return rc;
+
+ return intel_xpcie_ep_dma_reset(epf);
+}
diff --git a/drivers/misc/xlink-pcie/local_host/epf.c b/drivers/misc/xlink-pcie/local_host/epf.c
index ae07b364734d..fc29180aa508 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.c
+++ b/drivers/misc/xlink-pcie/local_host/epf.c
@@ -45,6 +45,8 @@ static irqreturn_t intel_xpcie_err_interrupt(int irq, void *args)
xpcie_epf = container_of(xpcie, struct xpcie_epf, xpcie);
val = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_ERR_INTR_FLAGS);
+ if (val & LINK_REQ_RST_FLG)
+ intel_xpcie_ep_dma_reset(xpcie_epf->epf);
iowrite32(val, xpcie_epf->apb_base + PCIE_REGS_PCIE_ERR_INTR_FLAGS);
@@ -325,8 +327,17 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
goto err_cleanup_bars;
}
+ ret = intel_xpcie_ep_dma_init(epf);
+ if (ret) {
+ dev_err(&epf->dev, "DMA initialization failed\n");
+ goto err_free_err_irq;
+ }
+
return 0;
+err_free_err_irq:
+ free_irq(xpcie_epf->irq_err, &xpcie_epf->xpcie);
+
err_cleanup_bars:
intel_xpcie_cleanup_bars(epf);
@@ -335,11 +346,9 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
static void intel_xpcie_epf_unbind(struct pci_epf *epf)
{
- struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
struct pci_epc *epc = epf->epc;
- free_irq(xpcie_epf->irq, &xpcie_epf->xpcie);
- free_irq(xpcie_epf->irq_err, &xpcie_epf->xpcie);
+ intel_xpcie_ep_dma_uninit(epf);
pci_epc_stop(epc);
diff --git a/drivers/misc/xlink-pcie/local_host/epf.h b/drivers/misc/xlink-pcie/local_host/epf.h
index 2b38c87b3701..6ce5260e67be 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.h
+++ b/drivers/misc/xlink-pcie/local_host/epf.h
@@ -20,6 +20,38 @@
#define KEEMBAY_XPCIE_STEPPING_MAXLEN 8
+#define DMA_CHAN_NUM (4)
+
+#define XPCIE_NUM_TX_DESCS (64)
+#define XPCIE_NUM_RX_DESCS (64)
+
+extern bool dma_ll_mode;
+
+struct xpcie_dma_ll_desc {
+ u32 dma_ch_control1;
+ u32 dma_transfer_size;
+ union {
+ struct {
+ u32 dma_sar_low;
+ u32 dma_sar_high;
+ };
+ phys_addr_t src_addr;
+ };
+ union {
+ struct {
+ u32 dma_dar_low;
+ u32 dma_dar_high;
+ };
+ phys_addr_t dst_addr;
+ };
+} __packed;
+
+struct xpcie_dma_ll_desc_buf {
+ struct xpcie_dma_ll_desc *virt;
+ dma_addr_t phys;
+ size_t size;
+};
+
struct xpcie_epf {
struct pci_epf *epf;
void *vaddr[BAR_5 + 1];
@@ -34,6 +66,15 @@ struct xpcie_epf {
void __iomem *dma_base;
void __iomem *dbi_base;
char stepping[KEEMBAY_XPCIE_STEPPING_MAXLEN];
+
+ struct xpcie_dma_ll_desc_buf tx_desc_buf[DMA_CHAN_NUM];
+ struct xpcie_dma_ll_desc_buf rx_desc_buf[DMA_CHAN_NUM];
};
+int intel_xpcie_ep_dma_init(struct pci_epf *epf);
+int intel_xpcie_ep_dma_uninit(struct pci_epf *epf);
+int intel_xpcie_ep_dma_reset(struct pci_epf *epf);
+int intel_xpcie_ep_dma_read_ll(struct pci_epf *epf, int chan, int descs_num);
+int intel_xpcie_ep_dma_write_ll(struct pci_epf *epf, int chan, int descs_num);
+
#endif /* XPCIE_EPF_HEADER_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (7 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 08/22] misc: xlink-pcie: lh: Add PCIe EP DMA functionality mgross
@ 2020-11-30 23:06 ` mgross
2020-12-01 10:15 ` Greg Kroah-Hartman
2020-11-30 23:06 ` [PATCH 10/22] misc: xlink-pcie: lh: Prepare changes for adding remote host driver mgross
` (13 subsequent siblings)
22 siblings, 1 reply; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add logic to establish communication with the remote host which is through
ring buffer management and MSI/Doorbell interrupts
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/local_host/Makefile | 2 +
drivers/misc/xlink-pcie/local_host/core.c | 894 ++++++++++++++++++++
drivers/misc/xlink-pcie/local_host/core.h | 247 ++++++
drivers/misc/xlink-pcie/local_host/epf.c | 116 ++-
drivers/misc/xlink-pcie/local_host/epf.h | 26 +
drivers/misc/xlink-pcie/local_host/util.c | 375 ++++++++
drivers/misc/xlink-pcie/local_host/util.h | 70 ++
drivers/misc/xlink-pcie/local_host/xpcie.h | 65 ++
include/linux/xlink_drv_inf.h | 60 ++
9 files changed, 1847 insertions(+), 8 deletions(-)
create mode 100644 drivers/misc/xlink-pcie/local_host/core.c
create mode 100644 drivers/misc/xlink-pcie/local_host/core.h
create mode 100644 drivers/misc/xlink-pcie/local_host/util.c
create mode 100644 drivers/misc/xlink-pcie/local_host/util.h
create mode 100644 include/linux/xlink_drv_inf.h
diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
index 54fc118e2dd1..28761751d43b 100644
--- a/drivers/misc/xlink-pcie/local_host/Makefile
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
mxlk_ep-objs := epf.o
mxlk_ep-objs += dma.o
+mxlk_ep-objs += core.o
+mxlk_ep-objs += util.o
diff --git a/drivers/misc/xlink-pcie/local_host/core.c b/drivers/misc/xlink-pcie/local_host/core.c
new file mode 100644
index 000000000000..aecaaa783153
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/core.c
@@ -0,0 +1,894 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include <linux/of_reserved_mem.h>
+
+#include "epf.h"
+#include "core.h"
+#include "util.h"
+
+static struct xpcie *global_xpcie;
+
+static int rx_pool_size = SZ_32M;
+module_param(rx_pool_size, int, 0664);
+MODULE_PARM_DESC(rx_pool_size, "receiving pool size (default 32 MiB)");
+
+static int tx_pool_size = SZ_32M;
+module_param(tx_pool_size, int, 0664);
+MODULE_PARM_DESC(tx_pool_size, "transmitting pool size (default 32 MiB)");
+
+static int fragment_size = XPCIE_FRAGMENT_SIZE;
+module_param(fragment_size, int, 0664);
+MODULE_PARM_DESC(fragment_size, "transfer descriptor size (default 128 KiB)");
+
+static bool tx_pool_coherent = true;
+module_param(tx_pool_coherent, bool, 0664);
+MODULE_PARM_DESC(tx_pool_coherent,
+ "transmitting pool using coherent memory (default true)");
+
+static bool rx_pool_coherent;
+module_param(rx_pool_coherent, bool, 0664);
+MODULE_PARM_DESC(rx_pool_coherent,
+ "receiving pool using coherent memory (default false)");
+
+static struct xpcie *intel_xpcie_core_get_by_id(u32 sw_device_id)
+{
+ return (sw_device_id == xlink_sw_id) ? global_xpcie : NULL;
+}
+
+static int intel_xpcie_map_dma(struct xpcie *xpcie, struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+ struct device *dma_dev = epf->epc->dev.parent;
+
+ bd->phys = dma_map_single(dma_dev, bd->data, bd->length, direction);
+
+ return dma_mapping_error(dma_dev, bd->phys);
+}
+
+static void intel_xpcie_unmap_dma(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd, int direction)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+ struct device *dma_dev = epf->epc->dev.parent;
+
+ dma_unmap_single(dma_dev, bd->phys, bd->length, direction);
+}
+
+static void intel_xpcie_set_cap_txrx(struct xpcie *xpcie)
+{
+ size_t tx_len = sizeof(struct xpcie_transfer_desc) *
+ XPCIE_NUM_TX_DESCS;
+ size_t rx_len = sizeof(struct xpcie_transfer_desc) *
+ XPCIE_NUM_RX_DESCS;
+ size_t hdr_len = sizeof(struct xpcie_cap_txrx);
+ u32 start = sizeof(struct xpcie_mmio);
+ struct xpcie_cap_txrx *cap;
+ struct xpcie_cap_hdr *hdr;
+ u16 next;
+
+ next = (u16)(start + hdr_len + tx_len + rx_len);
+ intel_xpcie_iowrite32(start, xpcie->mmio + XPCIE_MMIO_CAP_OFF);
+ cap = (void *)xpcie->mmio + start;
+ memset(cap, 0, sizeof(struct xpcie_cap_txrx));
+ cap->hdr.id = XPCIE_CAP_TXRX;
+ cap->hdr.next = next;
+ cap->fragment_size = fragment_size;
+ cap->tx.ring = start + hdr_len;
+ cap->tx.ndesc = XPCIE_NUM_TX_DESCS;
+ cap->rx.ring = start + hdr_len + tx_len;
+ cap->rx.ndesc = XPCIE_NUM_RX_DESCS;
+
+ hdr = (struct xpcie_cap_hdr *)((void *)xpcie->mmio + next);
+ hdr->id = XPCIE_CAP_NULL;
+}
+
+static int intel_xpcie_set_version(struct xpcie *xpcie)
+{
+ struct xpcie_version version;
+
+ version.major = XPCIE_VERSION_MAJOR;
+ version.minor = XPCIE_VERSION_MINOR;
+ version.build = XPCIE_VERSION_BUILD;
+
+ memcpy(xpcie->mmio + XPCIE_MMIO_VERSION, &version, sizeof(version));
+
+ dev_dbg(xpcie_to_dev(xpcie), "ver: device %u.%u.%u\n",
+ version.major, version.minor, version.build);
+
+ return 0;
+}
+
+static void intel_xpcie_txrx_cleanup(struct xpcie *xpcie)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_transfer_desc *td;
+ int index;
+
+ xpcie->stop_flag = true;
+ xpcie->no_tx_buffer = false;
+ inf->data_avail = true;
+ wake_up_interruptible(&xpcie->tx_waitq);
+ wake_up_interruptible(&inf->rx_waitq);
+ mutex_lock(&xpcie->wlock);
+ mutex_lock(&inf->rlock);
+
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ td = rx->pipe.tdr + index;
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ for (index = 0; index < tx->pipe.ndesc; index++) {
+ td = tx->pipe.tdr + index;
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+
+ intel_xpcie_list_cleanup(&xpcie->tx_pool);
+ intel_xpcie_list_cleanup(&xpcie->rx_pool);
+
+ if (rx_pool_coherent && xpcie_epf->rx_virt) {
+ dma_free_coherent(dma_dev, xpcie_epf->rx_size,
+ xpcie_epf->rx_virt, xpcie_epf->rx_phys);
+ }
+
+ if (tx_pool_coherent && xpcie_epf->tx_virt) {
+ dma_free_coherent(dma_dev, xpcie_epf->tx_size,
+ xpcie_epf->tx_virt, xpcie_epf->tx_phys);
+ }
+
+ mutex_unlock(&inf->rlock);
+ mutex_unlock(&xpcie->wlock);
+}
+
+/*
+ * The RX/TX are named for Remote Host, in Local Host
+ * RX/TX is reversed.
+ */
+static int intel_xpcie_txrx_init(struct xpcie *xpcie,
+ struct xpcie_cap_txrx *cap)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int index, ndesc, rc;
+
+ xpcie->txrx = cap;
+ xpcie->fragment_size = cap->fragment_size;
+ xpcie->stop_flag = false;
+
+ rx->pipe.ndesc = cap->tx.ndesc;
+ rx->pipe.head = &cap->tx.head;
+ rx->pipe.tail = &cap->tx.tail;
+ rx->pipe.tdr = (void *)xpcie->mmio + cap->tx.ring;
+
+ tx->pipe.ndesc = cap->rx.ndesc;
+ tx->pipe.head = &cap->rx.head;
+ tx->pipe.tail = &cap->rx.tail;
+ tx->pipe.tdr = (void *)xpcie->mmio + cap->rx.ring;
+
+ intel_xpcie_list_init(&xpcie->rx_pool);
+ rx_pool_size = roundup(rx_pool_size, xpcie->fragment_size);
+ ndesc = rx_pool_size / xpcie->fragment_size;
+
+ /* Initialize reserved memory resources */
+ rc = of_reserved_mem_device_init(dma_dev);
+ if (rc) {
+ dev_err(dma_dev, "Could not get reserved memory\n");
+ goto error;
+ }
+
+ if (rx_pool_coherent) {
+ xpcie_epf->rx_size = rx_pool_size;
+ xpcie_epf->rx_virt = dma_alloc_coherent(dma_dev,
+ xpcie_epf->rx_size,
+ &xpcie_epf->rx_phys,
+ GFP_KERNEL);
+ if (!xpcie_epf->rx_virt)
+ goto error;
+ }
+
+ for (index = 0; index < ndesc; index++) {
+ if (rx_pool_coherent) {
+ bd =
+ intel_xpcie_alloc_bd_reuse(xpcie->fragment_size,
+ xpcie_epf->rx_virt +
+ (index *
+ xpcie->fragment_size),
+ xpcie_epf->rx_phys +
+ (index *
+ xpcie->fragment_size));
+ } else {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ }
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to alloc all rx pool descriptors\n");
+ goto error;
+ }
+ }
+
+ intel_xpcie_list_init(&xpcie->tx_pool);
+ tx_pool_size = roundup(tx_pool_size, xpcie->fragment_size);
+ ndesc = tx_pool_size / xpcie->fragment_size;
+
+ if (tx_pool_coherent) {
+ xpcie_epf->tx_size = tx_pool_size;
+ xpcie_epf->tx_virt = dma_alloc_coherent(dma_dev,
+ xpcie_epf->tx_size,
+ &xpcie_epf->tx_phys,
+ GFP_KERNEL);
+ if (!xpcie_epf->tx_virt)
+ goto error;
+ }
+
+ for (index = 0; index < ndesc; index++) {
+ if (tx_pool_coherent) {
+ bd =
+ intel_xpcie_alloc_bd_reuse(xpcie->fragment_size,
+ xpcie_epf->tx_virt +
+ (index *
+ xpcie->fragment_size),
+ xpcie_epf->tx_phys +
+ (index *
+ xpcie->fragment_size));
+ } else {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ }
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to alloc all tx pool descriptors\n");
+ goto error;
+ }
+ }
+
+ return 0;
+
+error:
+ intel_xpcie_txrx_cleanup(xpcie);
+
+ return -ENOMEM;
+}
+
+static int intel_xpcie_discover_txrx(struct xpcie *xpcie)
+{
+ struct xpcie_cap_txrx *cap;
+ int error;
+
+ cap = intel_xpcie_cap_find(xpcie, 0, XPCIE_CAP_TXRX);
+ if (cap) {
+ error = intel_xpcie_txrx_init(xpcie, cap);
+ } else {
+ dev_err(xpcie_to_dev(xpcie), "xpcie txrx info not found\n");
+ error = -EIO;
+ }
+
+ return error;
+}
+
+static void intel_xpcie_start_tx(struct xpcie *xpcie, unsigned long delay)
+{
+ /*
+ * Use only one WQ for both Rx and Tx
+ *
+ * Synchronous Read and Writes to DDR is found to result in memory
+ * mismatch errors in stability tests due to silicon bug in A0 SoC.
+ */
+ if (xpcie->legacy_a0)
+ queue_delayed_work(xpcie->rx_wq, &xpcie->tx_event, delay);
+ else
+ queue_delayed_work(xpcie->tx_wq, &xpcie->tx_event, delay);
+}
+
+static void intel_xpcie_start_rx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->rx_wq, &xpcie->rx_event, delay);
+}
+
+static void intel_xpcie_rx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, rx_event.work);
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct xpcie_buf_desc *bd_head, *bd_tail, *bd;
+ u32 head, tail, ndesc, length, initial_head;
+ unsigned long delay = msecs_to_jiffies(1);
+ struct xpcie_stream *rx = &xpcie->rx;
+ int descs_num = 0, chan = 0, rc;
+ struct xpcie_dma_ll_desc *desc;
+ struct xpcie_transfer_desc *td;
+ bool reset_work = false;
+ u16 interface;
+ u64 address;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ bd_head = NULL;
+ bd_tail = NULL;
+ ndesc = rx->pipe.ndesc;
+ tail = intel_xpcie_get_tdr_tail(&rx->pipe);
+ initial_head = intel_xpcie_get_tdr_head(&rx->pipe);
+ head = initial_head;
+
+ while (head != tail) {
+ td = rx->pipe.tdr + head;
+
+ bd = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!bd) {
+ reset_work = true;
+ if (descs_num == 0) {
+ delay = msecs_to_jiffies(10);
+ goto task_exit;
+ }
+ break;
+ }
+
+ interface = intel_xpcie_get_td_interface(td);
+ length = intel_xpcie_get_td_length(td);
+ address = intel_xpcie_get_td_address(td);
+
+ bd->length = length;
+ bd->interface = interface;
+ if (!rx_pool_coherent) {
+ rc = intel_xpcie_map_dma(xpcie, bd, DMA_FROM_DEVICE);
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to map rx bd (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ break;
+ }
+ }
+
+ desc = &xpcie_epf->rx_desc_buf[chan].virt[descs_num++];
+ desc->dma_transfer_size = length;
+ desc->dst_addr = bd->phys;
+ desc->src_addr = address;
+
+ if (bd_head)
+ bd_tail->next = bd;
+ else
+ bd_head = bd;
+ bd_tail = bd;
+
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (descs_num == 0)
+ goto task_exit;
+
+ rc = intel_xpcie_copy_from_host_ll(xpcie, chan, descs_num);
+
+ bd = bd_head;
+ while (bd && !rx_pool_coherent) {
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_FROM_DEVICE);
+ bd = bd->next;
+ }
+
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to DMA from host (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, bd_head);
+ delay = msecs_to_jiffies(5);
+ reset_work = true;
+ goto task_exit;
+ }
+
+ head = initial_head;
+ bd = bd_head;
+ while (bd) {
+ td = rx->pipe.tdr + head;
+ bd_head = bd_head->next;
+ bd->next = NULL;
+
+ if (likely(bd->interface < XPCIE_NUM_INTERFACES)) {
+ intel_xpcie_set_td_status(td,
+ XPCIE_DESC_STATUS_SUCCESS);
+ intel_xpcie_add_bd_to_interface(xpcie, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "detected rx desc interface failure (%u)\n",
+ bd->interface);
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_ERROR);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ }
+
+ bd = bd_head;
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (head != initial_head) {
+ intel_xpcie_set_tdr_head(&rx->pipe, head);
+ intel_xpcie_raise_irq(xpcie, DATA_RECEIVED);
+ }
+
+task_exit:
+ if (reset_work)
+ intel_xpcie_start_rx(xpcie, delay);
+}
+
+static void intel_xpcie_tx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, tx_event.work);
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct xpcie_buf_desc *bd_head, *bd_tail, *bd;
+ struct xpcie_stream *tx = &xpcie->tx;
+ u32 head, tail, ndesc, initial_tail;
+ struct xpcie_dma_ll_desc *desc;
+ struct xpcie_transfer_desc *td;
+ int descs_num = 0, chan = 0, rc;
+ size_t buffers = 0, bytes = 0;
+ u64 address;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ bd_head = NULL;
+ bd_tail = NULL;
+ ndesc = tx->pipe.ndesc;
+ initial_tail = intel_xpcie_get_tdr_tail(&tx->pipe);
+ tail = initial_tail;
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+
+ /* add new entries */
+ while (XPCIE_CIRCULAR_INC(tail, ndesc) != head) {
+ bd = intel_xpcie_list_get(&xpcie->write);
+ if (!bd)
+ break;
+
+ if (!tx_pool_coherent) {
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_TO_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "dma map error bd addr %p, size %zu\n",
+ bd->data, bd->length);
+ intel_xpcie_list_put_head(&xpcie->write, bd);
+ break;
+ }
+ }
+
+ td = tx->pipe.tdr + tail;
+ address = intel_xpcie_get_td_address(td);
+
+ desc = &xpcie_epf->tx_desc_buf[chan].virt[descs_num++];
+ desc->dma_transfer_size = bd->length;
+ desc->src_addr = bd->phys;
+ desc->dst_addr = address;
+
+ if (bd_head)
+ bd_tail->next = bd;
+ else
+ bd_head = bd;
+ bd_tail = bd;
+
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (descs_num == 0)
+ goto task_exit;
+
+ rc = intel_xpcie_copy_to_host_ll(xpcie, chan, descs_num);
+
+ tail = initial_tail;
+ bd = bd_head;
+ while (bd) {
+ if (!tx_pool_coherent)
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+
+ if (rc) {
+ bd = bd->next;
+ continue;
+ }
+
+ td = tx->pipe.tdr + tail;
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_SUCCESS);
+ intel_xpcie_set_td_length(td, bd->length);
+ intel_xpcie_set_td_interface(td, bd->interface);
+
+ bd = bd->next;
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to DMA to host (%d)\n", rc);
+ intel_xpcie_list_put_head(&xpcie->write, bd_head);
+ return;
+ }
+
+ intel_xpcie_free_tx_bd(xpcie, bd_head);
+
+ if (intel_xpcie_get_tdr_tail(&tx->pipe) != tail) {
+ intel_xpcie_set_tdr_tail(&tx->pipe, tail);
+ intel_xpcie_raise_irq(xpcie, DATA_SENT);
+ }
+
+task_exit:
+ intel_xpcie_list_info(&xpcie->write, &bytes, &buffers);
+ if (buffers) {
+ xpcie->tx_pending = true;
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+ if (XPCIE_CIRCULAR_INC(tail, ndesc) != head)
+ intel_xpcie_start_tx(xpcie, 0);
+ } else {
+ xpcie->tx_pending = false;
+ }
+}
+
+static irqreturn_t intel_xpcie_core_irq_cb(int irq, void *args)
+{
+ struct xpcie *xpcie = args;
+
+ if (intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DATA_SENT)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_SENT, 0);
+ intel_xpcie_start_rx(xpcie, 0);
+ }
+ if (intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED, 0);
+ if (xpcie->tx_pending)
+ intel_xpcie_start_tx(xpcie, 0);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int intel_xpcie_events_init(struct xpcie *xpcie)
+{
+ xpcie->rx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->rx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ return -ENOMEM;
+ }
+
+ if (!xpcie->legacy_a0) {
+ xpcie->tx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM |
+ WQ_HIGHPRI);
+ if (!xpcie->tx_wq) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to allocate workqueue\n");
+ destroy_workqueue(xpcie->rx_wq);
+ return -ENOMEM;
+ }
+ }
+
+ INIT_DELAYED_WORK(&xpcie->rx_event, intel_xpcie_rx_event_handler);
+ INIT_DELAYED_WORK(&xpcie->tx_event, intel_xpcie_tx_event_handler);
+
+ return 0;
+}
+
+static void intel_xpcie_events_cleanup(struct xpcie *xpcie)
+{
+ cancel_delayed_work_sync(&xpcie->rx_event);
+ cancel_delayed_work_sync(&xpcie->tx_event);
+
+ destroy_workqueue(xpcie->rx_wq);
+ if (!xpcie->legacy_a0)
+ destroy_workqueue(xpcie->tx_wq);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie)
+{
+ int error;
+
+ global_xpcie = xpcie;
+
+ intel_xpcie_set_version(xpcie);
+ intel_xpcie_set_cap_txrx(xpcie);
+
+ error = intel_xpcie_events_init(xpcie);
+ if (error)
+ return error;
+
+ error = intel_xpcie_discover_txrx(xpcie);
+ if (error)
+ goto error_txrx;
+
+ intel_xpcie_interfaces_init(xpcie);
+
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_SENT, 0);
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED, 0);
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DEV_EVENT, NO_OP);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_SENT, 0);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED, 0);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DEV_EVENT, NO_OP);
+
+ intel_xpcie_register_host_irq(xpcie, intel_xpcie_core_irq_cb);
+
+ return 0;
+
+error_txrx:
+ intel_xpcie_events_cleanup(xpcie);
+
+ return error;
+}
+
+void intel_xpcie_core_cleanup(struct xpcie *xpcie)
+{
+ if (xpcie->status == XPCIE_STATUS_RUN) {
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_interfaces_cleanup(xpcie);
+ intel_xpcie_txrx_cleanup(xpcie);
+ }
+}
+
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer,
+ size_t *length, u32 timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd;
+ long jiffies_passed = 0;
+ size_t len, remaining;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ while (!inf->data_avail) {
+ mutex_unlock(&inf->rlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(inf->rx_waitq,
+ inf->data_avail);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(inf->rx_waitq,
+ inf->data_avail,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+ }
+
+ bd = (inf->partial_read) ? inf->partial_read :
+ intel_xpcie_list_get(&inf->read);
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(remaining, bd->length);
+ memcpy(buffer, bd->data, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->data += bcopy;
+ bd->length -= bcopy;
+
+ if (bd->length == 0) {
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ bd = intel_xpcie_list_get(&inf->read);
+ }
+ }
+
+ /* save for next time */
+ inf->partial_read = bd;
+
+ if (!bd)
+ inf->data_avail = false;
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&inf->rlock);
+
+ return 0;
+}
+
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer,
+ size_t *length, u32 timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd, *head;
+ long jiffies_passed = 0;
+ size_t remaining, len;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ while (!head) {
+ mutex_unlock(&xpcie->wlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ }
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(bd->length, remaining);
+ memcpy(bd->data, buffer, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->length = bcopy;
+ bd->interface = inf->id;
+
+ if (remaining) {
+ bd->next = intel_xpcie_alloc_tx_bd(xpcie);
+ bd = bd->next;
+ }
+ }
+
+ intel_xpcie_list_put(&inf->xpcie->write, head);
+ intel_xpcie_start_tx(xpcie, 0);
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&xpcie->wlock);
+
+ return 0;
+}
+
+int intel_xpcie_get_device_status_by_id(u32 id, u32 *status)
+{
+ struct xpcie *xpcie = intel_xpcie_core_get_by_id(id);
+
+ if (!xpcie)
+ return -ENODEV;
+
+ *status = xpcie->status;
+
+ return 0;
+}
+
+u32 intel_xpcie_get_device_num(u32 *id_list)
+{
+ u32 num_devices = 0;
+
+ if (xlink_sw_id) {
+ num_devices = 1;
+ *id_list = xlink_sw_id;
+ }
+
+ return num_devices;
+}
+
+int intel_xpcie_get_device_name_by_id(u32 id,
+ char *device_name, size_t name_size)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ memset(device_name, 0, name_size);
+ if (name_size > strlen(XPCIE_DRIVER_NAME))
+ name_size = strlen(XPCIE_DRIVER_NAME);
+ memcpy(device_name, XPCIE_DRIVER_NAME, name_size);
+
+ return 0;
+}
+
+int intel_xpcie_pci_connect_device(u32 id)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -EIO;
+
+ return 0;
+}
+
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ return intel_xpcie_core_read(xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ return intel_xpcie_core_write(xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_reset_device(u32 id)
+{
+ return 0;
+}
diff --git a/drivers/misc/xlink-pcie/local_host/core.h b/drivers/misc/xlink-pcie/local_host/core.h
new file mode 100644
index 000000000000..84985ef41a64
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/core.h
@@ -0,0 +1,247 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_CORE_HEADER_
+#define XPCIE_CORE_HEADER_
+
+#include <linux/io.h>
+#include <linux/types.h>
+#include <linux/workqueue.h>
+#include <linux/slab.h>
+#include <linux/mutex.h>
+#include <linux/mempool.h>
+#include <linux/dma-mapping.h>
+#include <linux/cache.h>
+#include <linux/wait.h>
+
+#include <linux/xlink_drv_inf.h>
+
+/* Number of interfaces to statically allocate resources for */
+#define XPCIE_NUM_INTERFACES (1)
+
+/* max should be always power of '2' */
+#define XPCIE_CIRCULAR_INC(val, max) (((val) + 1) & ((max) - 1))
+
+#define XPCIE_FRAGMENT_SIZE SZ_128K
+
+/* Status encoding of the transfer descriptors */
+#define XPCIE_DESC_STATUS_SUCCESS (0)
+#define XPCIE_DESC_STATUS_ERROR (0xFFFF)
+
+/* Layout transfer descriptors used by device and host */
+struct xpcie_transfer_desc {
+ u64 address;
+ u32 length;
+ u16 status;
+ u16 interface;
+} __packed;
+
+struct xpcie_pipe {
+ u32 old;
+ u32 ndesc;
+ u32 *head;
+ u32 *tail;
+ struct xpcie_transfer_desc *tdr;
+};
+
+struct xpcie_buf_desc {
+ struct xpcie_buf_desc *next;
+ void *head;
+ dma_addr_t phys;
+ size_t true_len;
+ void *data;
+ size_t length;
+ int interface;
+ bool own_mem;
+};
+
+struct xpcie_stream {
+ size_t frag;
+ struct xpcie_pipe pipe;
+};
+
+struct xpcie_list {
+ spinlock_t lock; /* list lock */
+ size_t bytes;
+ size_t buffers;
+ struct xpcie_buf_desc *head;
+ struct xpcie_buf_desc *tail;
+};
+
+struct xpcie_interface {
+ int id;
+ struct xpcie *xpcie;
+ struct mutex rlock; /* read lock */
+ struct xpcie_list read;
+ struct xpcie_buf_desc *partial_read;
+ bool data_avail;
+ wait_queue_head_t rx_waitq;
+};
+
+struct xpcie_debug_stats {
+ struct {
+ size_t cnts;
+ size_t bytes;
+ } tx_krn, rx_krn, tx_usr, rx_usr;
+ size_t send_ints;
+ size_t interrupts;
+ size_t rx_event_runs;
+ size_t tx_event_runs;
+};
+
+/* Defined capabilities located in mmio space */
+#define XPCIE_CAP_NULL (0)
+#define XPCIE_CAP_TXRX (1)
+
+#define XPCIE_CAP_TTL (32)
+#define XPCIE_CAP_HDR_ID (offsetof(struct xpcie_cap_hdr, id))
+#define XPCIE_CAP_HDR_NEXT (offsetof(struct xpcie_cap_hdr, next))
+
+/* Header at the beginning of each capability to define and link to next */
+struct xpcie_cap_hdr {
+ u16 id;
+ u16 next;
+} __packed;
+
+struct xpcie_cap_pipe {
+ u32 ring;
+ u32 ndesc;
+ u32 head;
+ u32 tail;
+} __packed;
+
+/* Transmit and Receive capability */
+struct xpcie_cap_txrx {
+ struct xpcie_cap_hdr hdr;
+ u32 fragment_size;
+ struct xpcie_cap_pipe tx;
+ struct xpcie_cap_pipe rx;
+} __packed;
+
+static inline u64 _ioread64(void __iomem *addr)
+{
+ u64 low, high;
+
+ low = ioread32(addr);
+ high = ioread32(addr + sizeof(u32));
+
+ return low | (high << 32);
+}
+
+static inline void _iowrite64(u64 value, void __iomem *addr)
+{
+ iowrite32(value, addr);
+ iowrite32(value >> 32, addr + sizeof(u32));
+}
+
+#define intel_xpcie_iowrite64(value, addr) \
+ _iowrite64(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite32(value, addr) \
+ iowrite32(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite16(value, addr) \
+ iowrite16(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite8(value, addr) \
+ iowrite8(value, (void __iomem *)addr)
+#define intel_xpcie_ioread64(addr) \
+ _ioread64((void __iomem *)addr)
+#define intel_xpcie_ioread32(addr) \
+ ioread32((void __iomem *)addr)
+#define intel_xpcie_ioread16(addr) \
+ ioread16((void __iomem *)addr)
+#define intel_xpcie_ioread8(addr) \
+ ioread8((void __iomem *)addr)
+
+static inline
+void intel_xpcie_set_td_address(struct xpcie_transfer_desc *td, u64 address)
+{
+ intel_xpcie_iowrite64(address, &td->address);
+}
+
+static inline
+u64 intel_xpcie_get_td_address(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread64(&td->address);
+}
+
+static inline
+void intel_xpcie_set_td_length(struct xpcie_transfer_desc *td, u32 length)
+{
+ intel_xpcie_iowrite32(length, &td->length);
+}
+
+static inline
+u32 intel_xpcie_get_td_length(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread32(&td->length);
+}
+
+static inline
+void intel_xpcie_set_td_interface(struct xpcie_transfer_desc *td, u16 interface)
+{
+ intel_xpcie_iowrite16(interface, &td->interface);
+}
+
+static inline
+u16 intel_xpcie_get_td_interface(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread16(&td->interface);
+}
+
+static inline
+void intel_xpcie_set_td_status(struct xpcie_transfer_desc *td, u16 status)
+{
+ intel_xpcie_iowrite16(status, &td->status);
+}
+
+static inline
+u16 intel_xpcie_get_td_status(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread16(&td->status);
+}
+
+static inline
+void intel_xpcie_set_tdr_head(struct xpcie_pipe *p, u32 head)
+{
+ intel_xpcie_iowrite32(head, p->head);
+}
+
+static inline
+u32 intel_xpcie_get_tdr_head(struct xpcie_pipe *p)
+{
+ return intel_xpcie_ioread32(p->head);
+}
+
+static inline
+void intel_xpcie_set_tdr_tail(struct xpcie_pipe *p, u32 tail)
+{
+ intel_xpcie_iowrite32(tail, p->tail);
+}
+
+static inline
+u32 intel_xpcie_get_tdr_tail(struct xpcie_pipe *p)
+{
+ return intel_xpcie_ioread32(p->tail);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie);
+void intel_xpcie_core_cleanup(struct xpcie *xpcie);
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer, size_t *length,
+ u32 timeout_ms);
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer, size_t *length,
+ u32 timeout_ms);
+u32 intel_xpcie_get_device_num(u32 *id_list);
+struct xpcie_dev *intel_xpcie_get_device_by_id(u32 id);
+int intel_xpcie_get_device_name_by_id(u32 id, char *device_name,
+ size_t name_size);
+int intel_xpcie_get_device_status_by_id(u32 id, u32 *status);
+int intel_xpcie_pci_connect_device(u32 id);
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout);
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout);
+int intel_xpcie_pci_reset_device(u32 id);
+#endif /* XPCIE_CORE_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/epf.c b/drivers/misc/xlink-pcie/local_host/epf.c
index fc29180aa508..f334d4749cce 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.c
+++ b/drivers/misc/xlink-pcie/local_host/epf.c
@@ -9,6 +9,7 @@
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/reboot.h>
#include "epf.h"
@@ -21,6 +22,12 @@
#define PCIE_REGS_PCIE_ERR_INTR_FLAGS 0x24
#define LINK_REQ_RST_FLG BIT(15)
+#define PCIE_REGS_PCIE_SYS_CFG_CORE 0x7C
+#define PCIE_CFG_PBUS_NUM_OFFSET 8
+#define PCIE_CFG_PBUS_NUM_MASK 0xFF
+#define PCIE_CFG_PBUS_DEV_NUM_OFFSET 16
+#define PCIE_CFG_PBUS_DEV_NUM_MASK 0x1F
+
static struct pci_epf_header xpcie_header = {
.vendorid = PCI_VENDOR_ID_INTEL,
.deviceid = PCI_DEVICE_ID_INTEL_KEEMBAY,
@@ -37,6 +44,45 @@ static const struct pci_epf_device_id xpcie_epf_ids[] = {
{},
};
+u32 xlink_sw_id;
+
+int intel_xpcie_copy_from_host_ll(struct xpcie *xpcie, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ return intel_xpcie_ep_dma_read_ll(epf, chan, descs_num);
+}
+
+int intel_xpcie_copy_to_host_ll(struct xpcie *xpcie, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ return intel_xpcie_ep_dma_write_ll(epf, chan, descs_num);
+}
+
+void intel_xpcie_register_host_irq(struct xpcie *xpcie, irq_handler_t func)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+
+ xpcie_epf->core_irq_callback = func;
+}
+
+int intel_xpcie_raise_irq(struct xpcie *xpcie, enum xpcie_doorbell_type type)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, type, 1);
+
+ return pci_epc_raise_irq(epf->epc, epf->func_no, PCI_EPC_IRQ_MSI, 1);
+}
+
static irqreturn_t intel_xpcie_err_interrupt(int irq, void *args)
{
struct xpcie_epf *xpcie_epf;
@@ -57,6 +103,7 @@ static irqreturn_t intel_xpcie_host_interrupt(int irq, void *args)
{
struct xpcie_epf *xpcie_epf;
struct xpcie *xpcie = args;
+ u8 event;
u32 val;
xpcie_epf = container_of(xpcie, struct xpcie_epf, xpcie);
@@ -64,6 +111,18 @@ static irqreturn_t intel_xpcie_host_interrupt(int irq, void *args)
if (val & LBC_CII_EVENT_FLAG) {
iowrite32(LBC_CII_EVENT_FLAG,
xpcie_epf->apb_base + PCIE_REGS_PCIE_INTR_FLAGS);
+
+ event = intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DEV_EVENT);
+ if (unlikely(event != NO_OP)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE,
+ DEV_EVENT, NO_OP);
+ if (event == REQUEST_RESET)
+ orderly_reboot();
+ return IRQ_HANDLED;
+ }
+
+ if (likely(xpcie_epf->core_irq_callback))
+ xpcie_epf->core_irq_callback(irq, xpcie);
}
return IRQ_HANDLED;
@@ -269,6 +328,7 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
const struct pci_epc_features *features;
struct pci_epc *epc = epf->epc;
+ u32 bus_num, dev_num;
struct device *dev;
size_t align = SZ_16K;
int ret;
@@ -300,12 +360,12 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
if (!strcmp(xpcie_epf->stepping, "A0")) {
xpcie_epf->xpcie.legacy_a0 = true;
- iowrite32(1, (void __iomem *)xpcie_epf->xpcie.mmio +
- XPCIE_MMIO_LEGACY_A0);
+ intel_xpcie_iowrite32(1, xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
} else {
xpcie_epf->xpcie.legacy_a0 = false;
- iowrite32(0, (void __iomem *)xpcie_epf->xpcie.mmio +
- XPCIE_MMIO_LEGACY_A0);
+ intel_xpcie_iowrite32(0, xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
}
/* Enable interrupt */
@@ -330,13 +390,46 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
ret = intel_xpcie_ep_dma_init(epf);
if (ret) {
dev_err(&epf->dev, "DMA initialization failed\n");
- goto err_free_err_irq;
+ goto err_cleanup_bars;
}
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_READY);
+
+ ret = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_SYS_CFG_CORE);
+ bus_num = (ret >> PCIE_CFG_PBUS_NUM_OFFSET) & PCIE_CFG_PBUS_NUM_MASK;
+ dev_num = (ret >> PCIE_CFG_PBUS_DEV_NUM_OFFSET) &
+ PCIE_CFG_PBUS_DEV_NUM_MASK;
+
+ xlink_sw_id = FIELD_PREP(XLINK_DEV_INF_TYPE_MASK,
+ XLINK_DEV_INF_PCIE) |
+ FIELD_PREP(XLINK_DEV_PHYS_ID_MASK,
+ bus_num << 8 | dev_num) |
+ FIELD_PREP(XLINK_DEV_TYPE_MASK, XLINK_DEV_TYPE_KMB) |
+ FIELD_PREP(XLINK_DEV_PCIE_ID_MASK, XLINK_DEV_PCIE_0) |
+ FIELD_PREP(XLINK_DEV_FUNC_MASK, XLINK_DEV_FUNC_VPU);
+
+ ret = intel_xpcie_core_init(&xpcie_epf->xpcie);
+ if (ret) {
+ dev_err(&epf->dev, "Core component configuration failed\n");
+ goto err_uninit_dma;
+ }
+
+ intel_xpcie_iowrite32(XPCIE_STATUS_UNINIT,
+ xpcie_epf->xpcie.mmio + XPCIE_MMIO_HOST_STATUS);
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_RUN);
+ intel_xpcie_set_doorbell(&xpcie_epf->xpcie, FROM_DEVICE,
+ DEV_EVENT, NO_OP);
+ memcpy(xpcie_epf->xpcie.mmio + XPCIE_MMIO_MAGIC_OFF, XPCIE_MAGIC_YOCTO,
+ strlen(XPCIE_MAGIC_YOCTO));
+
return 0;
-err_free_err_irq:
- free_irq(xpcie_epf->irq_err, &xpcie_epf->xpcie);
+err_uninit_dma:
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_ERROR);
+ memcpy(xpcie_epf->xpcie.mmio + XPCIE_MMIO_MAGIC_OFF, XPCIE_MAGIC_YOCTO,
+ strlen(XPCIE_MAGIC_YOCTO));
+
+ intel_xpcie_ep_dma_uninit(epf);
err_cleanup_bars:
intel_xpcie_cleanup_bars(epf);
@@ -346,8 +439,12 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
static void intel_xpcie_epf_unbind(struct pci_epf *epf)
{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
struct pci_epc *epc = epf->epc;
+ intel_xpcie_core_cleanup(&xpcie_epf->xpcie);
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_READY);
+
intel_xpcie_ep_dma_uninit(epf);
pci_epc_stop(epc);
@@ -379,8 +476,11 @@ static void intel_xpcie_epf_shutdown(struct device *dev)
xpcie_epf = epf_get_drvdata(epf);
/* Notify host in case PCIe hot plug not supported */
- if (xpcie_epf)
+ if (xpcie_epf && xpcie_epf->xpcie.status == XPCIE_STATUS_RUN) {
+ intel_xpcie_set_doorbell(&xpcie_epf->xpcie, FROM_DEVICE,
+ DEV_EVENT, DEV_SHUTDOWN);
pci_epc_raise_irq(epf->epc, epf->func_no, PCI_EPC_IRQ_MSI, 1);
+ }
}
static struct pci_epf_ops ops = {
diff --git a/drivers/misc/xlink-pcie/local_host/epf.h b/drivers/misc/xlink-pcie/local_host/epf.h
index 6ce5260e67be..675c07455ffa 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.h
+++ b/drivers/misc/xlink-pcie/local_host/epf.h
@@ -14,6 +14,7 @@
#include <linux/pci-epf.h>
#include "xpcie.h"
+#include "util.h"
#define XPCIE_DRIVER_NAME "mxlk_pcie_epf"
#define XPCIE_DRIVER_DESC "Intel(R) xLink PCIe endpoint function driver"
@@ -26,6 +27,7 @@
#define XPCIE_NUM_RX_DESCS (64)
extern bool dma_ll_mode;
+extern u32 xlink_sw_id;
struct xpcie_dma_ll_desc {
u32 dma_ch_control1;
@@ -67,14 +69,38 @@ struct xpcie_epf {
void __iomem *dbi_base;
char stepping[KEEMBAY_XPCIE_STEPPING_MAXLEN];
+ irq_handler_t core_irq_callback;
+ dma_addr_t tx_phys;
+ void *tx_virt;
+ size_t tx_size;
+ dma_addr_t rx_phys;
+ void *rx_virt;
+ size_t rx_size;
+
struct xpcie_dma_ll_desc_buf tx_desc_buf[DMA_CHAN_NUM];
struct xpcie_dma_ll_desc_buf rx_desc_buf[DMA_CHAN_NUM];
};
+static inline struct device *xpcie_to_dev(struct xpcie *xpcie)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+
+ return &xpcie_epf->epf->dev;
+}
+
int intel_xpcie_ep_dma_init(struct pci_epf *epf);
int intel_xpcie_ep_dma_uninit(struct pci_epf *epf);
int intel_xpcie_ep_dma_reset(struct pci_epf *epf);
int intel_xpcie_ep_dma_read_ll(struct pci_epf *epf, int chan, int descs_num);
int intel_xpcie_ep_dma_write_ll(struct pci_epf *epf, int chan, int descs_num);
+void intel_xpcie_register_host_irq(struct xpcie *xpcie,
+ irq_handler_t func);
+int intel_xpcie_raise_irq(struct xpcie *xpcie,
+ enum xpcie_doorbell_type type);
+int intel_xpcie_copy_from_host_ll(struct xpcie *xpcie,
+ int chan, int descs_num);
+int intel_xpcie_copy_to_host_ll(struct xpcie *xpcie,
+ int chan, int descs_num);
#endif /* XPCIE_EPF_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/util.c b/drivers/misc/xlink-pcie/local_host/util.c
new file mode 100644
index 000000000000..ec808b0cd72b
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/util.c
@@ -0,0 +1,375 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include "util.h"
+
+void intel_xpcie_set_device_status(struct xpcie *xpcie, u32 status)
+{
+ xpcie->status = status;
+ intel_xpcie_iowrite32(status, xpcie->mmio + XPCIE_MMIO_DEV_STATUS);
+}
+
+u32 intel_xpcie_get_device_status(struct xpcie *xpcie)
+{
+ return intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_DEV_STATUS);
+}
+
+static size_t intel_xpcie_doorbell_offset(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type)
+{
+ if (dirt == TO_DEVICE && type == DATA_SENT)
+ return XPCIE_MMIO_HTOD_TX_DOORBELL;
+ if (dirt == TO_DEVICE && type == DATA_RECEIVED)
+ return XPCIE_MMIO_HTOD_RX_DOORBELL;
+ if (dirt == TO_DEVICE && type == DEV_EVENT)
+ return XPCIE_MMIO_HTOD_EVENT_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DATA_SENT)
+ return XPCIE_MMIO_DTOH_TX_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DATA_RECEIVED)
+ return XPCIE_MMIO_DTOH_RX_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DEV_EVENT)
+ return XPCIE_MMIO_DTOH_EVENT_DOORBELL;
+
+ return 0;
+}
+
+void intel_xpcie_set_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type, u8 value)
+{
+ size_t offset = intel_xpcie_doorbell_offset(xpcie, dirt, type);
+
+ intel_xpcie_iowrite8(value, xpcie->mmio + offset);
+}
+
+u8 intel_xpcie_get_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type)
+{
+ size_t offset = intel_xpcie_doorbell_offset(xpcie, dirt, type);
+
+ return intel_xpcie_ioread8(xpcie->mmio + offset);
+}
+
+u32 intel_xpcie_get_host_status(struct xpcie *xpcie)
+{
+ return intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_HOST_STATUS);
+}
+
+void intel_xpcie_set_host_status(struct xpcie *xpcie, u32 status)
+{
+ xpcie->status = status;
+ intel_xpcie_iowrite32(status, xpcie->mmio + XPCIE_MMIO_HOST_STATUS);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd(size_t length)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = kzalloc(sizeof(*bd), GFP_KERNEL);
+ if (!bd)
+ return NULL;
+
+ bd->head = kzalloc(roundup(length, cache_line_size()), GFP_KERNEL);
+ if (!bd->head) {
+ kfree(bd);
+ return NULL;
+ }
+
+ bd->data = bd->head;
+ bd->length = length;
+ bd->true_len = length;
+ bd->next = NULL;
+ bd->own_mem = true;
+
+ return bd;
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd_reuse(size_t length, void *virt,
+ dma_addr_t phys)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = kzalloc(sizeof(*bd), GFP_KERNEL);
+ if (!bd)
+ return NULL;
+
+ bd->head = virt;
+ bd->phys = phys;
+ bd->data = bd->head;
+ bd->length = length;
+ bd->true_len = length;
+ bd->next = NULL;
+ bd->own_mem = false;
+
+ return bd;
+}
+
+void intel_xpcie_free_bd(struct xpcie_buf_desc *bd)
+{
+ if (bd) {
+ if (bd->own_mem)
+ kfree(bd->head);
+ kfree(bd);
+ }
+}
+
+int intel_xpcie_list_init(struct xpcie_list *list)
+{
+ spin_lock_init(&list->lock);
+ list->bytes = 0;
+ list->buffers = 0;
+ list->head = NULL;
+ list->tail = NULL;
+
+ return 0;
+}
+
+void intel_xpcie_list_cleanup(struct xpcie_list *list)
+{
+ struct xpcie_buf_desc *bd;
+
+ spin_lock(&list->lock);
+ while (list->head) {
+ bd = list->head;
+ list->head = bd->next;
+ intel_xpcie_free_bd(bd);
+ }
+
+ list->head = NULL;
+ list->tail = NULL;
+ spin_unlock(&list->lock);
+}
+
+int intel_xpcie_list_put(struct xpcie_list *list, struct xpcie_buf_desc *bd)
+{
+ if (!bd)
+ return -EINVAL;
+
+ spin_lock(&list->lock);
+ if (list->head)
+ list->tail->next = bd;
+ else
+ list->head = bd;
+
+ while (bd) {
+ list->tail = bd;
+ list->bytes += bd->length;
+ list->buffers++;
+ bd = bd->next;
+ }
+ spin_unlock(&list->lock);
+
+ return 0;
+}
+
+int intel_xpcie_list_put_head(struct xpcie_list *list,
+ struct xpcie_buf_desc *bd)
+{
+ struct xpcie_buf_desc *old_head;
+
+ if (!bd)
+ return -EINVAL;
+
+ spin_lock(&list->lock);
+ old_head = list->head;
+ list->head = bd;
+ while (bd) {
+ list->bytes += bd->length;
+ list->buffers++;
+ if (!bd->next) {
+ list->tail = list->tail ? list->tail : bd;
+ bd->next = old_head;
+ break;
+ }
+ bd = bd->next;
+ }
+ spin_unlock(&list->lock);
+
+ return 0;
+}
+
+struct xpcie_buf_desc *intel_xpcie_list_get(struct xpcie_list *list)
+{
+ struct xpcie_buf_desc *bd;
+
+ spin_lock(&list->lock);
+ bd = list->head;
+ if (list->head) {
+ list->head = list->head->next;
+ if (!list->head)
+ list->tail = NULL;
+ bd->next = NULL;
+ list->bytes -= bd->length;
+ list->buffers--;
+ }
+ spin_unlock(&list->lock);
+
+ return bd;
+}
+
+void intel_xpcie_list_info(struct xpcie_list *list,
+ size_t *bytes, size_t *buffers)
+{
+ spin_lock(&list->lock);
+ *bytes = list->bytes;
+ *buffers = list->buffers;
+ spin_unlock(&list->lock);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_rx_bd(struct xpcie *xpcie)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = intel_xpcie_list_get(&xpcie->rx_pool);
+ if (bd) {
+ bd->data = bd->head;
+ bd->length = bd->true_len;
+ bd->next = NULL;
+ bd->interface = 0;
+ }
+
+ return bd;
+}
+
+void intel_xpcie_free_rx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd)
+{
+ if (bd)
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_tx_bd(struct xpcie *xpcie)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = intel_xpcie_list_get(&xpcie->tx_pool);
+ if (bd) {
+ bd->data = bd->head;
+ bd->length = bd->true_len;
+ bd->next = NULL;
+ bd->interface = 0;
+ } else {
+ xpcie->no_tx_buffer = true;
+ }
+
+ return bd;
+}
+
+void intel_xpcie_free_tx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd)
+{
+ if (!bd)
+ return;
+
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+
+ xpcie->no_tx_buffer = false;
+ wake_up_interruptible(&xpcie->tx_waitq);
+}
+
+int intel_xpcie_interface_init(struct xpcie *xpcie, int id)
+{
+ struct xpcie_interface *inf = xpcie->interfaces + id;
+
+ inf->id = id;
+ inf->xpcie = xpcie;
+
+ inf->partial_read = NULL;
+ intel_xpcie_list_init(&inf->read);
+ mutex_init(&inf->rlock);
+ inf->data_avail = false;
+ init_waitqueue_head(&inf->rx_waitq);
+
+ return 0;
+}
+
+void intel_xpcie_interface_cleanup(struct xpcie_interface *inf)
+{
+ struct xpcie_buf_desc *bd;
+
+ intel_xpcie_free_rx_bd(inf->xpcie, inf->partial_read);
+ while ((bd = intel_xpcie_list_get(&inf->read)))
+ intel_xpcie_free_rx_bd(inf->xpcie, bd);
+
+ mutex_destroy(&inf->rlock);
+}
+
+void intel_xpcie_interfaces_cleanup(struct xpcie *xpcie)
+{
+ int index;
+
+ for (index = 0; index < XPCIE_NUM_INTERFACES; index++)
+ intel_xpcie_interface_cleanup(xpcie->interfaces + index);
+
+ intel_xpcie_list_cleanup(&xpcie->write);
+ mutex_destroy(&xpcie->wlock);
+}
+
+int intel_xpcie_interfaces_init(struct xpcie *xpcie)
+{
+ int index;
+
+ mutex_init(&xpcie->wlock);
+ intel_xpcie_list_init(&xpcie->write);
+ init_waitqueue_head(&xpcie->tx_waitq);
+ xpcie->no_tx_buffer = false;
+
+ for (index = 0; index < XPCIE_NUM_INTERFACES; index++)
+ intel_xpcie_interface_init(xpcie, index);
+
+ return 0;
+}
+
+void intel_xpcie_add_bd_to_interface(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd)
+{
+ struct xpcie_interface *inf;
+
+ inf = xpcie->interfaces + bd->interface;
+
+ intel_xpcie_list_put(&inf->read, bd);
+
+ mutex_lock(&inf->rlock);
+ inf->data_avail = true;
+ mutex_unlock(&inf->rlock);
+ wake_up_interruptible(&inf->rx_waitq);
+}
+
+void *intel_xpcie_cap_find(struct xpcie *xpcie, u32 start, u16 id)
+{
+ int ttl = XPCIE_CAP_TTL;
+ void *hdr;
+ u16 id_out, next;
+
+ /* If user didn't specify start, assume start of mmio */
+ if (!start)
+ start = intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_CAP_OFF);
+
+ /* Read header info */
+ hdr = xpcie->mmio + start;
+
+ /* Check if we still have time to live */
+ while (ttl--) {
+ id_out = intel_xpcie_ioread16(hdr + XPCIE_CAP_HDR_ID);
+ next = intel_xpcie_ioread16(hdr + XPCIE_CAP_HDR_NEXT);
+
+ /* If cap matches, return header */
+ if (id_out == id)
+ return hdr;
+ /* If cap is NULL, we are at the end of the list */
+ else if (id_out == XPCIE_CAP_NULL)
+ return NULL;
+ /* If no match and no end of list, traverse the linked list */
+ else
+ hdr = xpcie->mmio + next;
+ }
+
+ /* If we reached here, the capability list is corrupted */
+ return NULL;
+}
diff --git a/drivers/misc/xlink-pcie/local_host/util.h b/drivers/misc/xlink-pcie/local_host/util.h
new file mode 100644
index 000000000000..908be897a61d
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/util.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_UTIL_HEADER_
+#define XPCIE_UTIL_HEADER_
+
+#include "xpcie.h"
+
+enum xpcie_doorbell_direction {
+ TO_DEVICE,
+ FROM_DEVICE
+};
+
+enum xpcie_doorbell_type {
+ DATA_SENT,
+ DATA_RECEIVED,
+ DEV_EVENT
+};
+
+enum xpcie_event_type {
+ NO_OP,
+ REQUEST_RESET,
+ DEV_SHUTDOWN
+};
+
+void intel_xpcie_set_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type, u8 value);
+u8 intel_xpcie_get_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type);
+
+void intel_xpcie_set_device_status(struct xpcie *xpcie, u32 status);
+u32 intel_xpcie_get_device_status(struct xpcie *xpcie);
+u32 intel_xpcie_get_host_status(struct xpcie *xpcie);
+void intel_xpcie_set_host_status(struct xpcie *xpcie, u32 status);
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd(size_t length);
+struct xpcie_buf_desc *intel_xpcie_alloc_bd_reuse(size_t length, void *virt,
+ dma_addr_t phys);
+void intel_xpcie_free_bd(struct xpcie_buf_desc *bd);
+
+int intel_xpcie_list_init(struct xpcie_list *list);
+void intel_xpcie_list_cleanup(struct xpcie_list *list);
+int intel_xpcie_list_put(struct xpcie_list *list, struct xpcie_buf_desc *bd);
+int intel_xpcie_list_put_head(struct xpcie_list *list,
+ struct xpcie_buf_desc *bd);
+struct xpcie_buf_desc *intel_xpcie_list_get(struct xpcie_list *list);
+void intel_xpcie_list_info(struct xpcie_list *list, size_t *bytes,
+ size_t *buffers);
+
+struct xpcie_buf_desc *intel_xpcie_alloc_rx_bd(struct xpcie *xpcie);
+void intel_xpcie_free_rx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd);
+struct xpcie_buf_desc *intel_xpcie_alloc_tx_bd(struct xpcie *xpcie);
+void intel_xpcie_free_tx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd);
+
+int intel_xpcie_interface_init(struct xpcie *xpcie, int id);
+void intel_xpcie_interface_cleanup(struct xpcie_interface *inf);
+void intel_xpcie_interfaces_cleanup(struct xpcie *xpcie);
+int intel_xpcie_interfaces_init(struct xpcie *xpcie);
+void intel_xpcie_add_bd_to_interface(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd);
+void *intel_xpcie_cap_find(struct xpcie *xpcie, u32 start, u16 id);
+#endif /* XPCIE_UTIL_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/xpcie.h b/drivers/misc/xlink-pcie/local_host/xpcie.h
index 932ec5d5c718..5ae0b3dbd503 100644
--- a/drivers/misc/xlink-pcie/local_host/xpcie.h
+++ b/drivers/misc/xlink-pcie/local_host/xpcie.h
@@ -14,6 +14,8 @@
#include <linux/module.h>
#include <linux/pci_ids.h>
+#include "core.h"
+
#ifndef PCI_DEVICE_ID_INTEL_KEEMBAY
#define PCI_DEVICE_ID_INTEL_KEEMBAY 0x6240
#endif
@@ -35,20 +37,83 @@ struct xpcie_version {
u16 build;
} __packed;
+/* Status encoding of both device and host */
+#define XPCIE_STATUS_ERROR (0xFFFFFFFF)
+#define XPCIE_STATUS_UNINIT (0)
+#define XPCIE_STATUS_READY (1)
+#define XPCIE_STATUS_RECOVERY (2)
+#define XPCIE_STATUS_OFF (3)
+#define XPCIE_STATUS_RUN (4)
+
+#define XPCIE_MAGIC_STRLEN (16)
+#define XPCIE_MAGIC_YOCTO "VPUYOCTO"
+
/* MMIO layout and offsets shared between device and host */
struct xpcie_mmio {
struct xpcie_version version;
+ u32 device_status;
+ u32 host_status;
u8 legacy_a0;
+ u8 htod_tx_doorbell;
+ u8 htod_rx_doorbell;
+ u8 htod_event_doorbell;
+ u8 dtoh_tx_doorbell;
+ u8 dtoh_rx_doorbell;
+ u8 dtoh_event_doorbell;
+ u8 reserved;
+ u32 cap_offset;
+ u8 magic[XPCIE_MAGIC_STRLEN];
} __packed;
#define XPCIE_MMIO_VERSION (offsetof(struct xpcie_mmio, version))
#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_DEV_STATUS (offsetof(struct xpcie_mmio, device_status))
+#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_HOST_STATUS (offsetof(struct xpcie_mmio, host_status))
+#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_HTOD_TX_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_tx_doorbell))
+#define XPCIE_MMIO_HTOD_RX_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_rx_doorbell))
+#define XPCIE_MMIO_HTOD_EVENT_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_event_doorbell))
+#define XPCIE_MMIO_DTOH_TX_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_tx_doorbell))
+#define XPCIE_MMIO_DTOH_RX_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_rx_doorbell))
+#define XPCIE_MMIO_DTOH_EVENT_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_event_doorbell))
+#define XPCIE_MMIO_CAP_OFF (offsetof(struct xpcie_mmio, cap_offset))
+#define XPCIE_MMIO_MAGIC_OFF (offsetof(struct xpcie_mmio, magic))
struct xpcie {
u32 status;
bool legacy_a0;
void *mmio;
void *bar4;
+
+ struct workqueue_struct *rx_wq;
+ struct workqueue_struct *tx_wq;
+
+ struct xpcie_interface interfaces[XPCIE_NUM_INTERFACES];
+
+ size_t fragment_size;
+ struct xpcie_cap_txrx *txrx;
+ struct xpcie_stream tx;
+ struct xpcie_stream rx;
+
+ struct mutex wlock; /* write lock */
+ struct xpcie_list write;
+ bool no_tx_buffer;
+ wait_queue_head_t tx_waitq;
+ bool tx_pending;
+ bool stop_flag;
+
+ struct xpcie_list rx_pool;
+ struct xpcie_list tx_pool;
+
+ struct delayed_work rx_event;
+ struct delayed_work tx_event;
};
#endif /* XPCIE_HEADER_ */
diff --git a/include/linux/xlink_drv_inf.h b/include/linux/xlink_drv_inf.h
new file mode 100644
index 000000000000..ffe8f4c253e6
--- /dev/null
+++ b/include/linux/xlink_drv_inf.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef _XLINK_DRV_INF_H_
+#define _XLINK_DRV_INF_H_
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/types.h>
+
+#define XLINK_DEV_INF_TYPE_MASK GENMASK(27, 24)
+#define XLINK_DEV_PHYS_ID_MASK GENMASK(23, 8)
+#define XLINK_DEV_TYPE_MASK GENMASK(6, 4)
+#define XLINK_DEV_PCIE_ID_MASK GENMASK(3, 1)
+#define XLINK_DEV_FUNC_MASK GENMASK(0, 0)
+
+enum xlink_device_inf_type {
+ XLINK_DEV_INF_PCIE = 1,
+};
+
+enum xlink_device_type {
+ XLINK_DEV_TYPE_KMB = 0,
+};
+
+enum xlink_device_pcie {
+ XLINK_DEV_PCIE_0 = 0,
+};
+
+enum xlink_device_func {
+ XLINK_DEV_FUNC_VPU = 0,
+};
+
+enum _xlink_device_status {
+ _XLINK_DEV_OFF,
+ _XLINK_DEV_ERROR,
+ _XLINK_DEV_BUSY,
+ _XLINK_DEV_RECOVERY,
+ _XLINK_DEV_READY
+};
+
+int xlink_pcie_get_device_list(u32 *sw_device_id_list,
+ u32 *num_devices);
+int xlink_pcie_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size);
+int xlink_pcie_get_device_status(u32 sw_device_id,
+ u32 *device_status);
+int xlink_pcie_boot_device(u32 sw_device_id, const char *binary_name);
+int xlink_pcie_connect(u32 sw_device_id);
+int xlink_pcie_read(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout);
+int xlink_pcie_write(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout);
+int xlink_pcie_reset_device(u32 sw_device_id);
+#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic
2020-11-30 23:06 ` [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic mgross
@ 2020-12-01 10:15 ` Greg Kroah-Hartman
0 siblings, 0 replies; 34+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-01 10:15 UTC (permalink / raw)
To: mgross
Cc: linux-kernel, markgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann
On Mon, Nov 30, 2020 at 03:06:54PM -0800, mgross@linux.intel.com wrote:
> +static int rx_pool_size = SZ_32M;
> +module_param(rx_pool_size, int, 0664);
> +MODULE_PARM_DESC(rx_pool_size, "receiving pool size (default 32 MiB)");
> +
> +static int tx_pool_size = SZ_32M;
> +module_param(tx_pool_size, int, 0664);
> +MODULE_PARM_DESC(tx_pool_size, "transmitting pool size (default 32 MiB)");
> +
> +static int fragment_size = XPCIE_FRAGMENT_SIZE;
> +module_param(fragment_size, int, 0664);
> +MODULE_PARM_DESC(fragment_size, "transfer descriptor size (default 128 KiB)");
> +
> +static bool tx_pool_coherent = true;
> +module_param(tx_pool_coherent, bool, 0664);
> +MODULE_PARM_DESC(tx_pool_coherent,
> + "transmitting pool using coherent memory (default true)");
> +
> +static bool rx_pool_coherent;
> +module_param(rx_pool_coherent, bool, 0664);
> +MODULE_PARM_DESC(rx_pool_coherent,
> + "receiving pool using coherent memory (default false)");
This is not the 1990's anymore, please never use module parameters for
new drivers as they do not work well (i.e. they do not work at all on a
individual device basis.)
Make this all "just work" so that userspace does not have to "tune"
anything. But if you really must allow configuration like this, then
use configfs, as that's what it is there for.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 10/22] misc: xlink-pcie: lh: Prepare changes for adding remote host driver
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (8 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 11/22] misc: xlink-pcie: rh: Add PCIe EP driver for Remote Host mgross
` (12 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Move logic that can be reused between local host and remote host to
common/ folder
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/{local_host => common}/core.h | 0
drivers/misc/xlink-pcie/{local_host => common}/util.c | 0
drivers/misc/xlink-pcie/{local_host => common}/util.h | 0
drivers/misc/xlink-pcie/{local_host => common}/xpcie.h | 0
drivers/misc/xlink-pcie/local_host/Makefile | 2 +-
drivers/misc/xlink-pcie/local_host/core.c | 4 ++--
drivers/misc/xlink-pcie/local_host/epf.h | 4 ++--
7 files changed, 5 insertions(+), 5 deletions(-)
rename drivers/misc/xlink-pcie/{local_host => common}/core.h (100%)
rename drivers/misc/xlink-pcie/{local_host => common}/util.c (100%)
rename drivers/misc/xlink-pcie/{local_host => common}/util.h (100%)
rename drivers/misc/xlink-pcie/{local_host => common}/xpcie.h (100%)
diff --git a/drivers/misc/xlink-pcie/local_host/core.h b/drivers/misc/xlink-pcie/common/core.h
similarity index 100%
rename from drivers/misc/xlink-pcie/local_host/core.h
rename to drivers/misc/xlink-pcie/common/core.h
diff --git a/drivers/misc/xlink-pcie/local_host/util.c b/drivers/misc/xlink-pcie/common/util.c
similarity index 100%
rename from drivers/misc/xlink-pcie/local_host/util.c
rename to drivers/misc/xlink-pcie/common/util.c
diff --git a/drivers/misc/xlink-pcie/local_host/util.h b/drivers/misc/xlink-pcie/common/util.h
similarity index 100%
rename from drivers/misc/xlink-pcie/local_host/util.h
rename to drivers/misc/xlink-pcie/common/util.h
diff --git a/drivers/misc/xlink-pcie/local_host/xpcie.h b/drivers/misc/xlink-pcie/common/xpcie.h
similarity index 100%
rename from drivers/misc/xlink-pcie/local_host/xpcie.h
rename to drivers/misc/xlink-pcie/common/xpcie.h
diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
index 28761751d43b..65df94c7e860 100644
--- a/drivers/misc/xlink-pcie/local_host/Makefile
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -2,4 +2,4 @@ obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
mxlk_ep-objs := epf.o
mxlk_ep-objs += dma.o
mxlk_ep-objs += core.o
-mxlk_ep-objs += util.o
+mxlk_ep-objs += ../common/util.o
diff --git a/drivers/misc/xlink-pcie/local_host/core.c b/drivers/misc/xlink-pcie/local_host/core.c
index aecaaa783153..0f83b5ffc4f2 100644
--- a/drivers/misc/xlink-pcie/local_host/core.c
+++ b/drivers/misc/xlink-pcie/local_host/core.c
@@ -10,8 +10,8 @@
#include <linux/of_reserved_mem.h>
#include "epf.h"
-#include "core.h"
-#include "util.h"
+#include "../common/core.h"
+#include "../common/util.h"
static struct xpcie *global_xpcie;
diff --git a/drivers/misc/xlink-pcie/local_host/epf.h b/drivers/misc/xlink-pcie/local_host/epf.h
index 675c07455ffa..9cfd96923c6f 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.h
+++ b/drivers/misc/xlink-pcie/local_host/epf.h
@@ -13,8 +13,8 @@
#include <linux/pci-epc.h>
#include <linux/pci-epf.h>
-#include "xpcie.h"
-#include "util.h"
+#include "../common/xpcie.h"
+#include "../common/util.h"
#define XPCIE_DRIVER_NAME "mxlk_pcie_epf"
#define XPCIE_DRIVER_DESC "Intel(R) xLink PCIe endpoint function driver"
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 11/22] misc: xlink-pcie: rh: Add PCIe EP driver for Remote Host
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (9 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 10/22] misc: xlink-pcie: lh: Prepare changes for adding remote host driver mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 12/22] misc: xlink-pcie: rh: Add core communication logic mgross
` (11 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add PCIe Endpoint driver that configures PCIe BARs and MSIs on the
Remote Host
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
MAINTAINERS | 2 +-
drivers/misc/xlink-pcie/Kconfig | 11 +
drivers/misc/xlink-pcie/Makefile | 1 +
drivers/misc/xlink-pcie/common/xpcie.h | 1 +
drivers/misc/xlink-pcie/remote_host/Makefile | 3 +
drivers/misc/xlink-pcie/remote_host/main.c | 93 ++++
drivers/misc/xlink-pcie/remote_host/pci.c | 451 +++++++++++++++++++
drivers/misc/xlink-pcie/remote_host/pci.h | 64 +++
8 files changed, 625 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/xlink-pcie/remote_host/Makefile
create mode 100644 drivers/misc/xlink-pcie/remote_host/main.c
create mode 100644 drivers/misc/xlink-pcie/remote_host/pci.c
create mode 100644 drivers/misc/xlink-pcie/remote_host/pci.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 9af8685967ac..b3468dea6557 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1952,7 +1952,7 @@ F: Documentation/devicetree/bindings/arm/intel,keembay.yaml
F: arch/arm64/boot/dts/intel/keembay-evm.dts
F: arch/arm64/boot/dts/intel/keembay-soc.dtsi
-ARM KEEMBAY XLINK PCIE SUPPORT
+ARM/INTEL KEEMBAY XLINK PCIE SUPPORT
M: Srikanth Thokala <srikanth.thokala@intel.com>
M: Mark Gross <mgross@linux.intel.com>
S: Maintained
diff --git a/drivers/misc/xlink-pcie/Kconfig b/drivers/misc/xlink-pcie/Kconfig
index 46aa401d79b7..448b9bfbdfa2 100644
--- a/drivers/misc/xlink-pcie/Kconfig
+++ b/drivers/misc/xlink-pcie/Kconfig
@@ -1,3 +1,14 @@
+config XLINK_PCIE_RH_DRIVER
+ tristate "XLink PCIe Remote Host driver"
+ depends on PCI && X86_64
+ help
+ This option enables XLink PCIe Remote Host driver.
+
+ Choose M here to compile this driver as a module, name is mxlk.
+ This driver is used for XLink communication over PCIe,
+ and is to be loaded on the IA host which is connected to
+ the Intel Keem Bay.
+
config XLINK_PCIE_LH_DRIVER
tristate "XLink PCIe Local Host driver"
depends on PCI_ENDPOINT && ARCH_KEEMBAY
diff --git a/drivers/misc/xlink-pcie/Makefile b/drivers/misc/xlink-pcie/Makefile
index d693d382e9c6..1dd984d8d88c 100644
--- a/drivers/misc/xlink-pcie/Makefile
+++ b/drivers/misc/xlink-pcie/Makefile
@@ -1 +1,2 @@
+obj-$(CONFIG_XLINK_PCIE_RH_DRIVER) += remote_host/
obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += local_host/
diff --git a/drivers/misc/xlink-pcie/common/xpcie.h b/drivers/misc/xlink-pcie/common/xpcie.h
index 5ae0b3dbd503..c62443d61865 100644
--- a/drivers/misc/xlink-pcie/common/xpcie.h
+++ b/drivers/misc/xlink-pcie/common/xpcie.h
@@ -89,6 +89,7 @@ struct xpcie_mmio {
struct xpcie {
u32 status;
bool legacy_a0;
+ void *bar0;
void *mmio;
void *bar4;
diff --git a/drivers/misc/xlink-pcie/remote_host/Makefile b/drivers/misc/xlink-pcie/remote_host/Makefile
new file mode 100644
index 000000000000..96374a43023e
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_XLINK_PCIE_RH_DRIVER) += mxlk.o
+mxlk-objs := main.o
+mxlk-objs += pci.o
diff --git a/drivers/misc/xlink-pcie/remote_host/main.c b/drivers/misc/xlink-pcie/remote_host/main.c
new file mode 100644
index 000000000000..810da1509418
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/main.c
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include "pci.h"
+#include "../common/core.h"
+
+#define HW_ID_LO_MASK GENMASK(7, 0)
+#define HW_ID_HI_MASK GENMASK(15, 8)
+
+static const struct pci_device_id xpcie_pci_table[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_KEEMBAY), 0 },
+ { 0 }
+};
+
+static int intel_xpcie_probe(struct pci_dev *pdev,
+ const struct pci_device_id *ent)
+{
+ bool new_device = false;
+ struct xpcie_dev *xdev;
+ u32 sw_devid;
+ u16 hw_id;
+ int ret;
+
+ hw_id = FIELD_PREP(HW_ID_HI_MASK, pdev->bus->number) |
+ FIELD_PREP(HW_ID_LO_MASK, PCI_SLOT(pdev->devfn));
+
+ sw_devid = FIELD_PREP(XLINK_DEV_INF_TYPE_MASK,
+ XLINK_DEV_INF_PCIE) |
+ FIELD_PREP(XLINK_DEV_PHYS_ID_MASK, hw_id) |
+ FIELD_PREP(XLINK_DEV_TYPE_MASK, XLINK_DEV_TYPE_KMB) |
+ FIELD_PREP(XLINK_DEV_PCIE_ID_MASK, XLINK_DEV_PCIE_0) |
+ FIELD_PREP(XLINK_DEV_FUNC_MASK, XLINK_DEV_FUNC_VPU);
+
+ xdev = intel_xpcie_get_device_by_id(sw_devid);
+ if (!xdev) {
+ xdev = intel_xpcie_create_device(sw_devid, pdev);
+ if (!xdev)
+ return -ENOMEM;
+
+ new_device = true;
+ }
+
+ ret = intel_xpcie_pci_init(xdev, pdev);
+ if (ret) {
+ intel_xpcie_remove_device(xdev);
+ return ret;
+ }
+
+ if (new_device)
+ intel_xpcie_list_add_device(xdev);
+
+ return ret;
+}
+
+static void intel_xpcie_remove(struct pci_dev *pdev)
+{
+ struct xpcie_dev *xdev = pci_get_drvdata(pdev);
+
+ if (xdev) {
+ intel_xpcie_pci_cleanup(xdev);
+ intel_xpcie_remove_device(xdev);
+ }
+}
+
+static struct pci_driver xpcie_driver = {
+ .name = XPCIE_DRIVER_NAME,
+ .id_table = xpcie_pci_table,
+ .probe = intel_xpcie_probe,
+ .remove = intel_xpcie_remove
+};
+
+static int __init intel_xpcie_init_module(void)
+{
+ return pci_register_driver(&xpcie_driver);
+}
+
+static void __exit intel_xpcie_exit_module(void)
+{
+ pci_unregister_driver(&xpcie_driver);
+}
+
+module_init(intel_xpcie_init_module);
+module_exit(intel_xpcie_exit_module);
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION(XPCIE_DRIVER_DESC);
+MODULE_VERSION(XPCIE_DRIVER_VERSION);
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.c b/drivers/misc/xlink-pcie/remote_host/pci.c
new file mode 100644
index 000000000000..0ca0755b591f
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/pci.c
@@ -0,0 +1,451 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+
+#include "pci.h"
+
+#include "../common/core.h"
+#include "../common/util.h"
+
+static int aspm_enable;
+module_param(aspm_enable, int, 0664);
+MODULE_PARM_DESC(aspm_enable, "enable ASPM");
+
+static LIST_HEAD(dev_list);
+static DEFINE_MUTEX(dev_list_mutex);
+
+struct xpcie_dev *intel_xpcie_get_device_by_id(u32 id)
+{
+ struct xpcie_dev *xdev;
+
+ mutex_lock(&dev_list_mutex);
+
+ if (list_empty(&dev_list)) {
+ mutex_unlock(&dev_list_mutex);
+ return NULL;
+ }
+
+ list_for_each_entry(xdev, &dev_list, list) {
+ if (xdev->devid == id) {
+ mutex_unlock(&dev_list_mutex);
+ return xdev;
+ }
+ }
+
+ mutex_unlock(&dev_list_mutex);
+
+ return NULL;
+}
+
+struct xpcie_dev *intel_xpcie_create_device(u32 sw_device_id,
+ struct pci_dev *pdev)
+{
+ struct xpcie_dev *xdev = kzalloc(sizeof(*xdev), GFP_KERNEL);
+
+ if (!xdev)
+ return NULL;
+
+ xdev->devid = sw_device_id;
+ snprintf(xdev->name, XPCIE_MAX_NAME_LEN, "%02x:%02x.%x",
+ pdev->bus->number,
+ PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+
+ mutex_init(&xdev->lock);
+
+ return xdev;
+}
+
+void intel_xpcie_remove_device(struct xpcie_dev *xdev)
+{
+ mutex_destroy(&xdev->lock);
+ kfree(xdev);
+}
+
+void intel_xpcie_list_add_device(struct xpcie_dev *xdev)
+{
+ mutex_lock(&dev_list_mutex);
+
+ list_add_tail(&xdev->list, &dev_list);
+
+ mutex_unlock(&dev_list_mutex);
+}
+
+void intel_xpcie_list_del_device(struct xpcie_dev *xdev)
+{
+ mutex_lock(&dev_list_mutex);
+
+ list_del(&xdev->list);
+
+ mutex_unlock(&dev_list_mutex);
+}
+
+static void intel_xpcie_pci_set_aspm(struct xpcie_dev *xdev, int aspm)
+{
+ u16 link_control;
+ u8 cap_exp;
+
+ cap_exp = pci_find_capability(xdev->pci, PCI_CAP_ID_EXP);
+ if (!cap_exp) {
+ dev_err(&xdev->pci->dev, "failed to find pcie capability\n");
+ return;
+ }
+
+ pci_read_config_word(xdev->pci, cap_exp + PCI_EXP_LNKCTL,
+ &link_control);
+ link_control &= ~(PCI_EXP_LNKCTL_ASPMC);
+ link_control |= (aspm & PCI_EXP_LNKCTL_ASPMC);
+ pci_write_config_word(xdev->pci, cap_exp + PCI_EXP_LNKCTL,
+ link_control);
+}
+
+static void intel_xpcie_pci_unmap_bar(struct xpcie_dev *xdev)
+{
+ if (xdev->xpcie.bar0) {
+ iounmap((void __iomem *)xdev->xpcie.bar0);
+ xdev->xpcie.bar0 = NULL;
+ }
+
+ if (xdev->xpcie.mmio) {
+ iounmap((void __iomem *)(xdev->xpcie.mmio - XPCIE_MMIO_OFFSET));
+ xdev->xpcie.mmio = NULL;
+ }
+
+ if (xdev->xpcie.bar4) {
+ iounmap((void __iomem *)xdev->xpcie.bar4);
+ xdev->xpcie.bar4 = NULL;
+ }
+}
+
+static int intel_xpcie_pci_map_bar(struct xpcie_dev *xdev)
+{
+ if (pci_resource_len(xdev->pci, 2) < XPCIE_IO_COMM_SIZE) {
+ dev_err(&xdev->pci->dev, "device BAR region is too small\n");
+ return -EIO;
+ }
+
+ xdev->xpcie.bar0 = (void __force *)pci_ioremap_bar(xdev->pci, 0);
+ if (!xdev->xpcie.bar0) {
+ dev_err(&xdev->pci->dev, "failed to ioremap BAR0\n");
+ goto bar_error;
+ }
+
+ xdev->xpcie.mmio = (void __force *)
+ (pci_ioremap_bar(xdev->pci, 2) + XPCIE_MMIO_OFFSET);
+ if (!xdev->xpcie.mmio) {
+ dev_err(&xdev->pci->dev, "failed to ioremap BAR2\n");
+ goto bar_error;
+ }
+
+ xdev->xpcie.bar4 = (void __force *)pci_ioremap_wc_bar(xdev->pci, 4);
+ if (!xdev->xpcie.bar4) {
+ dev_err(&xdev->pci->dev, "failed to ioremap BAR4\n");
+ goto bar_error;
+ }
+
+ return 0;
+
+bar_error:
+ intel_xpcie_pci_unmap_bar(xdev);
+ return -EIO;
+}
+
+static void intel_xpcie_pci_irq_cleanup(struct xpcie_dev *xdev)
+{
+ int irq = pci_irq_vector(xdev->pci, 0);
+
+ if (irq < 0)
+ return;
+
+ synchronize_irq(irq);
+ free_irq(irq, xdev);
+ pci_free_irq_vectors(xdev->pci);
+}
+
+static int intel_xpcie_pci_irq_init(struct xpcie_dev *xdev,
+ irq_handler_t irq_handler)
+{
+ int rc, irq;
+
+ rc = pci_alloc_irq_vectors(xdev->pci, 1, 1, PCI_IRQ_MSI);
+ if (rc < 0) {
+ dev_err(&xdev->pci->dev,
+ "failed to allocate %d MSI vectors\n", 1);
+ return rc;
+ }
+
+ irq = pci_irq_vector(xdev->pci, 0);
+ if (irq < 0) {
+ dev_err(&xdev->pci->dev, "failed to get irq\n");
+ rc = irq;
+ goto error_irq;
+ }
+ rc = request_irq(irq, irq_handler, 0,
+ XPCIE_DRIVER_NAME, xdev);
+ if (rc) {
+ dev_err(&xdev->pci->dev, "failed to request irq\n");
+ goto error_irq;
+ }
+
+ return 0;
+
+error_irq:
+ pci_free_irq_vectors(xdev->pci);
+ return rc;
+}
+
+static void xpcie_device_poll(struct work_struct *work)
+{
+ struct xpcie_dev *xdev = container_of(work, struct xpcie_dev,
+ wait_event.work);
+ u32 dev_status = intel_xpcie_ioread32(xdev->xpcie.mmio +
+ XPCIE_MMIO_DEV_STATUS);
+
+ if (dev_status < XPCIE_STATUS_RUN)
+ schedule_delayed_work(&xdev->wait_event,
+ msecs_to_jiffies(100));
+ else
+ xdev->xpcie.status = XPCIE_STATUS_READY;
+}
+
+static int intel_xpcie_pci_prepare_dev_reset(struct xpcie_dev *xdev,
+ bool notify)
+{
+ if (mutex_lock_interruptible(&xdev->lock))
+ return -EINTR;
+
+ if (xdev->core_irq_callback)
+ xdev->core_irq_callback = NULL;
+
+ xdev->xpcie.status = XPCIE_STATUS_OFF;
+ if (notify)
+ intel_xpcie_pci_raise_irq(xdev, DEV_EVENT, REQUEST_RESET);
+
+ mutex_unlock(&xdev->lock);
+
+ return 0;
+}
+
+static void xpcie_device_shutdown(struct work_struct *work)
+{
+ struct xpcie_dev *xdev = container_of(work, struct xpcie_dev,
+ shutdown_event.work);
+
+ intel_xpcie_pci_prepare_dev_reset(xdev, false);
+}
+
+static int xpcie_device_init(struct xpcie_dev *xdev)
+{
+ INIT_DELAYED_WORK(&xdev->wait_event, xpcie_device_poll);
+ INIT_DELAYED_WORK(&xdev->shutdown_event, xpcie_device_shutdown);
+
+ pci_set_master(xdev->pci);
+
+ xdev->xpcie.status = XPCIE_STATUS_UNINIT;
+
+ init_waitqueue_head(&xdev->waitqueue);
+ schedule_delayed_work(&xdev->wait_event, 0);
+
+ return 0;
+}
+
+int intel_xpcie_pci_init(struct xpcie_dev *xdev, struct pci_dev *pdev)
+{
+ int rc;
+
+ if (mutex_lock_interruptible(&xdev->lock))
+ return -EINTR;
+
+ xdev->pci = pdev;
+ pci_set_drvdata(pdev, xdev);
+
+ rc = pci_enable_device_mem(xdev->pci);
+ if (rc) {
+ dev_err(&pdev->dev, "failed to enable pci device\n");
+ goto error_exit;
+ }
+
+ rc = pci_request_regions(xdev->pci, XPCIE_DRIVER_NAME);
+ if (rc) {
+ dev_err(&pdev->dev, "failed to request mmio regions\n");
+ goto error_req_mem;
+ }
+
+ rc = intel_xpcie_pci_map_bar(xdev);
+ if (rc)
+ goto error_map;
+
+ rc = dma_set_mask_and_coherent(&xdev->pci->dev, DMA_BIT_MASK(64));
+ if (rc) {
+ dev_err(&pdev->dev, "failed to set dma mask\n");
+ goto error_dma_mask;
+ }
+
+ intel_xpcie_pci_set_aspm(xdev, aspm_enable);
+
+ rc = xpcie_device_init(xdev);
+ if (!rc)
+ goto init_exit;
+
+error_dma_mask:
+ intel_xpcie_pci_unmap_bar(xdev);
+
+error_map:
+ pci_release_regions(xdev->pci);
+
+error_req_mem:
+ pci_disable_device(xdev->pci);
+
+error_exit:
+ xdev->xpcie.status = XPCIE_STATUS_ERROR;
+
+init_exit:
+ mutex_unlock(&xdev->lock);
+ if (rc)
+ mutex_destroy(&xdev->lock);
+ return rc;
+}
+
+int intel_xpcie_pci_cleanup(struct xpcie_dev *xdev)
+{
+ if (mutex_lock_interruptible(&xdev->lock))
+ return -EINTR;
+
+ cancel_delayed_work(&xdev->wait_event);
+ cancel_delayed_work(&xdev->shutdown_event);
+ xdev->core_irq_callback = NULL;
+ intel_xpcie_pci_irq_cleanup(xdev);
+
+ intel_xpcie_pci_unmap_bar(xdev);
+ pci_release_regions(xdev->pci);
+ pci_disable_device(xdev->pci);
+ pci_set_drvdata(xdev->pci, NULL);
+ xdev->xpcie.status = XPCIE_STATUS_OFF;
+ xdev->irq_enabled = false;
+
+ mutex_unlock(&xdev->lock);
+
+ return 0;
+}
+
+int intel_xpcie_pci_register_irq(struct xpcie_dev *xdev,
+ irq_handler_t irq_handler)
+{
+ int rc;
+
+ if (xdev->xpcie.status != XPCIE_STATUS_READY)
+ return -EINVAL;
+
+ rc = intel_xpcie_pci_irq_init(xdev, irq_handler);
+ if (rc)
+ dev_warn(&xdev->pci->dev, "failed to initialize pci irq\n");
+
+ return rc;
+}
+
+int intel_xpcie_pci_raise_irq(struct xpcie_dev *xdev,
+ enum xpcie_doorbell_type type,
+ u8 value)
+{
+ u16 pci_status;
+
+ pci_read_config_word(xdev->pci, PCI_STATUS, &pci_status);
+
+ return 0;
+}
+
+u32 intel_xpcie_get_device_num(u32 *id_list)
+{
+ struct xpcie_dev *p;
+ u32 num = 0;
+
+ mutex_lock(&dev_list_mutex);
+
+ if (list_empty(&dev_list)) {
+ mutex_unlock(&dev_list_mutex);
+ return 0;
+ }
+
+ list_for_each_entry(p, &dev_list, list) {
+ *id_list++ = p->devid;
+ num++;
+ }
+ mutex_unlock(&dev_list_mutex);
+
+ return num;
+}
+
+int intel_xpcie_get_device_name_by_id(u32 id,
+ char *device_name, size_t name_size)
+{
+ struct xpcie_dev *xdev;
+ size_t size;
+
+ xdev = intel_xpcie_get_device_by_id(id);
+ if (!xdev)
+ return -ENODEV;
+
+ mutex_lock(&xdev->lock);
+
+ size = (name_size > XPCIE_MAX_NAME_LEN) ?
+ XPCIE_MAX_NAME_LEN : name_size;
+ memcpy(device_name, xdev->name, size);
+
+ mutex_unlock(&xdev->lock);
+
+ return 0;
+}
+
+int intel_xpcie_get_device_status_by_id(u32 id, u32 *status)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENODEV;
+
+ mutex_lock(&xdev->lock);
+ *status = xdev->xpcie.status;
+ mutex_unlock(&xdev->lock);
+
+ return 0;
+}
+
+int intel_xpcie_pci_connect_device(u32 id)
+{
+ struct xpcie_dev *xdev;
+ int rc = 0;
+
+ xdev = intel_xpcie_get_device_by_id(id);
+ if (!xdev)
+ return -ENODEV;
+
+ if (mutex_lock_interruptible(&xdev->lock))
+ return -EINTR;
+
+ if (xdev->xpcie.status == XPCIE_STATUS_RUN)
+ goto connect_cleanup;
+
+ if (xdev->xpcie.status == XPCIE_STATUS_OFF) {
+ rc = -ENODEV;
+ goto connect_cleanup;
+ }
+
+ if (xdev->xpcie.status != XPCIE_STATUS_READY) {
+ rc = -EBUSY;
+ goto connect_cleanup;
+ }
+
+connect_cleanup:
+ mutex_unlock(&xdev->lock);
+ return rc;
+}
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.h b/drivers/misc/xlink-pcie/remote_host/pci.h
new file mode 100644
index 000000000000..72de3701f83a
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/pci.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_PCI_HEADER_
+#define XPCIE_PCI_HEADER_
+
+#include <linux/interrupt.h>
+#include <linux/list.h>
+#include <linux/pci.h>
+#include <linux/xlink_drv_inf.h>
+#include "../common/xpcie.h"
+#include "../common/util.h"
+
+#define XPCIE_DRIVER_NAME "mxlk"
+#define XPCIE_DRIVER_DESC "Intel(R) Keem Bay XLink PCIe driver"
+
+#define XPCIE_MAX_NAME_LEN (32)
+
+struct xpcie_dev {
+ struct list_head list;
+ struct mutex lock; /* Device Lock */
+
+ struct pci_dev *pci;
+ char name[XPCIE_MAX_NAME_LEN];
+ u32 devid;
+ char fw_name[XPCIE_MAX_NAME_LEN];
+
+ struct delayed_work wait_event;
+ struct delayed_work shutdown_event;
+ wait_queue_head_t waitqueue;
+ bool irq_enabled;
+ irq_handler_t core_irq_callback;
+
+ struct xpcie xpcie;
+};
+
+static inline struct device *xpcie_to_dev(struct xpcie *xpcie)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+
+ return &xdev->pci->dev;
+}
+
+int intel_xpcie_pci_init(struct xpcie_dev *xdev, struct pci_dev *pdev);
+int intel_xpcie_pci_cleanup(struct xpcie_dev *xdev);
+int intel_xpcie_pci_register_irq(struct xpcie_dev *xdev,
+ irq_handler_t irq_handler);
+int intel_xpcie_pci_raise_irq(struct xpcie_dev *xdev,
+ enum xpcie_doorbell_type type,
+ u8 value);
+
+struct xpcie_dev *intel_xpcie_create_device(u32 sw_device_id,
+ struct pci_dev *pdev);
+void intel_xpcie_remove_device(struct xpcie_dev *xdev);
+void intel_xpcie_list_add_device(struct xpcie_dev *xdev);
+void intel_xpcie_list_del_device(struct xpcie_dev *xdev);
+
+#endif /* XPCIE_PCI_HEADER_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 12/22] misc: xlink-pcie: rh: Add core communication logic
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (10 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 11/22] misc: xlink-pcie: rh: Add PCIe EP driver for Remote Host mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 13/22] misc: xlink-pcie: Add XLink API interface mgross
` (10 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add logic to establish communication with the local host which is through
ring buffer management and MSI/Doorbell interrupts
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/common/core.h | 11 +-
drivers/misc/xlink-pcie/remote_host/Makefile | 2 +
drivers/misc/xlink-pcie/remote_host/core.c | 647 +++++++++++++++++++
drivers/misc/xlink-pcie/remote_host/pci.c | 48 +-
4 files changed, 696 insertions(+), 12 deletions(-)
create mode 100644 drivers/misc/xlink-pcie/remote_host/core.c
diff --git a/drivers/misc/xlink-pcie/common/core.h b/drivers/misc/xlink-pcie/common/core.h
index 84985ef41a64..eec8566c19d9 100644
--- a/drivers/misc/xlink-pcie/common/core.h
+++ b/drivers/misc/xlink-pcie/common/core.h
@@ -10,15 +10,11 @@
#ifndef XPCIE_CORE_HEADER_
#define XPCIE_CORE_HEADER_
-#include <linux/io.h>
-#include <linux/types.h>
-#include <linux/workqueue.h>
-#include <linux/slab.h>
-#include <linux/mutex.h>
-#include <linux/mempool.h>
#include <linux/dma-mapping.h>
-#include <linux/cache.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/wait.h>
+#include <linux/workqueue.h>
#include <linux/xlink_drv_inf.h>
@@ -64,6 +60,7 @@ struct xpcie_buf_desc {
struct xpcie_stream {
size_t frag;
struct xpcie_pipe pipe;
+ struct xpcie_buf_desc **ddr;
};
struct xpcie_list {
diff --git a/drivers/misc/xlink-pcie/remote_host/Makefile b/drivers/misc/xlink-pcie/remote_host/Makefile
index 96374a43023e..e8074dbb1161 100644
--- a/drivers/misc/xlink-pcie/remote_host/Makefile
+++ b/drivers/misc/xlink-pcie/remote_host/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_XLINK_PCIE_RH_DRIVER) += mxlk.o
mxlk-objs := main.o
mxlk-objs += pci.o
+mxlk-objs += core.o
+mxlk-objs += ../common/util.o
diff --git a/drivers/misc/xlink-pcie/remote_host/core.c b/drivers/misc/xlink-pcie/remote_host/core.c
new file mode 100644
index 000000000000..668fded17e9c
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/core.c
@@ -0,0 +1,647 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include "pci.h"
+
+#include "../common/core.h"
+#include "../common/util.h"
+
+static int rx_pool_size = SZ_32M;
+module_param(rx_pool_size, int, 0664);
+MODULE_PARM_DESC(rx_pool_size, "receive pool size (default 32 MiB)");
+
+static int tx_pool_size = SZ_32M;
+module_param(tx_pool_size, int, 0664);
+MODULE_PARM_DESC(tx_pool_size, "transmit pool size (default 32 MiB)");
+
+static int intel_xpcie_version_check(struct xpcie *xpcie)
+{
+ struct xpcie_version version;
+
+ memcpy_fromio(&version,
+ (void __iomem *)(xpcie->mmio + XPCIE_MMIO_VERSION),
+ sizeof(version));
+
+ dev_dbg(xpcie_to_dev(xpcie), "ver: device %u.%u.%u, host %u.%u.%u\n",
+ version.major, version.minor, version.build,
+ XPCIE_VERSION_MAJOR, XPCIE_VERSION_MINOR, XPCIE_VERSION_BUILD);
+
+ if (intel_xpcie_ioread8(xpcie->mmio + XPCIE_MMIO_LEGACY_A0))
+ xpcie->legacy_a0 = true;
+
+ return 0;
+}
+
+static int intel_xpcie_map_dma(struct xpcie *xpcie, struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct device *dev = &xdev->pci->dev;
+
+ bd->phys = dma_map_single(dev, bd->data, bd->length, direction);
+
+ return dma_mapping_error(dev, bd->phys);
+}
+
+static void intel_xpcie_unmap_dma(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct device *dev = &xdev->pci->dev;
+
+ dma_unmap_single(dev, bd->phys, bd->length, direction);
+}
+
+static void intel_xpcie_txrx_cleanup(struct xpcie *xpcie)
+{
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int index;
+
+ xpcie->stop_flag = true;
+ xpcie->no_tx_buffer = false;
+ inf->data_avail = true;
+ wake_up_interruptible(&xpcie->tx_waitq);
+ wake_up_interruptible(&inf->rx_waitq);
+ mutex_lock(&xpcie->wlock);
+ mutex_lock(&inf->rlock);
+
+ if (tx->ddr) {
+ for (index = 0; index < tx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = tx->pipe.tdr + index;
+
+ bd = tx->ddr[index];
+ if (bd) {
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+ intel_xpcie_free_tx_bd(xpcie, bd);
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ }
+ kfree(tx->ddr);
+ }
+
+ if (rx->ddr) {
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = rx->pipe.tdr + index;
+
+ bd = rx->ddr[index];
+ if (bd) {
+ intel_xpcie_unmap_dma(xpcie,
+ bd, DMA_FROM_DEVICE);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ }
+ kfree(rx->ddr);
+ }
+
+ intel_xpcie_list_cleanup(&xpcie->tx_pool);
+ intel_xpcie_list_cleanup(&xpcie->rx_pool);
+
+ mutex_unlock(&inf->rlock);
+ mutex_unlock(&xpcie->wlock);
+}
+
+static int intel_xpcie_txrx_init(struct xpcie *xpcie,
+ struct xpcie_cap_txrx *cap)
+{
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int rc, index, ndesc;
+
+ xpcie->txrx = cap;
+ xpcie->fragment_size = intel_xpcie_ioread32(&cap->fragment_size);
+ xpcie->stop_flag = false;
+
+ tx->pipe.ndesc = intel_xpcie_ioread32(&cap->tx.ndesc);
+ tx->pipe.head = &cap->tx.head;
+ tx->pipe.tail = &cap->tx.tail;
+ tx->pipe.old = intel_xpcie_ioread32(&cap->tx.tail);
+ tx->pipe.tdr = (struct xpcie_transfer_desc *)(xpcie->mmio +
+ intel_xpcie_ioread32(&cap->tx.ring));
+
+ tx->ddr = kcalloc(tx->pipe.ndesc, sizeof(struct xpcie_buf_desc *),
+ GFP_KERNEL);
+ if (!tx->ddr) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ rx->pipe.ndesc = intel_xpcie_ioread32(&cap->rx.ndesc);
+ rx->pipe.head = &cap->rx.head;
+ rx->pipe.tail = &cap->rx.tail;
+ rx->pipe.old = intel_xpcie_ioread32(&cap->rx.head);
+ rx->pipe.tdr = (struct xpcie_transfer_desc *)(xpcie->mmio +
+ intel_xpcie_ioread32(&cap->rx.ring));
+
+ rx->ddr = kcalloc(rx->pipe.ndesc, sizeof(struct xpcie_buf_desc *),
+ GFP_KERNEL);
+ if (!rx->ddr) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ intel_xpcie_list_init(&xpcie->rx_pool);
+ rx_pool_size = roundup(rx_pool_size, xpcie->fragment_size);
+ ndesc = rx_pool_size / xpcie->fragment_size;
+
+ for (index = 0; index < ndesc; index++) {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+ } else {
+ rc = -ENOMEM;
+ goto error;
+ }
+ }
+
+ intel_xpcie_list_init(&xpcie->tx_pool);
+ tx_pool_size = roundup(tx_pool_size, xpcie->fragment_size);
+ ndesc = tx_pool_size / xpcie->fragment_size;
+
+ for (index = 0; index < ndesc; index++) {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+ } else {
+ rc = -ENOMEM;
+ goto error;
+ }
+ }
+
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = rx->pipe.tdr + index;
+
+ bd = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!bd) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_FROM_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie), "failed to map rx bd\n");
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ rx->ddr[index] = bd;
+ intel_xpcie_set_td_address(td, bd->phys);
+ intel_xpcie_set_td_length(td, bd->length);
+ }
+
+ return 0;
+
+error:
+ intel_xpcie_txrx_cleanup(xpcie);
+
+ return rc;
+}
+
+static int intel_xpcie_discover_txrx(struct xpcie *xpcie)
+{
+ struct xpcie_cap_txrx *cap;
+ int error;
+
+ cap = intel_xpcie_cap_find(xpcie, 0, XPCIE_CAP_TXRX);
+ if (cap)
+ error = intel_xpcie_txrx_init(xpcie, cap);
+ else
+ error = -EIO;
+
+ return error;
+}
+
+static void intel_xpcie_start_tx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->tx_wq, &xpcie->tx_event, delay);
+}
+
+static void intel_xpcie_start_rx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->rx_wq, &xpcie->rx_event, delay);
+}
+
+static void intel_xpcie_rx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, rx_event.work);
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct xpcie_buf_desc *bd, *replacement = NULL;
+ unsigned long delay = msecs_to_jiffies(1);
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_transfer_desc *td;
+ u32 head, tail, ndesc, length;
+ u16 status, interface;
+ int rc;
+
+ if (intel_xpcie_get_device_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ ndesc = rx->pipe.ndesc;
+ tail = intel_xpcie_get_tdr_tail(&rx->pipe);
+ head = intel_xpcie_get_tdr_head(&rx->pipe);
+
+ while (head != tail) {
+ td = rx->pipe.tdr + head;
+ bd = rx->ddr[head];
+
+ replacement = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!replacement) {
+ delay = msecs_to_jiffies(20);
+ break;
+ }
+
+ rc = intel_xpcie_map_dma(xpcie, replacement, DMA_FROM_DEVICE);
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to map rx bd (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, replacement);
+ break;
+ }
+
+ status = intel_xpcie_get_td_status(td);
+ interface = intel_xpcie_get_td_interface(td);
+ length = intel_xpcie_get_td_length(td);
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_FROM_DEVICE);
+
+ if (unlikely(status != XPCIE_DESC_STATUS_SUCCESS) ||
+ unlikely(interface >= XPCIE_NUM_INTERFACES)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "rx desc failure, status(%u), interface(%u)\n",
+ status, interface);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ } else {
+ bd->interface = interface;
+ bd->length = length;
+ bd->next = NULL;
+
+ intel_xpcie_add_bd_to_interface(xpcie, bd);
+ }
+
+ rx->ddr[head] = replacement;
+ intel_xpcie_set_td_address(td, replacement->phys);
+ intel_xpcie_set_td_length(td, replacement->length);
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (intel_xpcie_get_tdr_head(&rx->pipe) != head) {
+ intel_xpcie_set_tdr_head(&rx->pipe, head);
+ intel_xpcie_pci_raise_irq(xdev, DATA_RECEIVED, 1);
+ }
+
+ if (!replacement)
+ intel_xpcie_start_rx(xpcie, delay);
+}
+
+static void intel_xpcie_tx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, tx_event.work);
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_transfer_desc *td;
+ u32 head, tail, old, ndesc;
+ struct xpcie_buf_desc *bd;
+ size_t bytes, buffers;
+ u16 status;
+
+ if (intel_xpcie_get_device_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ ndesc = tx->pipe.ndesc;
+ old = tx->pipe.old;
+ tail = intel_xpcie_get_tdr_tail(&tx->pipe);
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+
+ /* clean old entries first */
+ while (old != head) {
+ bd = tx->ddr[old];
+ td = tx->pipe.tdr + old;
+ status = intel_xpcie_get_td_status(td);
+ if (status != XPCIE_DESC_STATUS_SUCCESS)
+ dev_err(xpcie_to_dev(xpcie),
+ "detected tx desc failure (%u)\n", status);
+
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+ intel_xpcie_free_tx_bd(xpcie, bd);
+ tx->ddr[old] = NULL;
+ old = XPCIE_CIRCULAR_INC(old, ndesc);
+ }
+ tx->pipe.old = old;
+
+ /* add new entries */
+ while (XPCIE_CIRCULAR_INC(tail, ndesc) != head) {
+ bd = intel_xpcie_list_get(&xpcie->write);
+ if (!bd)
+ break;
+
+ td = tx->pipe.tdr + tail;
+
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_TO_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "dma mapping error bd addr %p, size %zu\n",
+ bd->data, bd->length);
+ break;
+ }
+
+ tx->ddr[tail] = bd;
+ intel_xpcie_set_td_address(td, bd->phys);
+ intel_xpcie_set_td_length(td, bd->length);
+ intel_xpcie_set_td_interface(td, bd->interface);
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_ERROR);
+
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (intel_xpcie_get_tdr_tail(&tx->pipe) != tail) {
+ intel_xpcie_set_tdr_tail(&tx->pipe, tail);
+ intel_xpcie_pci_raise_irq(xdev, DATA_SENT, 1);
+ }
+
+ intel_xpcie_list_info(&xpcie->write, &bytes, &buffers);
+ if (buffers)
+ xpcie->tx_pending = true;
+ else
+ xpcie->tx_pending = false;
+}
+
+static irqreturn_t intel_xpcie_interrupt(int irq, void *args)
+{
+ struct xpcie_dev *xdev = args;
+ struct xpcie *xpcie;
+
+ xpcie = &xdev->xpcie;
+
+ if (intel_xpcie_get_doorbell(xpcie, FROM_DEVICE, DATA_SENT)) {
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_SENT, 0);
+ intel_xpcie_start_rx(xpcie, 0);
+ }
+ if (intel_xpcie_get_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED)) {
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED, 0);
+ if (xpcie->tx_pending)
+ intel_xpcie_start_tx(xpcie, 0);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int intel_xpcie_events_init(struct xpcie *xpcie)
+{
+ xpcie->rx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->rx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ return -ENOMEM;
+ }
+
+ xpcie->tx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->tx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ destroy_workqueue(xpcie->rx_wq);
+ return -ENOMEM;
+ }
+
+ INIT_DELAYED_WORK(&xpcie->rx_event, intel_xpcie_rx_event_handler);
+ INIT_DELAYED_WORK(&xpcie->tx_event, intel_xpcie_tx_event_handler);
+
+ return 0;
+}
+
+static void intel_xpcie_events_cleanup(struct xpcie *xpcie)
+{
+ cancel_delayed_work_sync(&xpcie->rx_event);
+ cancel_delayed_work_sync(&xpcie->tx_event);
+
+ destroy_workqueue(xpcie->rx_wq);
+ destroy_workqueue(xpcie->tx_wq);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ int status, rc;
+
+ status = intel_xpcie_get_device_status(xpcie);
+ if (status != XPCIE_STATUS_RUN) {
+ dev_err(&xdev->pci->dev,
+ "device status not RUNNING (%d)\n", status);
+ rc = -EBUSY;
+ return rc;
+ }
+
+ intel_xpcie_version_check(xpcie);
+
+ rc = intel_xpcie_events_init(xpcie);
+ if (rc)
+ return rc;
+
+ rc = intel_xpcie_discover_txrx(xpcie);
+ if (rc)
+ goto error_txrx;
+
+ intel_xpcie_interfaces_init(xpcie);
+
+ rc = intel_xpcie_pci_register_irq(xdev, &intel_xpcie_interrupt);
+ if (rc)
+ goto error_txrx;
+
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_RUN);
+
+ return 0;
+
+error_txrx:
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_ERROR);
+
+ return rc;
+}
+
+void intel_xpcie_core_cleanup(struct xpcie *xpcie)
+{
+ if (xpcie->status == XPCIE_STATUS_RUN) {
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_UNINIT);
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_interfaces_cleanup(xpcie);
+ intel_xpcie_txrx_cleanup(xpcie);
+ }
+}
+
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer, size_t *length,
+ uint32_t timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd;
+ size_t remaining, len;
+ long jiffies_passed = 0;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ while (!inf->data_avail) {
+ mutex_unlock(&inf->rlock);
+ if (timeout_ms == 0) {
+ ret = wait_event_interruptible(inf->rx_waitq,
+ inf->data_avail);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(inf->rx_waitq,
+ inf->data_avail,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+ }
+
+ bd = (inf->partial_read) ? inf->partial_read :
+ intel_xpcie_list_get(&inf->read);
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(remaining, bd->length);
+ memcpy(buffer, bd->data, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->data += bcopy;
+ bd->length -= bcopy;
+
+ if (bd->length == 0) {
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ bd = intel_xpcie_list_get(&inf->read);
+ }
+ }
+
+ /* save for next time */
+ inf->partial_read = bd;
+
+ if (!bd)
+ inf->data_avail = false;
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&inf->rlock);
+
+ return 0;
+}
+
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer, size_t *length,
+ uint32_t timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd, *head;
+ long jiffies_passed = 0;
+ size_t remaining, len;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ while (!head) {
+ mutex_unlock(&xpcie->wlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ }
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(bd->length, remaining);
+ memcpy(bd->data, buffer, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->length = bcopy;
+ bd->interface = inf->id;
+
+ if (remaining) {
+ bd->next = intel_xpcie_alloc_tx_bd(xpcie);
+ bd = bd->next;
+ }
+ }
+
+ intel_xpcie_list_put(&xpcie->write, head);
+ intel_xpcie_start_tx(xpcie, 0);
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&xpcie->wlock);
+
+ return 0;
+}
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.c b/drivers/misc/xlink-pcie/remote_host/pci.c
index 0ca0755b591f..f92a78f41324 100644
--- a/drivers/misc/xlink-pcie/remote_host/pci.c
+++ b/drivers/misc/xlink-pcie/remote_host/pci.c
@@ -208,10 +208,8 @@ static void xpcie_device_poll(struct work_struct *work)
{
struct xpcie_dev *xdev = container_of(work, struct xpcie_dev,
wait_event.work);
- u32 dev_status = intel_xpcie_ioread32(xdev->xpcie.mmio +
- XPCIE_MMIO_DEV_STATUS);
- if (dev_status < XPCIE_STATUS_RUN)
+ if (intel_xpcie_get_device_status(&xdev->xpcie) < XPCIE_STATUS_RUN)
schedule_delayed_work(&xdev->wait_event,
msecs_to_jiffies(100));
else
@@ -224,9 +222,10 @@ static int intel_xpcie_pci_prepare_dev_reset(struct xpcie_dev *xdev,
if (mutex_lock_interruptible(&xdev->lock))
return -EINTR;
- if (xdev->core_irq_callback)
+ if (xdev->core_irq_callback) {
xdev->core_irq_callback = NULL;
-
+ intel_xpcie_core_cleanup(&xdev->xpcie);
+ }
xdev->xpcie.status = XPCIE_STATUS_OFF;
if (notify)
intel_xpcie_pci_raise_irq(xdev, DEV_EVENT, REQUEST_RESET);
@@ -326,6 +325,8 @@ int intel_xpcie_pci_cleanup(struct xpcie_dev *xdev)
xdev->core_irq_callback = NULL;
intel_xpcie_pci_irq_cleanup(xdev);
+ intel_xpcie_core_cleanup(&xdev->xpcie);
+
intel_xpcie_pci_unmap_bar(xdev);
pci_release_regions(xdev->pci);
pci_disable_device(xdev->pci);
@@ -359,6 +360,7 @@ int intel_xpcie_pci_raise_irq(struct xpcie_dev *xdev,
{
u16 pci_status;
+ intel_xpcie_set_doorbell(&xdev->xpcie, TO_DEVICE, type, value);
pci_read_config_word(xdev->pci, PCI_STATUS, &pci_status);
return 0;
@@ -445,7 +447,43 @@ int intel_xpcie_pci_connect_device(u32 id)
goto connect_cleanup;
}
+ rc = intel_xpcie_core_init(&xdev->xpcie);
+ if (rc < 0) {
+ dev_err(&xdev->pci->dev, "failed to sync with device\n");
+ goto connect_cleanup;
+ }
+
connect_cleanup:
mutex_unlock(&xdev->lock);
return rc;
}
+
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENODEV;
+
+ return intel_xpcie_core_read(&xdev->xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENODEV;
+
+ return intel_xpcie_core_write(&xdev->xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_reset_device(u32 id)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENOMEM;
+
+ return intel_xpcie_pci_prepare_dev_reset(xdev, true);
+}
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 13/22] misc: xlink-pcie: Add XLink API interface
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (11 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 12/22] misc: xlink-pcie: rh: Add core communication logic mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:06 ` [PATCH 14/22] misc: xlink-pcie: Add asynchronous event notification support for XLink mgross
` (9 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Provide interface for XLink layer to interact with XLink PCIe transport
layer on both local host and remote host.
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/common/interface.c | 109 +++++++++++++++++++
drivers/misc/xlink-pcie/local_host/Makefile | 1 +
drivers/misc/xlink-pcie/remote_host/Makefile | 1 +
3 files changed, 111 insertions(+)
create mode 100644 drivers/misc/xlink-pcie/common/interface.c
diff --git a/drivers/misc/xlink-pcie/common/interface.c b/drivers/misc/xlink-pcie/common/interface.c
new file mode 100644
index 000000000000..56c1d9ed9d8f
--- /dev/null
+++ b/drivers/misc/xlink-pcie/common/interface.c
@@ -0,0 +1,109 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include <linux/xlink_drv_inf.h>
+
+#include "core.h"
+#include "xpcie.h"
+
+/* Define xpcie driver interface API */
+int xlink_pcie_get_device_list(u32 *sw_device_id_list, u32 *num_devices)
+{
+ if (!sw_device_id_list || !num_devices)
+ return -EINVAL;
+
+ *num_devices = intel_xpcie_get_device_num(sw_device_id_list);
+
+ return 0;
+}
+EXPORT_SYMBOL(xlink_pcie_get_device_list);
+
+int xlink_pcie_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size)
+{
+ if (!device_name)
+ return -EINVAL;
+
+ return intel_xpcie_get_device_name_by_id(sw_device_id,
+ device_name, name_size);
+}
+EXPORT_SYMBOL(xlink_pcie_get_device_name);
+
+int xlink_pcie_get_device_status(u32 sw_device_id, u32 *device_status)
+{
+ u32 status;
+ int rc;
+
+ if (!device_status)
+ return -EINVAL;
+
+ rc = intel_xpcie_get_device_status_by_id(sw_device_id, &status);
+ if (rc)
+ return rc;
+
+ switch (status) {
+ case XPCIE_STATUS_READY:
+ case XPCIE_STATUS_RUN:
+ *device_status = _XLINK_DEV_READY;
+ break;
+ case XPCIE_STATUS_ERROR:
+ *device_status = _XLINK_DEV_ERROR;
+ break;
+ case XPCIE_STATUS_RECOVERY:
+ *device_status = _XLINK_DEV_RECOVERY;
+ break;
+ case XPCIE_STATUS_OFF:
+ *device_status = _XLINK_DEV_OFF;
+ break;
+ default:
+ *device_status = _XLINK_DEV_BUSY;
+ break;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(xlink_pcie_get_device_status);
+
+int xlink_pcie_boot_device(u32 sw_device_id, const char *binary_name)
+{
+ return 0;
+}
+EXPORT_SYMBOL(xlink_pcie_boot_device);
+
+int xlink_pcie_connect(u32 sw_device_id)
+{
+ return intel_xpcie_pci_connect_device(sw_device_id);
+}
+EXPORT_SYMBOL(xlink_pcie_connect);
+
+int xlink_pcie_read(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout)
+{
+ if (!data || !size)
+ return -EINVAL;
+
+ return intel_xpcie_pci_read(sw_device_id, data, size, timeout);
+}
+EXPORT_SYMBOL(xlink_pcie_read);
+
+int xlink_pcie_write(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout)
+{
+ if (!data || !size)
+ return -EINVAL;
+
+ return intel_xpcie_pci_write(sw_device_id, data, size, timeout);
+}
+EXPORT_SYMBOL(xlink_pcie_write);
+
+int xlink_pcie_reset_device(u32 sw_device_id)
+{
+ return intel_xpcie_pci_reset_device(sw_device_id);
+}
+EXPORT_SYMBOL(xlink_pcie_reset_device);
diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
index 65df94c7e860..16bb1e7345ac 100644
--- a/drivers/misc/xlink-pcie/local_host/Makefile
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -3,3 +3,4 @@ mxlk_ep-objs := epf.o
mxlk_ep-objs += dma.o
mxlk_ep-objs += core.o
mxlk_ep-objs += ../common/util.o
+mxlk_ep-objs += ../common/interface.o
diff --git a/drivers/misc/xlink-pcie/remote_host/Makefile b/drivers/misc/xlink-pcie/remote_host/Makefile
index e8074dbb1161..088e121ad46e 100644
--- a/drivers/misc/xlink-pcie/remote_host/Makefile
+++ b/drivers/misc/xlink-pcie/remote_host/Makefile
@@ -3,3 +3,4 @@ mxlk-objs := main.o
mxlk-objs += pci.o
mxlk-objs += core.o
mxlk-objs += ../common/util.o
+mxlk-objs += ../common/interface.o
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 14/22] misc: xlink-pcie: Add asynchronous event notification support for XLink
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (12 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 13/22] misc: xlink-pcie: Add XLink API interface mgross
@ 2020-11-30 23:06 ` mgross
2020-11-30 23:07 ` [PATCH 15/22] xlink-ipc: Add xlink ipc device tree bindings mgross
` (8 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:06 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Srikanth Thokala,
Arnd Bergmann, Greg Kroah-Hartman
From: Srikanth Thokala <srikanth.thokala@intel.com>
Add support to notify XLink layer upon PCIe link UP/DOWN events
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Srikanth Thokala <srikanth.thokala@intel.com>
---
drivers/misc/xlink-pcie/common/core.h | 3 ++
drivers/misc/xlink-pcie/common/interface.c | 17 ++++++++++
drivers/misc/xlink-pcie/local_host/core.c | 11 +++++++
drivers/misc/xlink-pcie/remote_host/main.c | 3 ++
drivers/misc/xlink-pcie/remote_host/pci.c | 36 ++++++++++++++++++++++
drivers/misc/xlink-pcie/remote_host/pci.h | 3 ++
include/linux/xlink_drv_inf.h | 12 ++++++++
7 files changed, 85 insertions(+)
diff --git a/drivers/misc/xlink-pcie/common/core.h b/drivers/misc/xlink-pcie/common/core.h
index eec8566c19d9..34b6c268aac5 100644
--- a/drivers/misc/xlink-pcie/common/core.h
+++ b/drivers/misc/xlink-pcie/common/core.h
@@ -241,4 +241,7 @@ int intel_xpcie_pci_connect_device(u32 id);
int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout);
int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout);
int intel_xpcie_pci_reset_device(u32 id);
+int intel_xpcie_pci_register_device_event(u32 sw_device_id,
+ xlink_device_event event_notif_fn);
+int intel_xpcie_pci_unregister_device_event(u32 sw_device_id);
#endif /* XPCIE_CORE_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/common/interface.c b/drivers/misc/xlink-pcie/common/interface.c
index 56c1d9ed9d8f..4ad291ff97c8 100644
--- a/drivers/misc/xlink-pcie/common/interface.c
+++ b/drivers/misc/xlink-pcie/common/interface.c
@@ -107,3 +107,20 @@ int xlink_pcie_reset_device(u32 sw_device_id)
return intel_xpcie_pci_reset_device(sw_device_id);
}
EXPORT_SYMBOL(xlink_pcie_reset_device);
+
+int xlink_pcie_register_device_event(u32 sw_device_id,
+ xlink_device_event event_notif_fn)
+{
+ if (!event_notif_fn)
+ return -EINVAL;
+
+ return intel_xpcie_pci_register_device_event(sw_device_id,
+ event_notif_fn);
+}
+EXPORT_SYMBOL(xlink_pcie_register_device_event);
+
+int xlink_pcie_unregister_device_event(u32 sw_device_id)
+{
+ return intel_xpcie_pci_unregister_device_event(sw_device_id);
+}
+EXPORT_SYMBOL(xlink_pcie_unregister_device_event);
diff --git a/drivers/misc/xlink-pcie/local_host/core.c b/drivers/misc/xlink-pcie/local_host/core.c
index 0f83b5ffc4f2..628c383113ca 100644
--- a/drivers/misc/xlink-pcie/local_host/core.c
+++ b/drivers/misc/xlink-pcie/local_host/core.c
@@ -892,3 +892,14 @@ int intel_xpcie_pci_reset_device(u32 id)
{
return 0;
}
+
+int intel_xpcie_pci_register_device_event(u32 sw_device_id,
+ xlink_device_event event_notif_fn)
+{
+ return 0;
+}
+
+int intel_xpcie_pci_unregister_device_event(u32 sw_device_id)
+{
+ return 0;
+}
diff --git a/drivers/misc/xlink-pcie/remote_host/main.c b/drivers/misc/xlink-pcie/remote_host/main.c
index 810da1509418..421635321f1b 100644
--- a/drivers/misc/xlink-pcie/remote_host/main.c
+++ b/drivers/misc/xlink-pcie/remote_host/main.c
@@ -55,6 +55,8 @@ static int intel_xpcie_probe(struct pci_dev *pdev,
if (new_device)
intel_xpcie_list_add_device(xdev);
+ intel_xpcie_pci_notify_event(xdev, NOTIFY_DEVICE_CONNECTED);
+
return ret;
}
@@ -64,6 +66,7 @@ static void intel_xpcie_remove(struct pci_dev *pdev)
if (xdev) {
intel_xpcie_pci_cleanup(xdev);
+ intel_xpcie_pci_notify_event(xdev, NOTIFY_DEVICE_DISCONNECTED);
intel_xpcie_remove_device(xdev);
}
}
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.c b/drivers/misc/xlink-pcie/remote_host/pci.c
index f92a78f41324..0046bff5f604 100644
--- a/drivers/misc/xlink-pcie/remote_host/pci.c
+++ b/drivers/misc/xlink-pcie/remote_host/pci.c
@@ -8,6 +8,7 @@
****************************************************************************/
#include <linux/mutex.h>
+#include <linux/pci.h>
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/workqueue.h>
@@ -487,3 +488,38 @@ int intel_xpcie_pci_reset_device(u32 id)
return intel_xpcie_pci_prepare_dev_reset(xdev, true);
}
+
+int intel_xpcie_pci_register_device_event(u32 sw_device_id,
+ xlink_device_event event_notif_fn)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(sw_device_id);
+
+ if (!xdev)
+ return -ENOMEM;
+
+ xdev->event_fn = event_notif_fn;
+
+ return 0;
+}
+
+int intel_xpcie_pci_unregister_device_event(u32 sw_device_id)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(sw_device_id);
+
+ if (!xdev)
+ return -ENOMEM;
+
+ xdev->event_fn = NULL;
+
+ return 0;
+}
+
+void intel_xpcie_pci_notify_event(struct xpcie_dev *xdev,
+ enum xlink_device_event_type event_type)
+{
+ if (event_type >= NUM_EVENT_TYPE)
+ return;
+
+ if (xdev->event_fn)
+ xdev->event_fn(xdev->devid, event_type);
+}
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.h b/drivers/misc/xlink-pcie/remote_host/pci.h
index 72de3701f83a..a05dedf36a12 100644
--- a/drivers/misc/xlink-pcie/remote_host/pci.h
+++ b/drivers/misc/xlink-pcie/remote_host/pci.h
@@ -38,6 +38,7 @@ struct xpcie_dev {
irq_handler_t core_irq_callback;
struct xpcie xpcie;
+ xlink_device_event event_fn;
};
static inline struct device *xpcie_to_dev(struct xpcie *xpcie)
@@ -60,5 +61,7 @@ struct xpcie_dev *intel_xpcie_create_device(u32 sw_device_id,
void intel_xpcie_remove_device(struct xpcie_dev *xdev);
void intel_xpcie_list_add_device(struct xpcie_dev *xdev);
void intel_xpcie_list_del_device(struct xpcie_dev *xdev);
+void intel_xpcie_pci_notify_event(struct xpcie_dev *xdev,
+ enum xlink_device_event_type event_type);
#endif /* XPCIE_PCI_HEADER_ */
diff --git a/include/linux/xlink_drv_inf.h b/include/linux/xlink_drv_inf.h
index ffe8f4c253e6..5ca0ae1ae2e3 100644
--- a/include/linux/xlink_drv_inf.h
+++ b/include/linux/xlink_drv_inf.h
@@ -44,6 +44,15 @@ enum _xlink_device_status {
_XLINK_DEV_READY
};
+enum xlink_device_event_type {
+ NOTIFY_DEVICE_DISCONNECTED,
+ NOTIFY_DEVICE_CONNECTED,
+ NUM_EVENT_TYPE
+};
+
+typedef int (*xlink_device_event)(u32 sw_device_id,
+ enum xlink_device_event_type event_type);
+
int xlink_pcie_get_device_list(u32 *sw_device_id_list,
u32 *num_devices);
int xlink_pcie_get_device_name(u32 sw_device_id, char *device_name,
@@ -57,4 +66,7 @@ int xlink_pcie_read(u32 sw_device_id, void *data, size_t *const size,
int xlink_pcie_write(u32 sw_device_id, void *data, size_t *const size,
u32 timeout);
int xlink_pcie_reset_device(u32 sw_device_id);
+int xlink_pcie_register_device_event(u32 sw_device_id,
+ xlink_device_event event_notif_fn);
+int xlink_pcie_unregister_device_event(u32 sw_device_id);
#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 15/22] xlink-ipc: Add xlink ipc device tree bindings
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (13 preceding siblings ...)
2020-11-30 23:06 ` [PATCH 14/22] misc: xlink-pcie: Add asynchronous event notification support for XLink mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 16/22] xlink-ipc: Add xlink ipc driver mgross
` (7 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly, Rob Herring,
devicetree, Ryan Carnaghi
From: Seamus Kelly <seamus.kelly@intel.com>
Add device tree bindings for the xLink IPC driver which enables xLink to
control and communicate with the VPU IP present on the Intel Keem Bay
SoC.
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
Signed-off-by: Ryan Carnaghi <ryan.r.carnaghi@intel.com>
---
.../misc/intel,keembay-xlink-ipc.yaml | 49 +++++++++++++++++++
1 file changed, 49 insertions(+)
create mode 100644 Documentation/devicetree/bindings/misc/intel,keembay-xlink-ipc.yaml
diff --git a/Documentation/devicetree/bindings/misc/intel,keembay-xlink-ipc.yaml b/Documentation/devicetree/bindings/misc/intel,keembay-xlink-ipc.yaml
new file mode 100644
index 000000000000..699e43c4cd40
--- /dev/null
+++ b/Documentation/devicetree/bindings/misc/intel,keembay-xlink-ipc.yaml
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright (c) Intel Corporation. All rights reserved.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/misc/intel,keembay-xlink-ipc.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel Keem Bay xlink IPC
+
+maintainers:
+ - Kelly Seamus <seamus.kelly@intel.com>
+
+description: |
+ The Keem Bay xlink IPC driver enables the communication/control sub-system
+ for internal IPC communications within the Intel Keem Bay SoC.
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: intel,keembay-xlink-ipc
+
+ memory-region:
+ items:
+ - description: reference to the CSS xlink IPC reserved memory region.
+ - description: reference to the MSS xlink IPC reserved memory region.
+
+ intel,keembay-vpu-ipc-id:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description: The numeric ID identifying the VPU within the xLink stack.
+
+ intel,keembay-vpu-ipc-name:
+ $ref: "/schemas/types.yaml#/definitions/string"
+ description: User-friendly name for the VPU within the xLink stack.
+
+ intel,keembay-vpu-ipc:
+ $ref: "/schemas/types.yaml#/definitions/phandle"
+ description: reference to the corresponding intel,keembay-vpu-ipc node.
+
+examples:
+ - |
+ xlink-ipc {
+ compatible = "intel,keembay-xlink-ipc";
+ memory-region = <&css_xlink_reserved>,
+ <&mss_xlink_reserved>;
+ intel,keembay-vpu-ipc-id = <0x0>;
+ intel,keembay-vpu-ipc-name = "vpu-slice-0";
+ intel,keembay-vpu-ipc = <&vpuipc>;
+ };
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 16/22] xlink-ipc: Add xlink ipc driver
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (14 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 15/22] xlink-ipc: Add xlink ipc device tree bindings mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 17/22] xlink-core: Add xlink core device tree bindings mgross
` (6 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Jonathan Corbet, Cc : Derek Kiernan, Dragan Cvetic,
Arnd Bergmann, Greg Kroah-Hartman, linux-doc, Ryan Carnaghi
From: Seamus Kelly <seamus.kelly@intel.com>
Add xLink driver, which interfaces the xLink Core driver with the Keem
Bay VPU IPC driver, thus enabling xLink to control and communicate with
the VPU IP present on the Intel Keem Bay SoC.
Specifically the driver enables xLink Core to:
* Boot / Reset the VPU IP
* Register to VPU IP event notifications (device connected, device
disconnected, WDT event)
* Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
* Exchange data with the VPU IP, using the Keem Bay IPC mechanism
- Including the ability to send 'volatile' data (i.e., small amount of
data, up to 128-bytes that was not allocated in the CPU/VPU shared
memory region)
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Cc: Derek Kiernan <derek.kiernan@xilinx.com>
Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-doc@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
Signed-off-by: Ryan Carnaghi <ryan.r.carnaghi@intel.com>
---
Documentation/vpu/index.rst | 1 +
Documentation/vpu/xlink-ipc.rst | 50 ++
MAINTAINERS | 6 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/xlink-ipc/Kconfig | 7 +
drivers/misc/xlink-ipc/Makefile | 4 +
drivers/misc/xlink-ipc/xlink-ipc.c | 879 +++++++++++++++++++++++++++++
include/linux/xlink-ipc.h | 48 ++
9 files changed, 997 insertions(+)
create mode 100644 Documentation/vpu/xlink-ipc.rst
create mode 100644 drivers/misc/xlink-ipc/Kconfig
create mode 100644 drivers/misc/xlink-ipc/Makefile
create mode 100644 drivers/misc/xlink-ipc/xlink-ipc.c
create mode 100644 include/linux/xlink-ipc.h
diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
index 661cc700ee45..49c78bb65b83 100644
--- a/Documentation/vpu/index.rst
+++ b/Documentation/vpu/index.rst
@@ -15,3 +15,4 @@ This documentation contains information for the Intel VPU stack.
vpu-stack-overview
xlink-pcie
+ xlink-ipc
diff --git a/Documentation/vpu/xlink-ipc.rst b/Documentation/vpu/xlink-ipc.rst
new file mode 100644
index 000000000000..af583579e70d
--- /dev/null
+++ b/Documentation/vpu/xlink-ipc.rst
@@ -0,0 +1,50 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+Kernel driver: xLink IPC driver
+=================================
+Supported chips:
+
+* | Intel Edge.AI Computer Vision platforms: Keem Bay
+ | Suffix: Bay
+ | Datasheet: (not yet publicly available)
+
+------------
+Introduction
+------------
+
+The xLink IPC driver interfaces the xLink Core driver with the Keem Bay VPU IPC
+driver, thus enabling xLink to control and communicate with the VPU IP present
+on the Intel Keem Bay SoC.
+
+Specifically the driver enables xLink Core to:
+
+* Boot / Reset the VPU IP
+* Register to VPU IP event notifications (device connected, device disconnected,
+ WDT event)
+* Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
+* Exchange data with the VPU IP, using the Keem Bay IPC mechanism
+
+ * Including the ability to send 'volatile' data (i.e., small amount of data,
+ up to 128-bytes that was not allocated in the CPU/VPU shared memory region)
+
+Sending / Receiving 'volatile' data
+-----------------------------------
+
+Data to be exchanged with Keem Bay IPC needs to be allocated in the portion of
+DDR shared between the CPU and VPU.
+
+This can be impractical for small amount of data that user code can allocate
+on the stack.
+
+To reduce the burden on user code, xLink Core provides special send / receive
+functions to send up to 128 bytes of 'volatile data', i.e., data that is not
+allocated in the shared memory and that might also disappear after the xLink
+API is called (e.g., because allocated on the stack).
+
+The xLink IPC driver implements support for transferring such 'volatile data'
+to the VPU using Keem Bay IPC. To this end, the driver reserved some memory in
+the shared memory region.
+
+When volatile data is to be sent, xLink IPC allocates a buffer from the
+reserved memory region and copies the volatile data to the buffer. The buffer
+is then transferred to the VPU using Keem Bay IPC.
diff --git a/MAINTAINERS b/MAINTAINERS
index b3468dea6557..ebc2cce7251f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1952,6 +1952,12 @@ F: Documentation/devicetree/bindings/arm/intel,keembay.yaml
F: arch/arm64/boot/dts/intel/keembay-evm.dts
F: arch/arm64/boot/dts/intel/keembay-soc.dtsi
+ARM/INTEL XLINK IPC SUPPORT
+M: Seamus Kelly <seamus.kelly@intel.com>
+M: Mark Gross <mgross@linux.intel.com>
+S: Maintained
+F: drivers/misc/xlink-ipc/
+
ARM/INTEL KEEMBAY XLINK PCIE SUPPORT
M: Srikanth Thokala <srikanth.thokala@intel.com>
M: Mark Gross <mgross@linux.intel.com>
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index dfb98e444c6e..1f81ea915b95 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -482,4 +482,5 @@ source "drivers/misc/cardreader/Kconfig"
source "drivers/misc/habanalabs/Kconfig"
source "drivers/misc/uacce/Kconfig"
source "drivers/misc/xlink-pcie/Kconfig"
+source "drivers/misc/xlink-ipc/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index d17621fc43d5..b51495a2f1e0 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -58,3 +58,4 @@ obj-$(CONFIG_UACCE) += uacce/
obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
obj-y += xlink-pcie/
+obj-$(CONFIG_XLINK_IPC) += xlink-ipc/
diff --git a/drivers/misc/xlink-ipc/Kconfig b/drivers/misc/xlink-ipc/Kconfig
new file mode 100644
index 000000000000..6aa2592fe9a3
--- /dev/null
+++ b/drivers/misc/xlink-ipc/Kconfig
@@ -0,0 +1,7 @@
+config XLINK_IPC
+ tristate "Support for XLINK IPC"
+ depends on KEEMBAY_VPU_IPC
+ help
+ XLINK IPC enables the communication/control IPC Sub-System.
+
+ Select M if you have an Intel SoC with a Vision Processing Unit (VPU)
diff --git a/drivers/misc/xlink-ipc/Makefile b/drivers/misc/xlink-ipc/Makefile
new file mode 100644
index 000000000000..f92c95525e23
--- /dev/null
+++ b/drivers/misc/xlink-ipc/Makefile
@@ -0,0 +1,4 @@
+#
+# Makefile for xlink IPC Linux driver
+#
+obj-$(CONFIG_XLINK_IPC) += xlink-ipc.o
diff --git a/drivers/misc/xlink-ipc/xlink-ipc.c b/drivers/misc/xlink-ipc/xlink-ipc.c
new file mode 100644
index 000000000000..088bb7b3da46
--- /dev/null
+++ b/drivers/misc/xlink-ipc/xlink-ipc.c
@@ -0,0 +1,879 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Linux Kernel Platform API
+ *
+ * Copyright (C) 2018-2020 Intel Corporation
+ *
+ */
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include <linux/soc/intel/keembay-vpu-ipc.h>
+
+#include <linux/xlink-ipc.h>
+
+#define XLINK_IPC_MAX_DEVICE_NAME_SIZE 12
+
+/* used to extract fields from and create xlink sw device id */
+#define SW_DEVICE_ID_INTERFACE_SHIFT 24U
+#define SW_DEVICE_ID_INTERFACE_MASK 0x7
+#define SW_DEVICE_ID_INTERFACE_SMASK \
+ (SW_DEVICE_ID_INTERFACE_MASK << SW_DEVICE_ID_INTERFACE_SHIFT)
+#define SW_DEVICE_ID_INTERFACE_IPC_VALUE 0x0
+#define SW_DEVICE_ID_INTERFACE_IPC_SVALUE \
+ (SW_DEVICE_ID_INTERFACE_IPC_VALUE << SW_DEVICE_ID_INTERFACE_SHIFT)
+#define SW_DEVICE_ID_VPU_ID_SHIFT 1U
+#define SW_DEVICE_ID_VPU_ID_MASK 0x7
+#define SW_DEVICE_ID_VPU_ID_SMASK \
+ (SW_DEVICE_ID_VPU_ID_MASK << SW_DEVICE_ID_VPU_ID_SHIFT)
+#define GET_VPU_ID_FROM_SW_DEVICE_ID(id) \
+ (((id) >> SW_DEVICE_ID_VPU_ID_SHIFT) & SW_DEVICE_ID_VPU_ID_MASK)
+#define GET_SW_DEVICE_ID_FROM_VPU_ID(id) \
+ ((((id) << SW_DEVICE_ID_VPU_ID_SHIFT) & SW_DEVICE_ID_VPU_ID_SMASK) \
+ | SW_DEVICE_ID_INTERFACE_IPC_SVALUE)
+
+/* the maximum buffer size for volatile xlink operations */
+#define XLINK_MAX_BUF_SIZE 128U
+
+/* indices used to retrieve reserved memory from the dt */
+#define LOCAL_XLINK_IPC_BUFFER_IDX 0
+#define REMOTE_XLINK_IPC_BUFFER_IDX 1
+
+/* index used to retrieve the vpu ipc device phandle from the dt */
+#define VPU_IPC_DEVICE_PHANDLE_IDX 1
+
+/* the timeout (in ms) used to wait for the vpu ready message */
+#define XLINK_VPU_WAIT_FOR_READY_MS 3000
+
+/* xlink buffer memory region */
+struct xlink_buf_mem {
+ struct device *dev; /* child device managing the memory region */
+ void *vaddr; /* the virtual address of the memory region */
+ dma_addr_t dma_handle; /* the physical address of the memory region */
+ size_t size; /* the size of the memory region */
+};
+
+/* xlink buffer pool */
+struct xlink_buf_pool {
+ void *buf; /* pointer to the start of pool area */
+ size_t buf_cnt; /* pool size (i.e., number of buffers) */
+ size_t idx; /* current index */
+ spinlock_t lock; /* the lock protecting this pool */
+};
+
+/* xlink ipc device */
+struct xlink_ipc_dev {
+ struct platform_device *pdev; /* pointer to platform device */
+ u32 vpu_id; /* The VPU ID defined in the device tree */
+ u32 sw_device_id; /* the sw device id */
+ const char *device_name; /* the vpu device name */
+ struct xlink_buf_mem local_xlink_mem; /* tx buffer memory region */
+ struct xlink_buf_mem remote_xlink_mem; /* rx buffer memory region */
+ struct xlink_buf_pool xlink_buf_pool; /* tx buffer pool */
+ struct device *vpu_dev; /* pointer to vpu ipc device */
+ int (*callback)(u32 sw_device_id, u32 event);
+};
+
+/* Events that can be notified via callback, when registered. */
+enum xlink_vpu_event {
+ XLINK_VPU_NOTIFY_DISCONNECT = 0,
+ XLINK_VPU_NOTIFY_CONNECT,
+ XLINK_VPU_NOTIFY_MSS_WDT,
+ XLINK_VPU_NOTIFY_NCE_WDT,
+ NUM_EVENT_TYPE
+};
+
+#define VPU_ID 0
+#define VPU_DEVICE_NAME "vpu-0"
+#define VPU_SW_DEVICE_ID 0
+static struct xlink_ipc_dev *xlink_dev;
+
+/*
+ * Functions related to reserved-memory sub-devices.
+ */
+
+/*
+ * xlink_reserved_memory_remove() - Removes the reserved memory sub-devices.
+ *
+ * @xlink_dev: [in] The xlink ipc device with reserved memory sub-devices.
+ */
+static void xlink_reserved_memory_remove(struct xlink_ipc_dev *xlink_dev)
+{
+ device_unregister(xlink_dev->local_xlink_mem.dev);
+ device_unregister(xlink_dev->remote_xlink_mem.dev);
+}
+
+/*
+ * xlink_reserved_mem_release() - Reserved memory release callback function.
+ *
+ * @dev: [in] The reserved memory sub-device.
+ */
+static void xlink_reserved_mem_release(struct device *dev)
+{
+ of_reserved_mem_device_release(dev);
+}
+
+/*
+ * get_xlink_reserved_mem_size() - Gets the size of the reserved memory region.
+ *
+ * @dev: [in] The device the reserved memory region is allocated to.
+ * @idx: [in] The reserved memory region's index in the phandle table.
+ *
+ * Return: The reserved memory size, 0 on failure.
+ */
+static resource_size_t get_xlink_reserved_mem_size(struct device *dev, int idx)
+{
+ struct device_node *np;
+ struct resource mem;
+ int rc;
+
+ np = of_parse_phandle(dev->of_node, "memory-region", idx);
+ if (!np) {
+ dev_err(dev, "Couldn't find memory-region %d\n", idx);
+ return 0;
+ }
+
+ rc = of_address_to_resource(np, 0, &mem);
+ if (rc) {
+ dev_err(dev, "Couldn't map address to resource %d\n", idx);
+ return 0;
+ }
+ return resource_size(&mem);
+}
+
+/*
+ * init_xlink_reserved_mem_dev() - Initializes the reserved memory sub-devices.
+ *
+ * @dev: [in] The parent device of the reserved memory sub-device.
+ * @name: [in] The name to assign to the memory region.
+ * @idx: [in] The reserved memory region index in the phandle table.
+ *
+ * Return: The initialized sub-device, NULL on failure.
+ */
+static struct device *init_xlink_reserved_mem_dev(struct device *dev,
+ const char *name, int idx)
+{
+ struct device *child;
+ int rc;
+
+ child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+ if (!child)
+ return NULL;
+
+ device_initialize(child);
+ dev_set_name(child, "%s:%s", dev_name(dev), name);
+ dev_err(dev, " dev_name %s, name %s\n", dev_name(dev), name);
+ child->parent = dev;
+ child->dma_mask = dev->dma_mask;
+ child->coherent_dma_mask = dev->coherent_dma_mask;
+ /* set up dma configuration using information from parent's dt node */
+ rc = of_dma_configure(child, dev->of_node, true);
+ if (rc)
+ return NULL;
+ child->release = xlink_reserved_mem_release;
+
+ rc = device_add(child);
+ if (rc)
+ goto err;
+ rc = of_reserved_mem_device_init_by_idx(child, dev->of_node, idx);
+ if (rc) {
+ dev_err(dev, "Couldn't get reserved memory with idx = %d, %d\n",
+ idx, rc);
+ device_del(child);
+ goto err;
+ }
+ return child;
+
+err:
+ put_device(child);
+ return NULL;
+}
+
+/*
+ * xlink_reserved_memory_init() - Initialize reserved memory for the device.
+ *
+ * @xlink_dev: [in] The xlink ipc device the reserved memory is allocated to.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_reserved_memory_init(struct xlink_ipc_dev *xlink_dev)
+{
+ struct device *dev = &xlink_dev->pdev->dev;
+
+ xlink_dev->local_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
+ "xlink_local_reserved",
+ LOCAL_XLINK_IPC_BUFFER_IDX);
+ if (!xlink_dev->local_xlink_mem.dev)
+ return -ENOMEM;
+
+ xlink_dev->local_xlink_mem.size = get_xlink_reserved_mem_size(dev,
+ LOCAL_XLINK_IPC_BUFFER_IDX);
+
+ xlink_dev->remote_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
+ "xlink_remote_reserved",
+ REMOTE_XLINK_IPC_BUFFER_IDX);
+ if (!xlink_dev->remote_xlink_mem.dev) {
+ device_unregister(xlink_dev->local_xlink_mem.dev);
+ return -ENOMEM;
+ }
+
+ xlink_dev->remote_xlink_mem.size = get_xlink_reserved_mem_size(dev,
+ REMOTE_XLINK_IPC_BUFFER_IDX);
+
+ return 0;
+}
+
+/*
+ * xlink_reserved_memory_alloc() - Allocate reserved memory for the device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_reserved_memory_alloc(struct xlink_ipc_dev *xlink_dev)
+{
+ xlink_dev->local_xlink_mem.vaddr =
+ dmam_alloc_coherent(xlink_dev->local_xlink_mem.dev,
+ xlink_dev->local_xlink_mem.size,
+ &xlink_dev->local_xlink_mem.dma_handle,
+ GFP_KERNEL);
+ if (!xlink_dev->local_xlink_mem.vaddr) {
+ dev_err(&xlink_dev->pdev->dev,
+ "Failed to allocate from local reserved memory.\n");
+ return -ENOMEM;
+ }
+ xlink_dev->remote_xlink_mem.vaddr = dmam_alloc_coherent
+ (xlink_dev->remote_xlink_mem.dev,
+ xlink_dev->remote_xlink_mem.size,
+ &xlink_dev->remote_xlink_mem.dma_handle,
+ GFP_KERNEL);
+ if (!xlink_dev->remote_xlink_mem.vaddr) {
+ dev_err(&xlink_dev->pdev->dev,
+ "Failed to allocate from remote reserved memory.\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+/*
+ * init_xlink_buf_pool() - Initialize the device's tx buffer pool.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success.
+ */
+static int init_xlink_buf_pool(struct xlink_ipc_dev *xlink_dev)
+{
+ struct xlink_buf_mem *mem = &xlink_dev->local_xlink_mem;
+
+ memset(mem->vaddr, 0, mem->size);
+ xlink_dev->xlink_buf_pool.buf = mem->vaddr;
+ xlink_dev->xlink_buf_pool.buf_cnt =
+ mem->size / XLINK_MAX_BUF_SIZE;
+ xlink_dev->xlink_buf_pool.idx = 0;
+ dev_info(&xlink_dev->pdev->dev, "xlink Buffer Pool size: %zX\n",
+ xlink_dev->xlink_buf_pool.buf_cnt);
+ spin_lock_init(&xlink_dev->xlink_buf_pool.lock);
+
+ return 0;
+}
+
+/*
+ * xlink_phys_to_virt() - Convert an xlink physical addresses to a virtual one.
+ *
+ * @xlink_mem: [in] The memory region where the physical address is located.
+ * @paddr: [in] The physical address to convert to a virtual one.
+ *
+ * Return: The corresponding virtual address, or NULL if the
+ * physical address is not in the expected memory
+ * range.
+ */
+static void *xlink_phys_to_virt(const struct xlink_buf_mem *xlink_mem,
+ u32 paddr)
+{
+ if (unlikely(paddr < xlink_mem->dma_handle) ||
+ paddr >= (xlink_mem->dma_handle + xlink_mem->size))
+ return NULL;
+
+ return xlink_mem->vaddr + (paddr - xlink_mem->dma_handle);
+}
+
+/*
+ * xlink_virt_to_phys() - Convert an xlink virtual addresses to a physical one.
+ *
+ * @xlink_mem: [in] The memory region where the physical address is located.
+ * @vaddr: [in] The virtual address to convert to a physical one.
+ * @paddr: [out] Where to store the computed physical address.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_virt_to_phys(struct xlink_buf_mem *xlink_mem, void *vaddr,
+ u32 *paddr)
+{
+ if (unlikely((xlink_mem->dma_handle + xlink_mem->size) > 0xFFFFFFFF))
+ return -EINVAL;
+ if (unlikely(vaddr < xlink_mem->vaddr ||
+ vaddr >= (xlink_mem->vaddr + xlink_mem->size)))
+ return -EINVAL;
+ *paddr = xlink_mem->dma_handle + (vaddr - xlink_mem->vaddr);
+
+ return 0;
+}
+
+/*
+ * get_next_xlink_buf() - Get next xlink buffer from an xlink device's pool.
+ *
+ * @xlink_dev: [in] The xlink ipc device to get a buffer from.
+ * @buf: [out] Where to store the reference to the next buffer.
+ * @size: [in] The size of the buffer to get.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int get_next_xlink_buf(struct xlink_ipc_dev *xlink_dev, void **buf,
+ int size)
+{
+ struct xlink_buf_pool *pool;
+ unsigned long flags;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ if (size > XLINK_MAX_BUF_SIZE)
+ return -EINVAL;
+
+ pool = &xlink_dev->xlink_buf_pool;
+
+ spin_lock_irqsave(&pool->lock, flags);
+ if (pool->idx == pool->buf_cnt) {
+ /* reached end of buffers - wrap around */
+ pool->idx = 0;
+ }
+ *buf = pool->buf + (pool->idx * XLINK_MAX_BUF_SIZE);
+ pool->idx++;
+ spin_unlock_irqrestore(&pool->lock, flags);
+ return 0;
+}
+
+/*
+ * Functions related to the vpu ipc device reference.
+ */
+
+/*
+ * vpu_ipc_device_put() - Release the vpu ipc device held by the xlink device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ */
+static void vpu_ipc_device_put(struct xlink_ipc_dev *xlink_dev)
+{
+ put_device(xlink_dev->vpu_dev);
+}
+
+/*
+ * vpu_ipc_device_get() - Get the vpu ipc device reference for the xlink device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int vpu_ipc_device_get(struct xlink_ipc_dev *xlink_dev)
+{
+ struct device *dev = &xlink_dev->pdev->dev;
+ struct platform_device *pdev;
+ struct device_node *np;
+
+ np = of_parse_phandle(dev->of_node, "intel,keembay-vpu-ipc", 0);
+ if (!np)
+ return -ENODEV;
+
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ dev_info(dev, "IPC device not probed\n");
+ of_node_put(np);
+ return -EPROBE_DEFER;
+ }
+
+ xlink_dev->vpu_dev = get_device(&pdev->dev);
+ of_node_put(np);
+
+ dev_info(dev, "Using IPC device: %s\n", dev_name(xlink_dev->vpu_dev));
+ return 0;
+}
+
+/*
+ * xlink platform api - ipc interface functions
+ */
+
+/*
+ * xlink_ipc_connect() - platform connect interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to connect to.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_connect(u32 sw_device_id)
+{
+ if (!xlink_dev)
+ return -ENODEV;
+
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_connect);
+
+/*
+ * xlink_ipc_write() - platform write interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to write to.
+ * @data: [in] The data buffer to write.
+ * @size: [in-out] The amount of data to write/written.
+ * @timeout: [in] The time (in ms) to wait before timing out.
+ * @context: [in] The ipc operation context.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_write(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context)
+{
+ struct xlink_ipc_context *ctx = context;
+ void *vaddr = NULL;
+ u32 paddr;
+ int rc;
+
+ if (!ctx)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ if (ctx->is_volatile) {
+ rc = get_next_xlink_buf(xlink_dev, &vaddr, XLINK_MAX_BUF_SIZE);
+ if (rc)
+ return rc;
+ memcpy(vaddr, data, *size);
+ rc = xlink_virt_to_phys(&xlink_dev->local_xlink_mem, vaddr,
+ &paddr);
+ if (rc)
+ return rc;
+ } else {
+ paddr = *(u32 *)data;
+ }
+ rc = intel_keembay_vpu_ipc_send(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, ctx->chan,
+ paddr, *size);
+
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_write);
+
+/*
+ * xlink_ipc_read() - platform read interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to read from.
+ * @data: [out] The data buffer to read into.
+ * @size: [in-out] The amount of data to read/was read.
+ * @timeout: [in] The time (in ms) to wait before timing out.
+ * @context: [in] The ipc operation context.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_read(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context)
+{
+ struct xlink_ipc_context *ctx = context;
+ u32 addr = 0;
+ void *vaddr;
+ int rc;
+
+ if (!ctx)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_recv(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, ctx->chan,
+ &addr, size, timeout);
+
+ if (ctx->is_volatile) {
+ vaddr = xlink_phys_to_virt(&xlink_dev->remote_xlink_mem, addr);
+ if (vaddr)
+ memcpy(data, vaddr, *size);
+ else
+ return -ENXIO;
+ } else {
+ *(u32 *)data = addr;
+ }
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_read);
+
+/*
+ * xlink_ipc_get_device_list() - platform get device list interface.
+ *
+ * @sw_device_id_list: [out] The list of devices found.
+ * @num_devices: [out] The number of devices found.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_list(u32 *sw_device_id_list, u32 *num_devices)
+{
+ int i = 0;
+
+ if (!sw_device_id_list || !num_devices)
+ return -EINVAL;
+
+ if (xlink_dev) {
+ *sw_device_id_list = xlink_dev->sw_device_id;
+ i++;
+ }
+
+ *num_devices = i;
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_list);
+
+/*
+ * xlink_ipc_get_device_name() - platform get device name interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to get name of.
+ * @device_name: [out] The name of the xlink ipc device.
+ * @name_size: [in] The maximum size of the name.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size)
+{
+ size_t size;
+
+ if (!device_name)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ size = (name_size > XLINK_IPC_MAX_DEVICE_NAME_SIZE)
+ ? XLINK_IPC_MAX_DEVICE_NAME_SIZE
+ : name_size;
+ strncpy(device_name, xlink_dev->device_name, size);
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_name);
+
+/*
+ * xlink_ipc_get_device_status() - platform get device status interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ * @device_status: [out] The device status.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_status(u32 sw_device_id, u32 *device_status)
+{
+ if (!device_status)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ *device_status = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_status);
+
+static void kernel_callback(struct device *dev, enum intel_keembay_vpu_event event)
+{
+ if ((enum xlink_vpu_event)event >= NUM_EVENT_TYPE)
+ return;
+
+ if (xlink_dev) {
+ if (xlink_dev->callback)
+ xlink_dev->callback(xlink_dev->sw_device_id, event);
+ }
+}
+
+/*
+ * xlink_ipc_register_for_events() - platform register for events
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ * @callback: [in] Callback function for events
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_register_for_events(u32 sw_device_id,
+ int (*callback)(u32 sw_device_id, enum xlink_vpu_event event))
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+ xlink_dev->callback = callback;
+ rc = intel_keembay_vpu_register_for_events(xlink_dev->vpu_dev, kernel_callback);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_register_for_events);
+/*
+ * xlink_ipc_unregister_for_events() - platform register for events
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_unregister_for_events(u32 sw_device_id)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+ rc = intel_keembay_vpu_unregister_for_events(xlink_dev->vpu_dev);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_unregister_for_events);
+
+/*
+ * xlink_ipc_boot_device() - platform boot device interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to boot.
+ * @binary_name: [in] The file name of the firmware binary to boot.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_boot_device(u32 sw_device_id, const char *binary_name)
+{
+ enum intel_keembay_vpu_state state;
+ int rc;
+
+ if (!binary_name)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ pr_info("\nStart VPU 0x%x - %s\n", sw_device_id, binary_name);
+ rc = intel_keembay_vpu_startup(xlink_dev->vpu_dev, binary_name);
+ if (rc) {
+ pr_err("Failed to start VPU: %d\n", rc);
+ return -EBUSY;
+ }
+ pr_info("Successfully started VPU!\n");
+
+ /* Wait for VPU to be READY */
+ rc = intel_keembay_vpu_wait_for_ready(xlink_dev->vpu_dev,
+ XLINK_VPU_WAIT_FOR_READY_MS);
+ if (rc) {
+ pr_err("Tried to start VPU but never got READY.\n");
+ return -EBUSY;
+ }
+ pr_info("Successfully synchronised state with VPU!\n");
+
+ /* Check state */
+ state = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ if (state != KEEMBAY_VPU_READY) {
+ pr_err("VPU was not ready, it was %d\n", state);
+ return -EBUSY;
+ }
+ pr_info("VPU was ready.\n");
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_boot_device);
+
+/*
+ * xlink_ipc_reset_device() - platform reset device interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to reset.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_reset_device(u32 sw_device_id)
+{
+ enum intel_keembay_vpu_state state;
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ /* stop the vpu */
+ rc = intel_keembay_vpu_stop(xlink_dev->vpu_dev);
+ if (rc) {
+ pr_err("Failed to stop VPU: %d\n", rc);
+ return -EBUSY;
+ }
+ pr_info("Successfully stopped VPU!\n");
+
+ /* check state */
+ state = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ if (state != KEEMBAY_VPU_OFF) {
+ pr_err("VPU was not OFF after stop request, it was %d\n",
+ state);
+ return -EBUSY;
+ }
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_reset_device);
+
+/*
+ * xlink_ipc_open_channel() - platform open channel interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to open channel to.
+ * @channel: [in] The channel id to open.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_open_channel(u32 sw_device_id, u32 channel)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_open_channel(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, channel);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_open_channel);
+
+/*
+ * xlink_ipc_close_channel() - platform close channel interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to close channel to.
+ * @channel: [in] The channel id to close.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_close_channel(u32 sw_device_id, u32 channel)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_close_channel(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, channel);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_close_channel);
+
+/*
+ * xlink ipc driver functions
+ */
+
+static int keembay_xlink_ipc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ int rc;
+
+ /* allocate device data structure */
+ xlink_dev = kzalloc(sizeof(*xlink_dev), GFP_KERNEL);
+ if (!xlink_dev)
+ return -ENOMEM;
+
+ xlink_dev->pdev = pdev;
+ dev_info(dev, "Keem Bay xlink IPC driver probed.\n");
+
+ /* grab reserved memory regions and assign to child devices */
+ rc = xlink_reserved_memory_init(xlink_dev);
+ if (rc < 0) {
+ dev_err(&pdev->dev,
+ "Failed to set up reserved memory regions.\n");
+ goto r_cleanup;
+ }
+
+ /* allocate memory from the reserved memory regions */
+ rc = xlink_reserved_memory_alloc(xlink_dev);
+ if (rc < 0) {
+ dev_err(&pdev->dev,
+ "Failed to allocate reserved memory regions.\n");
+ goto r_cleanup;
+ }
+
+ /* init the xlink buffer pool used for rx/tx */
+ init_xlink_buf_pool(xlink_dev);
+
+ /* get reference to vpu ipc device */
+ rc = vpu_ipc_device_get(xlink_dev);
+ if (rc)
+ goto r_cleanup;
+
+ /* get device id */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-id",
+ &xlink_dev->vpu_id);
+ if (rc) {
+ dev_err(dev, "Cannot get VPU ID from DT.\n");
+ goto r_cleanup;
+ }
+
+ /* assign a sw device id */
+ xlink_dev->sw_device_id = GET_SW_DEVICE_ID_FROM_VPU_ID
+ (xlink_dev->vpu_id);
+
+ /* assign a device name */
+ rc = of_property_read_string(dev->of_node, "intel,keembay-vpu-ipc-name",
+ &xlink_dev->device_name);
+ if (rc) {
+ /* only warn for now; we will enforce this in the future */
+ dev_warn(dev, "VPU name not defined in DT, using %s as default.\n",
+ VPU_DEVICE_NAME);
+ dev_warn(dev, "WARNING: additional VPU devices may fail probing.\n");
+ xlink_dev->device_name = VPU_DEVICE_NAME;
+ }
+
+ /* get platform data reference */
+ platform_set_drvdata(pdev, xlink_dev);
+
+ dev_info(dev, "Device id=%u sw_device_id=0x%x name=%s probe complete.\n",
+ xlink_dev->vpu_id, xlink_dev->sw_device_id,
+ xlink_dev->device_name);
+ return 0;
+
+r_cleanup:
+ xlink_reserved_memory_remove(xlink_dev);
+ return rc;
+}
+
+static int keembay_xlink_ipc_remove(struct platform_device *pdev)
+{
+ struct xlink_ipc_dev *xlink_dev = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+
+ /*
+ * no need to de-alloc xlink mem (local_xlink_mem and remote_xlink_mem)
+ * since it was allocated with dmam_alloc
+ */
+ xlink_reserved_memory_remove(xlink_dev);
+
+ /* release vpu ipc device */
+ vpu_ipc_device_put(xlink_dev);
+
+ dev_info(dev, "Keem Bay xlink IPC driver removed.\n");
+ return 0;
+}
+
+static const struct of_device_id keembay_xlink_ipc_of_match[] = {
+ {
+ .compatible = "intel,keembay-xlink-ipc",
+ },
+ {}
+};
+
+static struct platform_driver keembay_xlink_ipc_driver = {
+ .driver = {
+ .name = "keembay-xlink-ipc",
+ .of_match_table = keembay_xlink_ipc_of_match,
+ },
+ .probe = keembay_xlink_ipc_probe,
+ .remove = keembay_xlink_ipc_remove,
+};
+module_platform_driver(keembay_xlink_ipc_driver);
+
+MODULE_DESCRIPTION("Keem Bay xlink IPC Driver");
+MODULE_AUTHOR("Ryan Carnaghi <ryan.r.carnaghi@intel.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/include/linux/xlink-ipc.h b/include/linux/xlink-ipc.h
new file mode 100644
index 000000000000..f26b53bf6506
--- /dev/null
+++ b/include/linux/xlink-ipc.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay xlink IPC Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef _XLINK_IPC_H_
+#define _XLINK_IPC_H_
+
+#include <linux/types.h>
+
+struct xlink_ipc_context {
+ u16 chan;
+ u8 is_volatile;
+};
+
+int xlink_ipc_connect(u32 sw_device_id);
+
+int xlink_ipc_write(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context);
+
+int xlink_ipc_read(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context);
+
+int xlink_ipc_get_device_list(u32 *sw_device_id_list, u32 *num_devices);
+
+int xlink_ipc_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size);
+
+int xlink_ipc_get_device_status(u32 sw_device_id, u32 *device_status);
+
+int xlink_ipc_boot_device(u32 sw_device_id, const char *binary_name);
+
+int xlink_ipc_reset_device(u32 sw_device_id);
+
+int xlink_ipc_open_channel(u32 sw_device_id, u32 channel);
+
+int xlink_ipc_close_channel(u32 sw_device_id, u32 channel);
+
+int xlink_ipc_register_for_events(u32 sw_device_id,
+ int (*callback)(u32 sw_device_id, u32 event));
+
+int xlink_ipc_unregister_for_events(u32 sw_device_id);
+
+#endif /* _XLINK_IPC_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 17/22] xlink-core: Add xlink core device tree bindings
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (15 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 16/22] xlink-ipc: Add xlink ipc driver mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 18/22] xlink-core: Add xlink core driver xLink mgross
` (5 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly, Rob Herring,
devicetree, Ryan Carnaghi
From: Seamus Kelly <seamus.kelly@intel.com>
Add device tree bindings for keembay-xlink.
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
Signed-off-by: Ryan Carnaghi <ryan.r.carnaghi@intel.com>
---
.../bindings/misc/intel,keembay-xlink.yaml | 27 +++++++++++++++++++
1 file changed, 27 insertions(+)
create mode 100644 Documentation/devicetree/bindings/misc/intel,keembay-xlink.yaml
diff --git a/Documentation/devicetree/bindings/misc/intel,keembay-xlink.yaml b/Documentation/devicetree/bindings/misc/intel,keembay-xlink.yaml
new file mode 100644
index 000000000000..89c34018fa04
--- /dev/null
+++ b/Documentation/devicetree/bindings/misc/intel,keembay-xlink.yaml
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright (c) Intel Corporation. All rights reserved.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/misc/intel,keembay-xlink.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel Keem Bay xlink
+
+maintainers:
+ - Seamus Kelly <seamus.kelly@intel.com>
+
+description: |
+ The Keem Bay xlink driver enables the communication/control sub-system
+ for internal and external communications to the Intel Keem Bay SoC.
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: intel,keembay-xlink
+
+examples:
+ - |
+ xlink {
+ compatible = "intel,keembay-xlink";
+ };
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 18/22] xlink-core: Add xlink core driver xLink
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (16 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 17/22] xlink-core: Add xlink core device tree bindings mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 19/22] xlink-core: Enable xlink protocol over pcie mgross
` (4 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Jonathan Corbet, Derek Kiernan, Dragan Cvetic, Arnd Bergmann,
Greg Kroah-Hartman, linux-doc
From: Seamus Kelly <seamus.kelly@intel.com>
Add xLink driver, which provides an abstracted control and communication
subsystem based on channel identification.
It is intended to support VPU technology both at SoC level as well as at
IP level, over multiple interfaces. This initial patch enables local host
user mode to open/close/read/write via IOCTLs.
Specifically the driver enables application/process to:
* Access a common xLink API across all interfaces from both kernel and
user space.
* Call typical APIs types (open, close, read, write) that you would
associate with a communication interface.
* Call other APIs that are related to other functions that the
device can perform e.g. boot, reset get/set device mode. Device mode
refers to the power load of the VPU and an API can be used to read and
control it.
* Use multiple commnication channels that the driver manages from one
interface to another, providing routing of data through these multiple
channels across a single physical interface.
xLink: Add xLink Core device tree bindings
Add device tree bindings for the xLink Core driver which enables xLink
to control and communicate with the VPU IP present on the Intel Keem Bay
SoC.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Derek Kiernan <derek.kiernan@xilinx.com>
Cc: Dragan Cvetic <dragan.cvetic@xilinx.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-doc@vger.kernel.org
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
---
Documentation/vpu/index.rst | 1 +
Documentation/vpu/xlink-core.rst | 80 ++
MAINTAINERS | 12 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/xlink-core/Kconfig | 33 +
drivers/misc/xlink-core/Makefile | 5 +
drivers/misc/xlink-core/xlink-core.c | 870 ++++++++++++++++++++
drivers/misc/xlink-core/xlink-defs.h | 175 ++++
drivers/misc/xlink-core/xlink-multiplexer.c | 534 ++++++++++++
drivers/misc/xlink-core/xlink-multiplexer.h | 35 +
drivers/misc/xlink-core/xlink-platform.c | 160 ++++
drivers/misc/xlink-core/xlink-platform.h | 65 ++
include/linux/xlink.h | 108 +++
include/uapi/misc/xlink_uapi.h | 145 ++++
15 files changed, 2225 insertions(+)
create mode 100644 Documentation/vpu/xlink-core.rst
create mode 100644 drivers/misc/xlink-core/Kconfig
create mode 100644 drivers/misc/xlink-core/Makefile
create mode 100644 drivers/misc/xlink-core/xlink-core.c
create mode 100644 drivers/misc/xlink-core/xlink-defs.h
create mode 100644 drivers/misc/xlink-core/xlink-multiplexer.c
create mode 100644 drivers/misc/xlink-core/xlink-multiplexer.h
create mode 100644 drivers/misc/xlink-core/xlink-platform.c
create mode 100644 drivers/misc/xlink-core/xlink-platform.h
create mode 100644 include/linux/xlink.h
create mode 100644 include/uapi/misc/xlink_uapi.h
diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
index 49c78bb65b83..cd4272e089ec 100644
--- a/Documentation/vpu/index.rst
+++ b/Documentation/vpu/index.rst
@@ -16,3 +16,4 @@ This documentation contains information for the Intel VPU stack.
vpu-stack-overview
xlink-pcie
xlink-ipc
+ xlink-core
diff --git a/Documentation/vpu/xlink-core.rst b/Documentation/vpu/xlink-core.rst
new file mode 100644
index 000000000000..5410755ff13a
--- /dev/null
+++ b/Documentation/vpu/xlink-core.rst
@@ -0,0 +1,80 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+xLink-core software sub-system
+==============================
+
+The purpose of the xLink software sub-system is to facilitate communication
+between multiple users on multiple nodes in the system.
+
+There are three types of xLink nodes:
+
+1. Remote Host: this is an external IA/x86 host system that is only capable of
+ communicating directly to the Local Host node on VPU 2.x products.
+2. Local Host: this is the ARM core within the VPU 2.x SoC. The Local Host can
+ communicate upstream to the Remote Host node, and downstream to the VPU IP
+ node.
+3. VPU IP: this is the Leon RT core within the VPU 2.x SoC. The VPU IP can only
+ communicate upstream to the Local Host node.
+
+xLink provides a common API across all interfaces for users to access xLink
+functions and provides user space APIs via an IOCTL interface implemented in
+the xLink core.
+
+xLink manages communications from one interface to another and provides routing
+of data through multiple channels across a single physical interface.
+
+It exposes a common API across all interfaces at both kernel and user level for
+processes/applications to access.
+
+It has typical APIs types (open, close, read, write) that you would associate
+with a communication interface.
+
+It also has other APIs that are related to other functions that the device can
+perform e.g. boot, reset get/set device mode.
+The driver is broken down into 4 source files.
+
+xlink-core:
+Contains driver initialization, driver API and IOCTL interface (for user
+space).
+
+xlink-multiplexer:
+The Multiplexer component is responsible for securely routing messages through
+multiple communication channels over a single physical interface.
+
+xlink-dispatcher:
+The Dispatcher component is responsible for queueing and handling xLink
+communication requests from all users in the system and invoking the underlying
+platform interface drivers.
+
+xlink-platform:
+provides abstraction to each interface supported (PCIe, USB, IPC, etc).
+
+Typical xLink transaction (simplified):
+When a user wants to send data across an interface via xLink it firstly calls
+xlink connect which connects to the relevant interface (PCIe, USB,IPC,etc) and
+then xlink open channel.
+
+Then it calls xlink write function, this takes the data passes it to the kernel
+which packages up the data and channel and then adds it to a tx transaction
+queue.
+
+A separate thread reads this transaction queue and pops off data if available
+and passes the data to the underlying interface (e.g. PCIe) write function.
+Using this thread provides serialization of transactions and decouples the user
+write from the platform write.
+
+On the other side of the interface, a thread is continually reading the
+interface (e.g. PCIe) via the platform interface read function and if it reads
+any data it adds it to channel packet container.
+
+The application at this side of the interface will have called xlink connect,
+opened the channel and called xlink read function to read data from the
+interface and if any exists for that channel , the data gets popped from the
+channel packet container and copied from kernel space to user space buffer
+provided by the call.
+
+xLink can handle API requests from multi-process and multi-threaded
+application/processes.
+
+xLink maintains 4096 channels per device connected ( via xlink connect ) and
+maintains a separate channel infrastructure for each device.
diff --git a/MAINTAINERS b/MAINTAINERS
index ebc2cce7251f..0d49622aa1a6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1952,6 +1952,12 @@ F: Documentation/devicetree/bindings/arm/intel,keembay.yaml
F: arch/arm64/boot/dts/intel/keembay-evm.dts
F: arch/arm64/boot/dts/intel/keembay-soc.dtsi
+ARM/INTEL XLINK CORE SUPPORT
+M: Seamus Kelly <seamus.kelly@intel.com>
+M: Mark Gross <mgross@linux.intel.com>
+S: Maintained
+F: drivers/misc/xlink-core/
+
ARM/INTEL XLINK IPC SUPPORT
M: Seamus Kelly <seamus.kelly@intel.com>
M: Mark Gross <mgross@linux.intel.com>
@@ -1964,6 +1970,12 @@ M: Mark Gross <mgross@linux.intel.com>
S: Maintained
F: drivers/misc/xlink-pcie/
+ARM/INTEL TSENS SUPPORT
+M: Udhayakumar C <udhayakumar.c@intel.com>
+S: Maintained
+F: drivers/misc/hddl_device/
+F: drivers/misc/intel_tsens/
+
ARM/INTEL RESEARCH IMOTE/STARGATE 2 MACHINE SUPPORT
M: Jonathan Cameron <jic23@cam.ac.uk>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index 1f81ea915b95..09ae65e98681 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -483,4 +483,5 @@ source "drivers/misc/habanalabs/Kconfig"
source "drivers/misc/uacce/Kconfig"
source "drivers/misc/xlink-pcie/Kconfig"
source "drivers/misc/xlink-ipc/Kconfig"
+source "drivers/misc/xlink-core/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index b51495a2f1e0..f3a6eb03bae9 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -59,3 +59,4 @@ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
obj-y += xlink-pcie/
obj-$(CONFIG_XLINK_IPC) += xlink-ipc/
+obj-$(CONFIG_XLINK_CORE) += xlink-core/
diff --git a/drivers/misc/xlink-core/Kconfig b/drivers/misc/xlink-core/Kconfig
new file mode 100644
index 000000000000..3ae7dc01007f
--- /dev/null
+++ b/drivers/misc/xlink-core/Kconfig
@@ -0,0 +1,33 @@
+config XLINK_CORE
+ tristate "Support for XLINK CORE"
+ depends on ((XLINK_PCIE_RH_DRIVER || XBAY_XLINK_PCIE_RH_DRIVER) || (XLINK_LOCAL_HOST && (XLINK_PCIE_LH_DRIVER || XBAY_XLINK_PCIE_RH_DRIVER)))
+ help
+ XLINK CORE enables the communication/control Sub-System.
+
+ If unsure, say N.
+
+ To compile this driver as a module, choose M here: the
+ module will be called xlink.ko.
+
+config XLINK_LOCAL_HOST
+ tristate "Support for XLINK LOCAL HOST"
+ depends on XLINK_IPC
+ help
+ XLINK LOCAL HOST enables local host functionality for
+ the communication/control Sub-System.
+
+ Enable this config when building the kernel for the Inel Vision
+ Processing Unit (VPU) Local Host core.
+
+ If building for a Remote Host kernel, say N.
+
+config XLINK_PSS
+ tristate "Support for XLINK PSS"
+ depends on XLINK_LOCAL_HOST
+ help
+ XLINK PSS enables the communication/control Sub-System on a PSS platform.
+
+ Enable this config when building the kernel for the Intel Vision
+ Processing Unit (VPU) in a simulated env.
+
+ If building for a VPU silicon, say N.
diff --git a/drivers/misc/xlink-core/Makefile b/drivers/misc/xlink-core/Makefile
new file mode 100644
index 000000000000..6b708340dbfe
--- /dev/null
+++ b/drivers/misc/xlink-core/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for KeemBay xlink Linux driver
+#
+obj-$(CONFIG_XLINK_CORE) += xlink.o
+xlink-objs += xlink-core.o xlink-multiplexer.o xlink-platform.o
diff --git a/drivers/misc/xlink-core/xlink-core.c b/drivers/misc/xlink-core/xlink-core.c
new file mode 100644
index 000000000000..3813208ffa95
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-core.c
@@ -0,0 +1,870 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Core Driver.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/fs.h>
+#include <linux/cdev.h>
+#include <linux/platform_device.h>
+#include <linux/mod_devicetable.h>
+#include <linux/uaccess.h>
+#include <linux/slab.h>
+#include <linux/kref.h>
+
+#ifdef CONFIG_XLINK_LOCAL_HOST
+#include <linux/xlink-ipc.h>
+#endif
+
+#include "xlink-defs.h"
+#include "xlink-multiplexer.h"
+#include "xlink-platform.h"
+
+// xlink version number
+#define XLINK_VERSION_MAJOR 0
+#define XLINK_VERSION_MINOR 1
+#define XLINK_VERSION_REVISION 2
+#define XLINK_VERSION_SUB_REV "a"
+
+// timeout in milliseconds used to wait for the reay message from the VPU
+#ifdef CONFIG_XLINK_PSS
+#define XLINK_VPU_WAIT_FOR_READY (3000000)
+#else
+#define XLINK_VPU_WAIT_FOR_READY (3000)
+#endif
+
+// device, class, and driver names
+#define DEVICE_NAME "xlnk"
+#define CLASS_NAME "xlkcore"
+#define DRV_NAME "xlink-driver"
+
+// used to determine if an API was called from user or kernel space
+#define CHANNEL_SET_USER_BIT(chan) ((chan) |= (1 << 15))
+#define CHANNEL_USER_BIT_IS_SET(chan) ((chan) & (1 << 15))
+#define CHANNEL_CLEAR_USER_BIT(chan) ((chan) &= ~(1 << 15))
+
+static dev_t xdev;
+static struct class *dev_class;
+static struct cdev xlink_cdev;
+
+static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
+static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage,
+ u32 size);
+
+static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size);
+static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size, u32 user_flag);
+
+static const struct file_operations fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = xlink_ioctl,
+};
+
+struct xlink_link {
+ u32 id;
+ struct xlink_handle handle;
+ struct kref refcount;
+};
+
+struct keembay_xlink_dev {
+ struct platform_device *pdev;
+ struct xlink_link links[XLINK_MAX_CONNECTIONS];
+ u32 nmb_connected_links;
+ struct mutex lock; // protect access to xlink_dev
+};
+
+/*
+ * global variable pointing to our xlink device.
+ *
+ * This is meant to be used only when platform_get_drvdata() cannot be used
+ * because we lack a reference to our platform_device.
+ */
+static struct keembay_xlink_dev *xlink;
+
+/*
+ * get_next_link() - Searches the list of links to find the next available.
+ *
+ * Note: This function is only used in xlink_connect, where the xlink mutex is
+ * already locked.
+ *
+ * Return: the next available link, or NULL if maximum connections reached.
+ */
+static struct xlink_link *get_next_link(void)
+{
+ struct xlink_link *link = NULL;
+ int i;
+
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
+ if (xlink->links[i].handle.sw_device_id == XLINK_INVALID_SW_DEVICE_ID) {
+ link = &xlink->links[i];
+ break;
+ }
+ }
+ return link;
+}
+
+/*
+ * get_link_by_sw_device_id()
+ *
+ * Searches the list of links to find a link by sw device id.
+ *
+ * Return: the handle, or NULL if the handle is not found.
+ */
+static struct xlink_link *get_link_by_sw_device_id(u32 sw_device_id)
+{
+ struct xlink_link *link = NULL;
+ int i;
+
+ mutex_lock(&xlink->lock);
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
+ if (xlink->links[i].handle.sw_device_id == sw_device_id) {
+ link = &xlink->links[i];
+ break;
+ }
+ }
+ mutex_unlock(&xlink->lock);
+ return link;
+}
+
+// For now , do nothing and leave for further consideration
+static void release_after_kref_put(struct kref *ref) {}
+
+/* Driver probing. */
+static int kmb_xlink_probe(struct platform_device *pdev)
+{
+ struct keembay_xlink_dev *xlink_dev;
+ struct device *dev_ret;
+ int rc, i;
+
+ dev_info(&pdev->dev, "KeemBay xlink v%d.%d.%d:%s\n", XLINK_VERSION_MAJOR,
+ XLINK_VERSION_MINOR, XLINK_VERSION_REVISION, XLINK_VERSION_SUB_REV);
+
+ xlink_dev = devm_kzalloc(&pdev->dev, sizeof(*xlink), GFP_KERNEL);
+ if (!xlink_dev)
+ return -ENOMEM;
+
+ xlink_dev->pdev = pdev;
+
+ // initialize multiplexer
+ rc = xlink_multiplexer_init(xlink_dev->pdev);
+ if (rc != X_LINK_SUCCESS) {
+ pr_err("Multiplexer initialization failed\n");
+ goto r_multiplexer;
+ }
+
+ // initialize xlink data structure
+ xlink_dev->nmb_connected_links = 0;
+ mutex_init(&xlink_dev->lock);
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
+ xlink_dev->links[i].id = i;
+ xlink_dev->links[i].handle.sw_device_id =
+ XLINK_INVALID_SW_DEVICE_ID;
+ }
+
+ platform_set_drvdata(pdev, xlink_dev);
+
+ /* Set the global reference to our device. */
+ xlink = xlink_dev;
+
+ /*Allocating Major number*/
+ if ((alloc_chrdev_region(&xdev, 0, 1, "xlinkdev")) < 0) {
+ dev_info(&pdev->dev, "Cannot allocate major number\n");
+ goto r_multiplexer;
+ }
+ dev_info(&pdev->dev, "Major = %d Minor = %d\n", MAJOR(xdev),
+ MINOR(xdev));
+
+ /*Creating struct class*/
+ dev_class = class_create(THIS_MODULE, CLASS_NAME);
+ if (IS_ERR(dev_class)) {
+ dev_info(&pdev->dev, "Cannot create the struct class - Err %ld\n",
+ PTR_ERR(dev_class));
+ goto r_class;
+ }
+
+ /*Creating device*/
+ dev_ret = device_create(dev_class, NULL, xdev, NULL, DEVICE_NAME);
+ if (IS_ERR(dev_ret)) {
+ dev_info(&pdev->dev, "Cannot create the Device 1 - Err %ld\n",
+ PTR_ERR(dev_ret));
+ goto r_device;
+ }
+ dev_info(&pdev->dev, "Device Driver Insert...Done!!!\n");
+
+ /*Creating cdev structure*/
+ cdev_init(&xlink_cdev, &fops);
+
+ /*Adding character device to the system*/
+ if ((cdev_add(&xlink_cdev, xdev, 1)) < 0) {
+ dev_info(&pdev->dev, "Cannot add the device to the system\n");
+ goto r_class;
+ }
+
+ return 0;
+
+r_device:
+ class_destroy(dev_class);
+r_class:
+ unregister_chrdev_region(xdev, 1);
+r_multiplexer:
+ xlink_multiplexer_destroy();
+ return -1;
+}
+
+/* Driver removal. */
+static int kmb_xlink_remove(struct platform_device *pdev)
+{
+ int rc;
+
+ mutex_lock(&xlink->lock);
+ // destroy multiplexer
+ rc = xlink_multiplexer_destroy();
+ if (rc != X_LINK_SUCCESS)
+ pr_err("Multiplexer destroy failed\n");
+
+ mutex_unlock(&xlink->lock);
+ mutex_destroy(&xlink->lock);
+ // unregister and destroy device
+ unregister_chrdev_region(xdev, 1);
+ device_destroy(dev_class, xdev);
+ cdev_del(&xlink_cdev);
+ class_destroy(dev_class);
+ pr_info("XLink Driver removed\n");
+ return 0;
+}
+
+/*
+ * IOCTL function for User Space access to xlink kernel functions
+ *
+ */
+
+static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ int interface = NULL_INTERFACE;
+ struct xlink_handle devh = {0};
+ struct xlinkopenchannel op = {0};
+ struct xlinkwritedata wr = {0};
+ struct xlinkreaddata rd = {0};
+ struct xlinkconnect con = {0};
+ struct xlinkrelease rel = {0};
+ u8 volbuf[XLINK_MAX_BUF_SIZE];
+ u8 reladdr;
+ u8 *rdaddr;
+ u32 size;
+ int rc;
+
+ switch (cmd) {
+ case XL_CONNECT:
+ if (copy_from_user(&con, (void __user *)arg,
+ sizeof(struct xlinkconnect)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)con.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_connect(&devh);
+ if (rc == X_LINK_SUCCESS) {
+ if (copy_to_user((void __user *)con.handle,
+ &devh, sizeof(struct xlink_handle)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)con.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_OPEN_CHANNEL:
+ if (copy_from_user(&op, (void __user *)arg,
+ sizeof(struct xlinkopenchannel)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)op.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_open_channel(&devh, op.chan, op.mode, op.data_size,
+ op.timeout);
+ if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_READ_DATA:
+ if (copy_from_user(&rd, (void __user *)arg,
+ sizeof(struct xlinkreaddata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rd.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_read_data(&devh, rd.chan, &rdaddr, &size);
+ if (!rc) {
+ interface = get_interface_from_sw_device_id(devh.sw_device_id);
+ if (interface == IPC_INTERFACE) {
+ if (copy_to_user((void __user *)rd.pmessage, (void *)&rdaddr,
+ sizeof(u32)))
+ return -EFAULT;
+ } else {
+ if (copy_to_user((void __user *)rd.pmessage, (void *)rdaddr,
+ size))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rd.size, (void *)&size, sizeof(size)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rd.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_WRITE_DATA:
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_DATA_SIZE) {
+ rc = xlink_write_data_user(&devh, wr.chan, wr.pmessage,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ break;
+ case XL_WRITE_VOLATILE:
+ if (copy_from_user(&wr, (void __user *)arg, sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_BUF_SIZE) {
+ if (copy_from_user(volbuf, (void __user *)wr.pmessage, wr.size))
+ return -EFAULT;
+ rc = xlink_write_volatile_user(&devh, wr.chan, volbuf, wr.size);
+ if (copy_to_user((void __user *)wr.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ break;
+ case XL_RELEASE_DATA:
+ if (copy_from_user(&rel, (void __user *)arg,
+ sizeof(struct xlinkrelease)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rel.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (rel.addr) {
+ if (get_user(reladdr, (u32 __user *const)rel.addr))
+ return -EFAULT;
+ rc = xlink_release_data(&devh, rel.chan,
+ (u8 *)&reladdr);
+ } else {
+ rc = xlink_release_data(&devh, rel.chan, NULL);
+ }
+ if (copy_to_user((void __user *)rel.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_CLOSE_CHANNEL:
+ if (copy_from_user(&op, (void __user *)arg,
+ sizeof(struct xlinkopenchannel)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)op.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_close_channel(&devh, op.chan);
+ if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_DISCONNECT:
+ if (copy_from_user(&con, (void __user *)arg,
+ sizeof(struct xlinkconnect)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)con.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_disconnect(&devh);
+ if (copy_to_user((void __user *)con.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ }
+ if (rc)
+ return -EIO;
+ else
+ return 0;
+}
+
+/*
+ * xlink Kernel API.
+ */
+
+enum xlink_error xlink_initialize(void)
+{
+ return X_LINK_SUCCESS;
+}
+EXPORT_SYMBOL(xlink_initialize);
+
+enum xlink_error xlink_connect(struct xlink_handle *handle)
+{
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ mutex_lock(&xlink->lock);
+ if (!link) {
+ link = get_next_link();
+ if (!link) {
+ pr_err("max connections reached %d\n",
+ XLINK_MAX_CONNECTIONS);
+ mutex_unlock(&xlink->lock);
+ return X_LINK_ERROR;
+ }
+ // platform connect
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ rc = xlink_platform_connect(interface, handle->sw_device_id);
+ if (rc) {
+ pr_err("platform connect failed %d\n", rc);
+ mutex_unlock(&xlink->lock);
+ return X_LINK_ERROR;
+ }
+ // set link handle reference and link id
+ link->handle = *handle;
+ xlink->nmb_connected_links++;
+ kref_init(&link->refcount);
+ // initialize multiplexer connection
+ rc = xlink_multiplexer_connect(link->id);
+ if (rc) {
+ pr_err("multiplexer connect failed\n");
+ goto r_cleanup;
+ }
+ pr_info("dev 0x%x connected - dev_type %d - nmb_connected_links %d\n",
+ link->handle.sw_device_id,
+ link->handle.dev_type,
+ xlink->nmb_connected_links);
+ } else {
+ // already connected
+ pr_info("dev 0x%x ALREADY connected - dev_type %d\n",
+ link->handle.sw_device_id,
+ link->handle.dev_type);
+ kref_get(&link->refcount);
+ *handle = link->handle;
+ }
+ mutex_unlock(&xlink->lock);
+ // TODO: implement ping
+ return X_LINK_SUCCESS;
+
+r_cleanup:
+ link->handle.sw_device_id = XLINK_INVALID_SW_DEVICE_ID;
+ mutex_unlock(&xlink->lock);
+ return X_LINK_ERROR;
+}
+EXPORT_SYMBOL(xlink_connect);
+
+enum xlink_error xlink_open_channel(struct xlink_handle *handle,
+ u16 chan, enum xlink_opmode mode,
+ u32 data_size, u32 timeout)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ int event_queued = 0;
+ enum xlink_error rc;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_OPEN_CHANNEL_REQ,
+ &link->handle, chan, data_size, timeout);
+ if (!event)
+ return X_LINK_ERROR;
+
+ event->data = (void *)mode;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_open_channel);
+
+enum xlink_error xlink_close_channel(struct xlink_handle *handle,
+ u16 chan)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int event_queued = 0;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_CLOSE_CHANNEL_REQ,
+ &link->handle, chan, 0, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_close_channel);
+
+enum xlink_error xlink_write_data(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage, u32 size)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int event_queued = 0;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ if (size > XLINK_MAX_DATA_SIZE)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_WRITE_REQ, &link->handle,
+ chan, size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ if (chan < XLINK_IPC_MAX_CHANNELS &&
+ event->interface == IPC_INTERFACE) {
+ /* only passing message address across IPC interface */
+ event->data = &pmessage;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ xlink_destroy_event(event);
+ } else {
+ event->data = (u8 *)pmessage;
+ event->paddr = 0;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ }
+ return rc;
+}
+EXPORT_SYMBOL(xlink_write_data);
+
+static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage,
+ u32 size)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int event_queued = 0;
+ dma_addr_t paddr = 0;
+ u32 addr;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ if (size > XLINK_MAX_DATA_SIZE)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_WRITE_REQ, &link->handle,
+ chan, size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+ event->user_data = 1;
+
+ if (chan < XLINK_IPC_MAX_CHANNELS &&
+ event->interface == IPC_INTERFACE) {
+ /* only passing message address across IPC interface */
+ if (get_user(addr, (u32 __user *)pmessage)) {
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ event->data = &addr;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ xlink_destroy_event(event);
+ } else {
+ event->data = xlink_platform_allocate(&xlink->pdev->dev, &paddr,
+ size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ if (!event->data) {
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ if (copy_from_user(event->data, (void __user *)pmessage, size)) {
+ xlink_platform_deallocate(&xlink->pdev->dev,
+ event->data, paddr, size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ event->paddr = paddr;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued) {
+ xlink_platform_deallocate(&xlink->pdev->dev,
+ event->data, paddr, size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ xlink_destroy_event(event);
+ }
+ }
+ return rc;
+}
+
+static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
+ u16 chan, u8 const *message, u32 size)
+{
+ enum xlink_error rc = 0;
+
+ rc = do_xlink_write_volatile(handle, chan, message, size, 1);
+ return rc;
+}
+
+enum xlink_error xlink_write_volatile(struct xlink_handle *handle,
+ u16 chan, u8 const *message, u32 size)
+{
+ enum xlink_error rc = 0;
+
+ rc = do_xlink_write_volatile(handle, chan, message, size, 0);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_write_volatile);
+
+static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size, u32 user_flag)
+{
+ enum xlink_error rc = 0;
+ struct xlink_link *link = NULL;
+ struct xlink_event *event = NULL;
+ int event_queued = 0;
+ dma_addr_t paddr;
+ int region = 0;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ if (size > XLINK_MAX_BUF_SIZE)
+ return X_LINK_ERROR; // TODO: XLink Parameter Error
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_WRITE_VOLATILE_REQ,
+ &link->handle, chan, size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ region = XLINK_NORMAL_MEMORY;
+ event->data = xlink_platform_allocate(&xlink->pdev->dev, &paddr, size,
+ XLINK_PACKET_ALIGNMENT, region);
+ if (!event->data) {
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ memcpy(event->data, message, size);
+ event->user_data = user_flag;
+ event->paddr = paddr;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued) {
+ xlink_platform_deallocate(&xlink->pdev->dev, event->data, paddr, size,
+ XLINK_PACKET_ALIGNMENT, region);
+ xlink_destroy_event(event);
+ }
+ return rc;
+}
+
+enum xlink_error xlink_read_data(struct xlink_handle *handle,
+ u16 chan, u8 **pmessage, u32 *size)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ int event_queued = 0;
+ enum xlink_error rc;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_READ_REQ, &link->handle,
+ chan, *size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ event->pdata = (void **)pmessage;
+ event->length = size;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_read_data);
+
+enum xlink_error xlink_read_data_to_buffer(struct xlink_handle *handle,
+ u16 chan, u8 *const message, u32 *size)
+{
+ enum xlink_error rc = 0;
+ struct xlink_link *link = NULL;
+ struct xlink_event *event = NULL;
+ int event_queued = 0;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_READ_TO_BUFFER_REQ,
+ &link->handle, chan, *size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ event->data = message;
+ event->length = size;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_read_data_to_buffer);
+
+enum xlink_error xlink_release_data(struct xlink_handle *handle,
+ u16 chan, u8 * const data_addr)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ int event_queued = 0;
+ enum xlink_error rc;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ event = xlink_create_event(link->id, XLINK_RELEASE_REQ, &link->handle,
+ chan, 0, 0);
+ if (!event)
+ return X_LINK_ERROR;
+
+ event->data = data_addr;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_release_data);
+
+enum xlink_error xlink_disconnect(struct xlink_handle *handle)
+{
+ struct xlink_link *link;
+ enum xlink_error rc = X_LINK_ERROR;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+
+ // decrement refcount, if count is 0 lock mutex and disconnect
+ if (kref_put_mutex(&link->refcount, release_after_kref_put,
+ &xlink->lock)) {
+ // deinitialize multiplexer connection
+ rc = xlink_multiplexer_disconnect(link->id);
+ if (rc) {
+ pr_err("multiplexer disconnect failed\n");
+ mutex_unlock(&xlink->lock);
+ return X_LINK_ERROR;
+ }
+ // TODO: reset device?
+ // invalidate link handle reference
+ link->handle.sw_device_id = XLINK_INVALID_SW_DEVICE_ID;
+ xlink->nmb_connected_links--;
+ mutex_unlock(&xlink->lock);
+ }
+ return rc;
+}
+EXPORT_SYMBOL(xlink_disconnect);
+
+/* Device tree driver match. */
+static const struct of_device_id kmb_xlink_of_match[] = {
+ {
+ .compatible = "intel,keembay-xlink",
+ },
+ {}
+};
+
+/* The xlink driver is a platform device. */
+static struct platform_driver kmb_xlink_driver = {
+ .probe = kmb_xlink_probe,
+ .remove = kmb_xlink_remove,
+ .driver = {
+ .name = DRV_NAME,
+ .of_match_table = kmb_xlink_of_match,
+ },
+};
+
+/*
+ * The remote host system will need to create an xlink platform
+ * device for the platform driver to match with
+ */
+#ifndef CONFIG_XLINK_LOCAL_HOST
+static struct platform_device pdev;
+void kmb_xlink_release(struct device *dev) { return; }
+#endif
+
+static int kmb_xlink_init(void)
+{
+ int rc;
+
+ rc = platform_driver_register(&kmb_xlink_driver);
+#ifndef CONFIG_XLINK_LOCAL_HOST
+ pdev.dev.release = kmb_xlink_release;
+ pdev.name = DRV_NAME;
+ pdev.id = -1;
+ if (!rc) {
+ rc = platform_device_register(&pdev);
+ if (rc)
+ platform_driver_unregister(&kmb_xlink_driver);
+ }
+#endif
+ return rc;
+}
+module_init(kmb_xlink_init);
+
+static void kmb_xlink_exit(void)
+{
+#ifndef CONFIG_XLINK_LOCAL_HOST
+ platform_device_unregister(&pdev);
+#endif
+ platform_driver_unregister(&kmb_xlink_driver);
+}
+module_exit(kmb_xlink_exit);
+
+MODULE_DESCRIPTION("KeemBay xlink Kernel Driver");
+MODULE_AUTHOR("Seamus Kelly <seamus.kelly@intel.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/misc/xlink-core/xlink-defs.h b/drivers/misc/xlink-core/xlink-defs.h
new file mode 100644
index 000000000000..09aee36d5542
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-defs.h
@@ -0,0 +1,175 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink Defines.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_DEFS_H
+#define __XLINK_DEFS_H
+
+#include <linux/slab.h>
+#include <linux/xlink.h>
+
+#define XLINK_MAX_BUF_SIZE 128U
+#define XLINK_MAX_DATA_SIZE (1024U * 1024U * 1024U)
+#define XLINK_MAX_CONTROL_DATA_SIZE 100U
+#define XLINK_MAX_CONNECTIONS 16
+#define XLINK_PACKET_ALIGNMENT 64
+#define XLINK_INVALID_EVENT_ID 0xDEADBEEF
+#define XLINK_INVALID_CHANNEL_ID 0xDEAD
+#define XLINK_PACKET_QUEUE_CAPACITY 10000
+#define XLINK_EVENT_QUEUE_CAPACITY 10000
+#define XLINK_EVENT_HEADER_MAGIC 0x786C6E6B
+#define XLINK_PING_TIMEOUT_MS 5000U
+#define XLINK_MAX_DEVICE_NAME_SIZE 128
+#define XLINK_MAX_DEVICE_LIST_SIZE 8
+#define XLINK_INVALID_LINK_ID 0xDEADBEEF
+#define XLINK_INVALID_SW_DEVICE_ID 0xDEADBEEF
+
+#define NMB_CHANNELS 4096
+#define IP_CONTROL_CHANNEL 0x0A
+#define VPU_CONTROL_CHANNEL 0x400
+#define CONTROL_CHANNEL_OPMODE RXB_TXB // blocking
+#define CONTROL_CHANNEL_DATASIZE 128U // size of internal rx/tx buffers
+#define CONTROL_CHANNEL_TIMEOUT_MS 0U // wait indefinitely
+#define SIGXLNK 44 // signal XLink uses for callback signalling
+
+#define UNUSED(x) (void)(x)
+
+// the end of the IPC channel range (starting at zero)
+#define XLINK_IPC_MAX_CHANNELS 1024
+
+// used to extract the interface type from a sw device id
+#define SW_DEVICE_ID_INTERFACE_SHIFT 24U
+#define SW_DEVICE_ID_INTERFACE_MASK 0x7
+#define GET_INTERFACE_FROM_SW_DEVICE_ID(id) (((id) >> SW_DEVICE_ID_INTERFACE_SHIFT) & \
+ SW_DEVICE_ID_INTERFACE_MASK)
+#define SW_DEVICE_ID_IPC_INTERFACE 0x0
+#define SW_DEVICE_ID_PCIE_INTERFACE 0x1
+#define SW_DEVICE_ID_USB_INTERFACE 0x2
+#define SW_DEVICE_ID_ETH_INTERFACE 0x3
+
+enum xlink_interface {
+ NULL_INTERFACE = -1,
+ IPC_INTERFACE = 0,
+ PCIE_INTERFACE,
+ USB_CDC_INTERFACE,
+ ETH_INTERFACE,
+ NMB_OF_INTERFACES,
+};
+
+static inline int get_interface_from_sw_device_id(u32 sw_device_id)
+{
+ u32 interface = 0;
+
+ interface = GET_INTERFACE_FROM_SW_DEVICE_ID(sw_device_id);
+ switch (interface) {
+ case SW_DEVICE_ID_IPC_INTERFACE:
+ return IPC_INTERFACE;
+ case SW_DEVICE_ID_PCIE_INTERFACE:
+ return PCIE_INTERFACE;
+ case SW_DEVICE_ID_USB_INTERFACE:
+ return USB_CDC_INTERFACE;
+ case SW_DEVICE_ID_ETH_INTERFACE:
+ return ETH_INTERFACE;
+ default:
+ return NULL_INTERFACE;
+ }
+}
+
+enum xlink_channel_status {
+ CHAN_CLOSED = 0x0000,
+ CHAN_OPEN = 0x0001,
+ CHAN_BLOCKED_READ = 0x0010,
+ CHAN_BLOCKED_WRITE = 0x0100,
+ CHAN_OPEN_PEER = 0x1000,
+};
+
+enum xlink_event_origin {
+ EVENT_TX = 0, // outgoing events
+ EVENT_RX, // incoming events
+};
+
+enum xlink_event_type {
+ // request events
+ XLINK_WRITE_REQ = 0x00,
+ XLINK_WRITE_VOLATILE_REQ,
+ XLINK_READ_REQ,
+ XLINK_READ_TO_BUFFER_REQ,
+ XLINK_RELEASE_REQ,
+ XLINK_OPEN_CHANNEL_REQ,
+ XLINK_CLOSE_CHANNEL_REQ,
+ XLINK_PING_REQ,
+ XLINK_REQ_LAST,
+ // response events
+ XLINK_WRITE_RESP = 0x10,
+ XLINK_WRITE_VOLATILE_RESP,
+ XLINK_READ_RESP,
+ XLINK_READ_TO_BUFFER_RESP,
+ XLINK_RELEASE_RESP,
+ XLINK_OPEN_CHANNEL_RESP,
+ XLINK_CLOSE_CHANNEL_RESP,
+ XLINK_PING_RESP,
+ XLINK_RESP_LAST,
+};
+
+struct xlink_event_header {
+ u32 magic;
+ u32 id;
+ enum xlink_event_type type;
+ u32 chan;
+ size_t size;
+ u32 timeout;
+ u8 control_data[XLINK_MAX_CONTROL_DATA_SIZE];
+};
+
+struct xlink_event {
+ struct xlink_event_header header;
+ enum xlink_event_origin origin;
+ u32 link_id;
+ struct xlink_handle *handle;
+ int interface;
+ void *data;
+ struct task_struct *calling_pid;
+ char callback_origin;
+ char user_data;
+ void **pdata;
+ dma_addr_t paddr;
+ u32 *length;
+ struct list_head list;
+};
+
+static inline struct xlink_event *xlink_create_event(u32 link_id,
+ enum xlink_event_type type,
+ struct xlink_handle *handle,
+ u32 chan, u32 size, u32 timeout)
+{
+ struct xlink_event *new_event = NULL;
+
+ // allocate new event
+ new_event = kzalloc(sizeof(*new_event), GFP_KERNEL);
+ if (!new_event)
+ return NULL;
+
+ // set event context
+ new_event->link_id = link_id;
+ new_event->handle = handle;
+ new_event->interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ new_event->user_data = 0;
+
+ // set event header
+ new_event->header.magic = XLINK_EVENT_HEADER_MAGIC;
+ new_event->header.id = XLINK_INVALID_EVENT_ID;
+ new_event->header.type = type;
+ new_event->header.chan = chan;
+ new_event->header.size = size;
+ new_event->header.timeout = timeout;
+ return new_event;
+}
+
+static inline void xlink_destroy_event(struct xlink_event *event)
+{
+ kfree(event);
+}
+#endif /* __XLINK_DEFS_H */
diff --git a/drivers/misc/xlink-core/xlink-multiplexer.c b/drivers/misc/xlink-core/xlink-multiplexer.c
new file mode 100644
index 000000000000..9b1ed008bb56
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-multiplexer.c
@@ -0,0 +1,534 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Multiplexer.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/version.h>
+#include <linux/cdev.h>
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/dma-direct.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/uaccess.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/completion.h>
+#include <linux/sched/signal.h>
+
+#ifdef CONFIG_XLINK_LOCAL_HOST
+#include <linux/xlink-ipc.h>
+#endif
+
+#include "xlink-multiplexer.h"
+#include "xlink-platform.h"
+
+#define THR_UPR 85
+#define THR_LWR 80
+
+// timeout used for open channel
+#define OPEN_CHANNEL_TIMEOUT_MSEC 5000
+
+/* Channel mapping table. */
+struct xlink_channel_type {
+ enum xlink_interface remote_to_local;
+ enum xlink_interface local_to_ip;
+};
+
+struct xlink_channel_table_entry {
+ u16 start_range;
+ u16 stop_range;
+ struct xlink_channel_type type;
+};
+
+static const struct xlink_channel_table_entry default_channel_table[] = {
+ {0x0, 0x1, {PCIE_INTERFACE, IPC_INTERFACE}},
+ {0x2, 0x9, {USB_CDC_INTERFACE, IPC_INTERFACE}},
+ {0xA, 0x3FD, {PCIE_INTERFACE, IPC_INTERFACE}},
+ {0x3FE, 0x3FF, {ETH_INTERFACE, IPC_INTERFACE}},
+ {0x400, 0xFFE, {PCIE_INTERFACE, NULL_INTERFACE}},
+ {0xFFF, 0xFFF, {ETH_INTERFACE, NULL_INTERFACE}},
+ {NMB_CHANNELS, NMB_CHANNELS, {NULL_INTERFACE, NULL_INTERFACE}},
+};
+
+struct channel {
+ struct open_channel *opchan;
+ enum xlink_opmode mode;
+ enum xlink_channel_status status;
+ enum xlink_channel_status ipc_status;
+ u32 size;
+ u32 timeout;
+};
+
+struct packet {
+ u8 *data;
+ u32 length;
+ dma_addr_t paddr;
+ struct list_head list;
+};
+
+struct packet_queue {
+ u32 count;
+ u32 capacity;
+ struct list_head head;
+ struct mutex lock; // lock to protect packet queue
+};
+
+struct open_channel {
+ u16 id;
+ struct channel *chan;
+ struct packet_queue rx_queue;
+ struct packet_queue tx_queue;
+ s32 rx_fill_level;
+ s32 tx_fill_level;
+ s32 tx_packet_level;
+ s32 tx_up_limit;
+ struct completion opened;
+ struct completion pkt_available;
+ struct completion pkt_consumed;
+ struct completion pkt_released;
+ struct task_struct *ready_calling_pid;
+ void *ready_callback;
+ struct task_struct *consumed_calling_pid;
+ void *consumed_callback;
+ char callback_origin;
+ struct mutex lock; // lock to protect channel config
+};
+
+struct xlink_multiplexer {
+ struct device *dev;
+ struct channel channels[XLINK_MAX_CONNECTIONS][NMB_CHANNELS];
+};
+
+static struct xlink_multiplexer *xmux;
+
+/*
+ * Multiplexer Internal Functions
+ *
+ */
+
+static inline int chan_is_non_blocking_read(struct open_channel *opchan)
+{
+ if (opchan->chan->mode == RXN_TXN || opchan->chan->mode == RXN_TXB)
+ return 1;
+
+ return 0;
+}
+
+static inline int chan_is_non_blocking_write(struct open_channel *opchan)
+{
+ if (opchan->chan->mode == RXN_TXN || opchan->chan->mode == RXB_TXN)
+ return 1;
+
+ return 0;
+}
+
+static struct xlink_channel_type const *get_channel_type(u16 chan)
+{
+ struct xlink_channel_type const *type = NULL;
+ int i = 0;
+
+ while (default_channel_table[i].start_range < NMB_CHANNELS) {
+ if (chan >= default_channel_table[i].start_range &&
+ chan <= default_channel_table[i].stop_range) {
+ type = &default_channel_table[i].type;
+ break;
+ }
+ i++;
+ }
+ return type;
+}
+
+static int is_channel_for_device(u16 chan, u32 sw_device_id,
+ enum xlink_dev_type dev_type)
+{
+ struct xlink_channel_type const *chan_type = get_channel_type(chan);
+ int interface = NULL_INTERFACE;
+
+ if (chan_type) {
+ interface = get_interface_from_sw_device_id(sw_device_id);
+ if (dev_type == VPUIP_DEVICE) {
+ if (chan_type->local_to_ip == interface)
+ return 1;
+ } else {
+ if (chan_type->remote_to_local == interface)
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static int is_control_channel(u16 chan)
+{
+ if (chan == IP_CONTROL_CHANNEL || chan == VPU_CONTROL_CHANNEL)
+ return 1;
+ else
+ return 0;
+}
+
+static int release_packet_from_channel(struct open_channel *opchan,
+ struct packet_queue *queue,
+ u8 * const addr, u32 *size)
+{
+ u8 packet_found = 0;
+ struct packet *pkt = NULL;
+
+ if (!addr) {
+ // address is null, release first packet in queue
+ if (!list_empty(&queue->head)) {
+ pkt = list_first_entry(&queue->head, struct packet,
+ list);
+ packet_found = 1;
+ }
+ } else {
+ // find packet in channel rx queue
+ list_for_each_entry(pkt, &queue->head, list) {
+ if (pkt->data == addr) {
+ packet_found = 1;
+ break;
+ }
+ }
+ }
+ if (!pkt || !packet_found)
+ return X_LINK_ERROR;
+ // packet found, deallocate and remove from queue
+ xlink_platform_deallocate(xmux->dev, pkt->data, pkt->paddr, pkt->length,
+ XLINK_PACKET_ALIGNMENT, XLINK_NORMAL_MEMORY);
+ list_del(&pkt->list);
+ queue->count--;
+ opchan->rx_fill_level -= pkt->length;
+ if (size)
+ *size = pkt->length;
+ kfree(pkt);
+ return X_LINK_SUCCESS;
+}
+
+static int multiplexer_open_channel(u32 link_id, u16 chan)
+{
+ struct open_channel *opchan;
+
+ // channel already open
+ if (xmux->channels[link_id][chan].opchan)
+ return X_LINK_SUCCESS;
+
+ // allocate open channel
+ opchan = kzalloc(sizeof(*opchan), GFP_KERNEL);
+ if (!opchan)
+ return X_LINK_ERROR;
+
+ // initialize open channel
+ opchan->id = chan;
+ opchan->chan = &xmux->channels[link_id][chan];
+ // TODO: remove circular dependency
+ xmux->channels[link_id][chan].opchan = opchan;
+ INIT_LIST_HEAD(&opchan->rx_queue.head);
+ opchan->rx_queue.count = 0;
+ opchan->rx_queue.capacity = XLINK_PACKET_QUEUE_CAPACITY;
+ INIT_LIST_HEAD(&opchan->tx_queue.head);
+ opchan->tx_queue.count = 0;
+ opchan->tx_queue.capacity = XLINK_PACKET_QUEUE_CAPACITY;
+ opchan->rx_fill_level = 0;
+ opchan->tx_fill_level = 0;
+ opchan->tx_packet_level = 0;
+ opchan->tx_up_limit = 0;
+ init_completion(&opchan->opened);
+ init_completion(&opchan->pkt_available);
+ init_completion(&opchan->pkt_consumed);
+ init_completion(&opchan->pkt_released);
+ mutex_init(&opchan->lock);
+ return X_LINK_SUCCESS;
+}
+
+static int multiplexer_close_channel(struct open_channel *opchan)
+{
+ if (!opchan)
+ return X_LINK_ERROR;
+
+ // free remaining packets
+ while (!list_empty(&opchan->rx_queue.head)) {
+ release_packet_from_channel(opchan, &opchan->rx_queue,
+ NULL, NULL);
+ }
+
+ while (!list_empty(&opchan->tx_queue.head)) {
+ release_packet_from_channel(opchan, &opchan->tx_queue,
+ NULL, NULL);
+ }
+
+ // deallocate data structure and destroy
+ opchan->chan->opchan = NULL; // TODO: remove circular dependency
+ mutex_destroy(&opchan->rx_queue.lock);
+ mutex_destroy(&opchan->tx_queue.lock);
+ mutex_unlock(&opchan->lock);
+ mutex_destroy(&opchan->lock);
+ kfree(opchan);
+ return X_LINK_SUCCESS;
+}
+
+/*
+ * Multiplexer External Functions
+ *
+ */
+
+enum xlink_error xlink_multiplexer_init(void *dev)
+{
+ struct platform_device *plat_dev = (struct platform_device *)dev;
+
+ // allocate multiplexer data structure
+ xmux = kzalloc(sizeof(*xmux), GFP_KERNEL);
+ if (!xmux)
+ return X_LINK_ERROR;
+
+ xmux->dev = &plat_dev->dev;
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_multiplexer_connect(u32 link_id)
+{
+ int rc;
+
+ if (!xmux)
+ return X_LINK_ERROR;
+
+ // open ip control channel
+ rc = multiplexer_open_channel(link_id, IP_CONTROL_CHANNEL);
+ if (rc) {
+ goto r_cleanup;
+ } else {
+ xmux->channels[link_id][IP_CONTROL_CHANNEL].size = CONTROL_CHANNEL_DATASIZE;
+ xmux->channels[link_id][IP_CONTROL_CHANNEL].timeout = CONTROL_CHANNEL_TIMEOUT_MS;
+ xmux->channels[link_id][IP_CONTROL_CHANNEL].mode = CONTROL_CHANNEL_OPMODE;
+ xmux->channels[link_id][IP_CONTROL_CHANNEL].status = CHAN_OPEN;
+ }
+ // open vpu control channel
+ rc = multiplexer_open_channel(link_id, VPU_CONTROL_CHANNEL);
+ if (rc) {
+ goto r_cleanup;
+ } else {
+ xmux->channels[link_id][VPU_CONTROL_CHANNEL].size = CONTROL_CHANNEL_DATASIZE;
+ xmux->channels[link_id][VPU_CONTROL_CHANNEL].timeout = CONTROL_CHANNEL_TIMEOUT_MS;
+ xmux->channels[link_id][VPU_CONTROL_CHANNEL].mode = CONTROL_CHANNEL_OPMODE;
+ xmux->channels[link_id][VPU_CONTROL_CHANNEL].status = CHAN_OPEN;
+ }
+ return X_LINK_SUCCESS;
+
+r_cleanup:
+ xlink_multiplexer_disconnect(link_id);
+ return X_LINK_ERROR;
+}
+
+enum xlink_error xlink_multiplexer_disconnect(u32 link_id)
+{
+ int i;
+
+ if (!xmux)
+ return X_LINK_ERROR;
+
+ for (i = 0; i < NMB_CHANNELS; i++) {
+ if (xmux->channels[link_id][i].opchan)
+ multiplexer_close_channel(xmux->channels[link_id][i].opchan);
+ }
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_multiplexer_destroy(void)
+{
+ int i;
+
+ if (!xmux)
+ return X_LINK_ERROR;
+
+ // close all open channels and deallocate remaining packets
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++)
+ xlink_multiplexer_disconnect(i);
+
+ // destroy multiplexer
+ kfree(xmux);
+ xmux = NULL;
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
+ int *event_queued)
+{
+ int rc = X_LINK_SUCCESS;
+ u16 chan = 0;
+
+ if (!xmux || !event)
+ return X_LINK_ERROR;
+
+ chan = event->header.chan;
+
+ // verify channel ID is in range
+ if (chan >= NMB_CHANNELS)
+ return X_LINK_ERROR;
+
+ // verify communication to device on channel is valid
+ if (!is_channel_for_device(chan, event->handle->sw_device_id,
+ event->handle->dev_type))
+ return X_LINK_ERROR;
+
+ // verify this is not a control channel
+ if (is_control_channel(chan))
+ return X_LINK_ERROR;
+
+ if (chan < XLINK_IPC_MAX_CHANNELS && event->interface == IPC_INTERFACE)
+ // event should be handled by passthrough
+ rc = xlink_passthrough(event);
+ return rc;
+}
+
+enum xlink_error xlink_passthrough(struct xlink_event *event)
+{
+ int rc = 0;
+#ifdef CONFIG_XLINK_LOCAL_HOST
+ struct xlink_ipc_context ipc = {0};
+ phys_addr_t physaddr = 0;
+ dma_addr_t vpuaddr = 0;
+ u32 timeout = 0;
+ u32 link_id;
+ u16 chan;
+
+ if (!xmux || !event)
+ return X_LINK_ERROR;
+
+ link_id = event->link_id;
+ chan = event->header.chan;
+ ipc.chan = chan;
+
+ if (ipc.chan >= XLINK_IPC_MAX_CHANNELS)
+ return rc;
+
+ switch (event->header.type) {
+ case XLINK_WRITE_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ /* Translate physical address to VPU address */
+ vpuaddr = phys_to_dma(xmux->dev, *(u32 *)event->data);
+ event->data = &vpuaddr;
+ rc = xlink_platform_write(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ event->data,
+ &event->header.size, 0, &ipc);
+ } else {
+ /* channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_WRITE_VOLATILE_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ ipc.is_volatile = 1;
+ rc = xlink_platform_write(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ event->data,
+ &event->header.size, 0, &ipc);
+ } else {
+ /* channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_READ_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ /* if channel has receive blocking set,
+ * then set timeout to U32_MAX
+ */
+ if (xmux->channels[link_id][chan].mode == RXB_TXN ||
+ xmux->channels[link_id][chan].mode == RXB_TXB) {
+ timeout = U32_MAX;
+ } else {
+ timeout = xmux->channels[link_id][chan].timeout;
+ }
+ rc = xlink_platform_read(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ &vpuaddr,
+ (size_t *)event->length,
+ timeout, &ipc);
+ /* Translate VPU address to physical address */
+ physaddr = dma_to_phys(xmux->dev, vpuaddr);
+ *(phys_addr_t *)event->pdata = physaddr;
+ } else {
+ /* channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_READ_TO_BUFFER_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ /* if channel has receive blocking set,
+ * then set timeout to U32_MAX
+ */
+ if (xmux->channels[link_id][chan].mode == RXB_TXN ||
+ xmux->channels[link_id][chan].mode == RXB_TXB) {
+ timeout = U32_MAX;
+ } else {
+ timeout = xmux->channels[link_id][chan].timeout;
+ }
+ ipc.is_volatile = 1;
+ rc = xlink_platform_read(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ event->data,
+ (size_t *)event->length,
+ timeout, &ipc);
+ if (rc || *event->length > XLINK_MAX_BUF_SIZE)
+ rc = X_LINK_ERROR;
+ } else {
+ /* channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_RELEASE_REQ:
+ break;
+ case XLINK_OPEN_CHANNEL_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_CLOSED) {
+ xmux->channels[link_id][chan].size = event->header.size;
+ xmux->channels[link_id][chan].timeout = event->header.timeout;
+ xmux->channels[link_id][chan].mode = (uintptr_t)event->data;
+ rc = xlink_platform_open_channel(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ chan);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ xmux->channels[link_id][chan].ipc_status = CHAN_OPEN;
+ } else {
+ /* channel already open */
+ rc = X_LINK_ALREADY_OPEN;
+ }
+ break;
+ case XLINK_CLOSE_CHANNEL_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ rc = xlink_platform_close_channel(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ chan);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ xmux->channels[link_id][chan].ipc_status = CHAN_CLOSED;
+ } else {
+ /* can't close channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_PING_REQ:
+ case XLINK_WRITE_RESP:
+ case XLINK_WRITE_VOLATILE_RESP:
+ case XLINK_READ_RESP:
+ case XLINK_READ_TO_BUFFER_RESP:
+ case XLINK_RELEASE_RESP:
+ case XLINK_OPEN_CHANNEL_RESP:
+ case XLINK_CLOSE_CHANNEL_RESP:
+ case XLINK_PING_RESP:
+ break;
+ default:
+ rc = X_LINK_ERROR;
+ }
+#else
+ rc = 0;
+#endif // CONFIG_XLINK_LOCAL_HOST
+ return rc;
+}
diff --git a/drivers/misc/xlink-core/xlink-multiplexer.h b/drivers/misc/xlink-core/xlink-multiplexer.h
new file mode 100644
index 000000000000..c978e5683b45
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-multiplexer.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink Multiplexer.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_MULTIPLEXER_H
+#define __XLINK_MULTIPLEXER_H
+
+#include "xlink-defs.h"
+
+enum xlink_error xlink_multiplexer_init(void *dev);
+
+enum xlink_error xlink_multiplexer_connect(u32 link_id);
+
+enum xlink_error xlink_multiplexer_disconnect(u32 link_id);
+
+enum xlink_error xlink_multiplexer_destroy(void);
+
+enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
+ int *event_queued);
+
+enum xlink_error xlink_multiplexer_rx(struct xlink_event *event);
+
+enum xlink_error xlink_passthrough(struct xlink_event *event);
+
+void *find_allocated_buffer(dma_addr_t paddr);
+
+int unregister_allocated_buffer(void *buf, dma_addr_t paddr);
+
+int core_release_packet_from_channel(u32 link_id, uint16_t chan,
+ uint8_t * const addr);
+
+#endif /* __XLINK_MULTIPLEXER_H */
diff --git a/drivers/misc/xlink-core/xlink-platform.c b/drivers/misc/xlink-core/xlink-platform.c
new file mode 100644
index 000000000000..c34b69ee206b
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-platform.c
@@ -0,0 +1,160 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Linux Kernel Platform API
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/fs.h>
+#include <linux/cdev.h>
+#include <linux/platform_device.h>
+#include <linux/uaccess.h>
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/xlink_drv_inf.h>
+
+#include "xlink-platform.h"
+
+#ifdef CONFIG_XLINK_LOCAL_HOST
+#include <linux/xlink-ipc.h>
+#else /* !CONFIG_XLINK_LOCAL_HOST */
+
+static inline int xlink_ipc_connect(u32 sw_device_id)
+{ return -1; }
+
+static inline int xlink_ipc_write(u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context)
+{ return -1; }
+
+static inline int xlink_ipc_read(u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context)
+{ return -1; }
+
+static inline int xlink_ipc_open_channel(u32 sw_device_id,
+ u32 channel)
+{ return -1; }
+
+static inline int xlink_ipc_close_channel(u32 sw_device_id,
+ u32 channel)
+{ return -1; }
+
+#endif /* CONFIG_XLINK_LOCAL_HOST */
+
+/*
+ * xlink low-level driver interface arrays
+ *
+ * note: array indices based on xlink_interface enum definition
+ */
+
+static int (*connect_fcts[NMB_OF_INTERFACES])(u32) = {
+ xlink_ipc_connect, xlink_pcie_connect, NULL, NULL};
+
+static int (*write_fcts[NMB_OF_INTERFACES])(u32, void *, size_t * const, u32) = {
+ NULL, xlink_pcie_write, NULL, NULL};
+
+static int (*read_fcts[NMB_OF_INTERFACES])(u32, void *, size_t * const, u32) = {
+ NULL, xlink_pcie_read, NULL, NULL};
+
+static int (*open_chan_fcts[NMB_OF_INTERFACES])(u32, u32) = {
+ xlink_ipc_open_channel, NULL, NULL, NULL};
+
+static int (*close_chan_fcts[NMB_OF_INTERFACES])(u32, u32) = {
+ xlink_ipc_close_channel, NULL, NULL, NULL};
+
+/*
+ * xlink low-level driver interface
+ */
+
+int xlink_platform_connect(u32 interface, u32 sw_device_id)
+{
+ if (interface >= NMB_OF_INTERFACES || !connect_fcts[interface])
+ return -1;
+
+ return connect_fcts[interface](sw_device_id);
+}
+
+int xlink_platform_write(u32 interface, u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context)
+{
+ if (interface == IPC_INTERFACE)
+ return xlink_ipc_write(sw_device_id, data, size, timeout,
+ context);
+
+ if (interface >= NMB_OF_INTERFACES || !write_fcts[interface])
+ return -1;
+
+ return write_fcts[interface](sw_device_id, data, size, timeout);
+}
+
+int xlink_platform_read(u32 interface, u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context)
+{
+ if (interface == IPC_INTERFACE)
+ return xlink_ipc_read(sw_device_id, data, size, timeout,
+ context);
+
+ if (interface >= NMB_OF_INTERFACES || !read_fcts[interface])
+ return -1;
+
+ return read_fcts[interface](sw_device_id, data, size, timeout);
+}
+
+int xlink_platform_open_channel(u32 interface, u32 sw_device_id,
+ u32 channel)
+{
+ if (interface >= NMB_OF_INTERFACES || !open_chan_fcts[interface])
+ return -1;
+
+ return open_chan_fcts[interface](sw_device_id, channel);
+}
+
+int xlink_platform_close_channel(u32 interface, u32 sw_device_id,
+ u32 channel)
+{
+ if (interface >= NMB_OF_INTERFACES || !close_chan_fcts[interface])
+ return -1;
+
+ return close_chan_fcts[interface](sw_device_id, channel);
+}
+
+void *xlink_platform_allocate(struct device *dev, dma_addr_t *handle,
+ u32 size, u32 alignment,
+ enum xlink_memory_region region)
+{
+#if defined(CONFIG_XLINK_PSS) || !defined(CONFIG_XLINK_LOCAL_HOST)
+ *handle = 0;
+ return kzalloc(size, GFP_KERNEL);
+#else
+ void *p;
+
+ if (region == XLINK_CMA_MEMORY) {
+ // size needs to be at least 4097 to be allocated from CMA
+ size = (size < PAGE_SIZE * 2) ? (PAGE_SIZE * 2) : size;
+ p = dma_alloc_coherent(dev, size, handle, GFP_KERNEL);
+ return p;
+ }
+ *handle = 0;
+ return kzalloc(size, GFP_KERNEL);
+#endif
+}
+
+void xlink_platform_deallocate(struct device *dev, void *buf,
+ dma_addr_t handle, u32 size, u32 alignment,
+ enum xlink_memory_region region)
+{
+#if defined(CONFIG_XLINK_PSS) || !defined(CONFIG_XLINK_LOCAL_HOST)
+ kfree(buf);
+#else
+ if (region == XLINK_CMA_MEMORY) {
+ // size needs to be at least 4097 to be allocated from CMA
+ size = (size < PAGE_SIZE * 2) ? (PAGE_SIZE * 2) : size;
+ dma_free_coherent(dev, size, buf, handle);
+ } else {
+ kfree(buf);
+ }
+#endif
+}
diff --git a/drivers/misc/xlink-core/xlink-platform.h b/drivers/misc/xlink-core/xlink-platform.h
new file mode 100644
index 000000000000..2c7c4c418099
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-platform.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink Linux Kernel Platform API
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_PLATFORM_H
+#define __XLINK_PLATFORM_H
+
+#include "xlink-defs.h"
+
+int xlink_platform_connect(u32 interface, u32 sw_device_id);
+
+int xlink_platform_write(u32 interface, u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context);
+
+int xlink_platform_read(u32 interface, u32 sw_device_id, void *data,
+ size_t * const size, u32 timeout, void *context);
+
+int xlink_platform_reset_device(u32 interface, u32 sw_device_id);
+
+int xlink_platform_boot_device(u32 interface, u32 sw_device_id,
+ const char *binary_name);
+
+int xlink_platform_get_device_name(u32 interface, u32 sw_device_id,
+ char *device_name, size_t name_size);
+
+int xlink_platform_get_device_list(u32 interface, u32 *sw_device_id_list,
+ u32 *num_devices);
+
+int xlink_platform_get_device_status(u32 interface, u32 sw_device_id,
+ u32 *device_status);
+
+int xlink_platform_set_device_mode(u32 interface, u32 sw_device_id,
+ u32 power_mode);
+
+int xlink_platform_get_device_mode(u32 interface, u32 sw_device_id,
+ u32 *power_mode);
+
+int xlink_platform_open_channel(u32 interface, u32 sw_device_id,
+ u32 channel);
+
+int xlink_platform_close_channel(u32 interface, u32 sw_device_id,
+ u32 channel);
+
+int xlink_platform_register_for_events(u32 interface, u32 sw_device_id,
+ xlink_device_event_cb event_notif_fn);
+
+int xlink_platform_unregister_for_events(u32 interface, u32 sw_device_id);
+
+enum xlink_memory_region {
+ XLINK_NORMAL_MEMORY = 0,
+ XLINK_CMA_MEMORY
+};
+
+void *xlink_platform_allocate(struct device *dev, dma_addr_t *handle,
+ u32 size, u32 alignment,
+ enum xlink_memory_region region);
+
+void xlink_platform_deallocate(struct device *dev, void *buf,
+ dma_addr_t handle, u32 size, u32 alignment,
+ enum xlink_memory_region region);
+
+#endif /* __XLINK_PLATFORM_H */
diff --git a/include/linux/xlink.h b/include/linux/xlink.h
new file mode 100644
index 000000000000..c22439d5aade
--- /dev/null
+++ b/include/linux/xlink.h
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink Linux Kernel API
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_H
+#define __XLINK_H
+
+#include <uapi/misc/xlink_uapi.h>
+
+enum xlink_dev_type {
+ HOST_DEVICE = 0, /* used when communicating host to host*/
+ VPUIP_DEVICE /* used when communicating host to vpu ip */
+};
+
+struct xlink_handle {
+ u32 sw_device_id; /* identifies a device in the system */
+ enum xlink_dev_type dev_type; /* determines direction of comms */
+};
+
+enum xlink_opmode {
+ RXB_TXB = 0, /* blocking read, blocking write */
+ RXN_TXN, /* non-blocking read, non-blocking write */
+ RXB_TXN, /* blocking read, non-blocking write */
+ RXN_TXB /* non-blocking read, blocking write */
+};
+
+enum xlink_device_power_mode {
+ POWER_DEFAULT_NOMINAL_MAX = 0, /* no load reduction, default mode */
+ POWER_SUBNOMINAL_HIGH, /* slight load reduction */
+ POWER_MEDIUM, /* average load reduction */
+ POWER_LOW, /* significant load reduction */
+ POWER_MIN, /* maximum load reduction */
+ POWER_SUSPENDED /* power off or device suspend */
+};
+
+enum xlink_error {
+ X_LINK_SUCCESS = 0, /* xlink operation completed successfully */
+ X_LINK_ALREADY_INIT, /* xlink already initialized */
+ X_LINK_ALREADY_OPEN, /* channel already open */
+ X_LINK_COMMUNICATION_NOT_OPEN, /* operation on a closed channel */
+ X_LINK_COMMUNICATION_FAIL, /* communication failure */
+ X_LINK_COMMUNICATION_UNKNOWN_ERROR, /* error unknown */
+ X_LINK_DEVICE_NOT_FOUND, /* device specified not found */
+ X_LINK_TIMEOUT, /* operation timed out */
+ X_LINK_ERROR, /* parameter error */
+ X_LINK_CHAN_FULL /* channel has reached fill level */
+};
+
+enum xlink_device_status {
+ XLINK_DEV_OFF = 0, /* device is off */
+ XLINK_DEV_ERROR, /* device is busy and not available */
+ XLINK_DEV_BUSY, /* device is available for use */
+ XLINK_DEV_RECOVERY, /* device is in recovery mode */
+ XLINK_DEV_READY /* device HW failure is detected */
+};
+
+/* xlink API */
+
+typedef void (*xlink_event)(u16 chan);
+typedef int (*xlink_device_event_cb)(u32 sw_device_id, u32 event_type);
+
+enum xlink_error xlink_initialize(void);
+
+enum xlink_error xlink_connect(struct xlink_handle *handle);
+
+enum xlink_error xlink_open_channel(struct xlink_handle *handle,
+ u16 chan, enum xlink_opmode mode,
+ u32 data_size, u32 timeout);
+
+enum xlink_error xlink_close_channel(struct xlink_handle *handle, u16 chan);
+
+enum xlink_error xlink_write_data(struct xlink_handle *handle,
+ u16 chan, u8 const *message, u32 size);
+
+enum xlink_error xlink_write_volatile(struct xlink_handle *handle,
+ u16 chan, u8 const *message, u32 size);
+
+enum xlink_error xlink_read_data(struct xlink_handle *handle,
+ u16 chan, u8 **message, u32 *size);
+
+enum xlink_error xlink_read_data_to_buffer(struct xlink_handle *handle,
+ u16 chan, u8 * const message,
+ u32 *size);
+
+enum xlink_error xlink_release_data(struct xlink_handle *handle,
+ u16 chan, u8 * const data_addr);
+
+enum xlink_error xlink_disconnect(struct xlink_handle *handle);
+
+/* API functions to be implemented
+ *
+ * enum xlink_error xlink_write_crc_data(struct xlink_handle *handle,
+ * u16 chan, u8 const *message, size_t * const size);
+ *
+ * enum xlink_error xlink_read_crc_data(struct xlink_handle *handle,
+ * u16 chan, u8 **message, size_t * const size);
+ *
+ * enum xlink_error xlink_read_crc_data_to_buffer(struct xlink_handle *handle,
+ * u16 chan, u8 * const message, size_t * const size);
+ *
+ * enum xlink_error xlink_reset_all(void);
+ *
+ */
+
+#endif /* __XLINK_H */
diff --git a/include/uapi/misc/xlink_uapi.h b/include/uapi/misc/xlink_uapi.h
new file mode 100644
index 000000000000..77794299c4a6
--- /dev/null
+++ b/include/uapi/misc/xlink_uapi.h
@@ -0,0 +1,145 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * xlink Linux Kernel API
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_UAPI_H
+#define __XLINK_UAPI_H
+
+#include <linux/types.h>
+
+#define XLINK_MAGIC 'x'
+#define XL_OPEN_CHANNEL _IOW(XLINK_MAGIC, 1, void*)
+#define XL_READ_DATA _IOW(XLINK_MAGIC, 2, void*)
+#define XL_WRITE_DATA _IOW(XLINK_MAGIC, 3, void*)
+#define XL_CLOSE_CHANNEL _IOW(XLINK_MAGIC, 4, void*)
+#define XL_WRITE_VOLATILE _IOW(XLINK_MAGIC, 5, void*)
+#define XL_READ_TO_BUFFER _IOW(XLINK_MAGIC, 6, void*)
+#define XL_START_VPU _IOW(XLINK_MAGIC, 7, void*)
+#define XL_STOP_VPU _IOW(XLINK_MAGIC, 8, void*)
+#define XL_RESET_VPU _IOW(XLINK_MAGIC, 9, void*)
+#define XL_CONNECT _IOW(XLINK_MAGIC, 10, void*)
+#define XL_RELEASE_DATA _IOW(XLINK_MAGIC, 11, void*)
+#define XL_DISCONNECT _IOW(XLINK_MAGIC, 12, void*)
+#define XL_WRITE_CONTROL_DATA _IOW(XLINK_MAGIC, 13, void*)
+#define XL_DATA_READY_CALLBACK _IOW(XLINK_MAGIC, 14, void*)
+#define XL_DATA_CONSUMED_CALLBACK _IOW(XLINK_MAGIC, 15, void*)
+#define XL_GET_DEVICE_NAME _IOW(XLINK_MAGIC, 16, void*)
+#define XL_GET_DEVICE_LIST _IOW(XLINK_MAGIC, 17, void*)
+#define XL_GET_DEVICE_STATUS _IOW(XLINK_MAGIC, 18, void*)
+#define XL_BOOT_DEVICE _IOW(XLINK_MAGIC, 19, void*)
+#define XL_RESET_DEVICE _IOW(XLINK_MAGIC, 20, void*)
+#define XL_GET_DEVICE_MODE _IOW(XLINK_MAGIC, 21, void*)
+#define XL_SET_DEVICE_MODE _IOW(XLINK_MAGIC, 22, void*)
+#define XL_REGISTER_DEV_EVENT _IOW(XLINK_MAGIC, 23, void*)
+#define XL_UNREGISTER_DEV_EVENT _IOW(XLINK_MAGIC, 24, void*)
+
+struct xlinkopenchannel {
+ void *handle;
+ __u16 chan;
+ int mode;
+ __u32 data_size;
+ __u32 timeout;
+ __u32 *return_code;
+};
+
+struct xlinkcallback {
+ void *handle;
+ __u16 chan;
+ void (*callback)(__u16 chan);
+ __u32 *return_code;
+};
+
+struct xlinkwritedata {
+ void *handle;
+ __u16 chan;
+ void const *pmessage;
+ __u32 size;
+ __u32 *return_code;
+};
+
+struct xlinkreaddata {
+ void *handle;
+ __u16 chan;
+ void *pmessage;
+ __u32 *size;
+ __u32 *return_code;
+};
+
+struct xlinkreadtobuffer {
+ void *handle;
+ __u16 chan;
+ void *pmessage;
+ __u32 *size;
+ __u32 *return_code;
+};
+
+struct xlinkconnect {
+ void *handle;
+ __u32 *return_code;
+};
+
+struct xlinkrelease {
+ void *handle;
+ __u16 chan;
+ void *addr;
+ __u32 *return_code;
+};
+
+struct xlinkstartvpu {
+ char *filename;
+ int namesize;
+ __u32 *return_code;
+};
+
+struct xlinkstopvpu {
+ __u32 *return_code;
+};
+
+struct xlinkgetdevicename {
+ void *handle;
+ char *name;
+ __u32 name_size;
+ __u32 *return_code;
+};
+
+struct xlinkgetdevicelist {
+ __u32 *sw_device_id_list;
+ __u32 *num_devices;
+ __u32 *return_code;
+};
+
+struct xlinkgetdevicestatus {
+ void *handle;
+ __u32 *device_status;
+ __u32 *return_code;
+};
+
+struct xlinkbootdevice {
+ void *handle;
+ const char *binary_name;
+ __u32 binary_name_size;
+ __u32 *return_code;
+};
+
+struct xlinkresetdevice {
+ void *handle;
+ __u32 *return_code;
+};
+
+struct xlinkdevmode {
+ void *handle;
+ int *device_mode;
+ __u32 *return_code;
+};
+
+struct xlinkregdevevent {
+ void *handle;
+ __u32 *event_list;
+ __u32 num_events;
+ __u32 *return_code;
+};
+
+#endif /* __XLINK_UAPI_H */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 19/22] xlink-core: Enable xlink protocol over pcie
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (17 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 18/22] xlink-core: Add xlink core driver xLink mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 20/22] xlink-core: Enable VPU IP management and runtime control mgross
` (3 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Arnd Bergmann, Greg Kroah-Hartman
From: Seamus Kelly <seamus.kelly@intel.com>
Enable host system access to the VPU over the xlink protocol over PCIe by
enabling channel multiplexing and dispatching. This allows for remote host
communication channels across pcie links.
add dispatcher
update multiplexer to utilise dispatcher
xlink-core: Patch set 2
Add xlink-dispatcher
creates tx and rx threads
enables queueing of messages for transmission and on reception
Update multiplexer to utilise dispatcher:
handle multiplexing channels over single interface link e.g. PCIe
process messages received by dispatcher
pass messages created by API calls to dispatcher for transmission
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
---
drivers/misc/xlink-core/Makefile | 2 +-
drivers/misc/xlink-core/xlink-core.c | 38 +-
drivers/misc/xlink-core/xlink-dispatcher.c | 441 +++++++++++++++++
drivers/misc/xlink-core/xlink-dispatcher.h | 26 +
drivers/misc/xlink-core/xlink-multiplexer.c | 498 +++++++++++++++++++-
5 files changed, 1001 insertions(+), 4 deletions(-)
create mode 100644 drivers/misc/xlink-core/xlink-dispatcher.c
create mode 100644 drivers/misc/xlink-core/xlink-dispatcher.h
diff --git a/drivers/misc/xlink-core/Makefile b/drivers/misc/xlink-core/Makefile
index 6b708340dbfe..6e604c0b8962 100644
--- a/drivers/misc/xlink-core/Makefile
+++ b/drivers/misc/xlink-core/Makefile
@@ -2,4 +2,4 @@
# Makefile for KeemBay xlink Linux driver
#
obj-$(CONFIG_XLINK_CORE) += xlink.o
-xlink-objs += xlink-core.o xlink-multiplexer.o xlink-platform.o
+xlink-objs += xlink-core.o xlink-multiplexer.o xlink-dispatcher.o xlink-platform.o
diff --git a/drivers/misc/xlink-core/xlink-core.c b/drivers/misc/xlink-core/xlink-core.c
index 3813208ffa95..e7d5f68a5ba1 100644
--- a/drivers/misc/xlink-core/xlink-core.c
+++ b/drivers/misc/xlink-core/xlink-core.c
@@ -20,6 +20,7 @@
#endif
#include "xlink-defs.h"
+#include "xlink-dispatcher.h"
#include "xlink-multiplexer.h"
#include "xlink-platform.h"
@@ -159,6 +160,12 @@ static int kmb_xlink_probe(struct platform_device *pdev)
goto r_multiplexer;
}
+ // initialize dispatcher
+ rc = xlink_dispatcher_init(xlink_dev->pdev);
+ if (rc != X_LINK_SUCCESS) {
+ pr_err("Dispatcher initialization failed\n");
+ goto r_dispatcher;
+ }
// initialize xlink data structure
xlink_dev->nmb_connected_links = 0;
mutex_init(&xlink_dev->lock);
@@ -176,7 +183,7 @@ static int kmb_xlink_probe(struct platform_device *pdev)
/*Allocating Major number*/
if ((alloc_chrdev_region(&xdev, 0, 1, "xlinkdev")) < 0) {
dev_info(&pdev->dev, "Cannot allocate major number\n");
- goto r_multiplexer;
+ goto r_dispatcher;
}
dev_info(&pdev->dev, "Major = %d Minor = %d\n", MAJOR(xdev),
MINOR(xdev));
@@ -213,6 +220,8 @@ static int kmb_xlink_probe(struct platform_device *pdev)
class_destroy(dev_class);
r_class:
unregister_chrdev_region(xdev, 1);
+r_dispatcher:
+ xlink_dispatcher_destroy();
r_multiplexer:
xlink_multiplexer_destroy();
return -1;
@@ -228,6 +237,10 @@ static int kmb_xlink_remove(struct platform_device *pdev)
rc = xlink_multiplexer_destroy();
if (rc != X_LINK_SUCCESS)
pr_err("Multiplexer destroy failed\n");
+ // stop dispatchers and destroy
+ rc = xlink_dispatcher_destroy();
+ if (rc != X_LINK_SUCCESS)
+ pr_err("Dispatcher destroy failed\n");
mutex_unlock(&xlink->lock);
mutex_destroy(&xlink->lock);
@@ -436,6 +449,14 @@ enum xlink_error xlink_connect(struct xlink_handle *handle)
link->handle = *handle;
xlink->nmb_connected_links++;
kref_init(&link->refcount);
+ if (interface != IPC_INTERFACE) {
+ // start dispatcher
+ rc = xlink_dispatcher_start(link->id, &link->handle);
+ if (rc) {
+ pr_err("dispatcher start failed\n");
+ goto r_cleanup;
+ }
+ }
// initialize multiplexer connection
rc = xlink_multiplexer_connect(link->id);
if (rc) {
@@ -629,7 +650,8 @@ static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
}
static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
- u16 chan, u8 const *message, u32 size)
+ u16 chan, u8 const *message,
+ u32 size)
{
enum xlink_error rc = 0;
@@ -781,6 +803,7 @@ EXPORT_SYMBOL(xlink_release_data);
enum xlink_error xlink_disconnect(struct xlink_handle *handle)
{
struct xlink_link *link;
+ int interface = NULL_INTERFACE;
enum xlink_error rc = X_LINK_ERROR;
if (!xlink || !handle)
@@ -793,6 +816,17 @@ enum xlink_error xlink_disconnect(struct xlink_handle *handle)
// decrement refcount, if count is 0 lock mutex and disconnect
if (kref_put_mutex(&link->refcount, release_after_kref_put,
&xlink->lock)) {
+ // stop dispatcher
+ interface = get_interface_from_sw_device_id(link->handle.sw_device_id);
+ if (interface != IPC_INTERFACE) {
+ // stop dispatcher
+ rc = xlink_dispatcher_stop(link->id);
+ if (rc != X_LINK_SUCCESS) {
+ pr_err("dispatcher stop failed\n");
+ mutex_unlock(&xlink->lock);
+ return X_LINK_ERROR;
+ }
+ }
// deinitialize multiplexer connection
rc = xlink_multiplexer_disconnect(link->id);
if (rc) {
diff --git a/drivers/misc/xlink-core/xlink-dispatcher.c b/drivers/misc/xlink-core/xlink-dispatcher.c
new file mode 100644
index 000000000000..11ef8e4110ca
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-dispatcher.c
@@ -0,0 +1,441 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Dispatcher.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/semaphore.h>
+#include <linux/mutex.h>
+#include <linux/completion.h>
+#include <linux/sched/signal.h>
+#include <linux/platform_device.h>
+
+#include "xlink-dispatcher.h"
+#include "xlink-multiplexer.h"
+#include "xlink-platform.h"
+
+#define DISPATCHER_RX_TIMEOUT_MSEC 0
+
+/* state of a dispatcher servicing a link to a device*/
+enum dispatcher_state {
+ XLINK_DISPATCHER_INIT, /* initialized but not used */
+ XLINK_DISPATCHER_RUNNING, /* currently servicing a link */
+ XLINK_DISPATCHER_STOPPED, /* no longer servicing a link */
+ XLINK_DISPATCHER_ERROR, /* fatal error */
+};
+
+/* queue for dispatcher tx thread event handling */
+struct event_queue {
+ u32 count; /* number of events in the queue */
+ u32 capacity; /* capacity of events in the queue */
+ struct list_head head; /* head of event linked list */
+ struct mutex lock; /* locks queue while accessing */
+};
+
+/* dispatcher servicing a single link to a device */
+struct dispatcher {
+ u32 link_id; /* id of link being serviced */
+ enum dispatcher_state state; /* state of the dispatcher */
+ struct xlink_handle *handle; /* xlink device handle */
+ int interface; /* underlying interface of link */
+ struct task_struct *rxthread; /* kthread servicing rx */
+ struct task_struct *txthread; /* kthread servicing tx */
+ struct event_queue queue; /* xlink event queue */
+ struct semaphore event_sem; /* signals tx kthread of events */
+ struct completion rx_done; /* sync start/stop of rx kthread */
+ struct completion tx_done; /* sync start/stop of tx thread */
+};
+
+/* xlink dispatcher system component */
+struct xlink_dispatcher {
+ struct dispatcher dispatchers[XLINK_MAX_CONNECTIONS]; /* disp queue */
+ struct device *dev; /* deallocate data */
+ struct mutex lock; /* locks when start new disp */
+};
+
+/* global reference to the xlink dispatcher data structure */
+static struct xlink_dispatcher *xlinkd;
+
+/*
+ * Dispatcher Internal Functions
+ *
+ */
+
+static struct dispatcher *get_dispatcher_by_id(u32 id)
+{
+ if (!xlinkd)
+ return NULL;
+
+ if (id >= XLINK_MAX_CONNECTIONS)
+ return NULL;
+
+ return &xlinkd->dispatchers[id];
+}
+
+static u32 event_generate_id(void)
+{
+ static u32 id = 0xa; // TODO: temporary solution
+
+ return id++;
+}
+
+static struct xlink_event *event_dequeue(struct event_queue *queue)
+{
+ struct xlink_event *event = NULL;
+
+ mutex_lock(&queue->lock);
+ if (!list_empty(&queue->head)) {
+ event = list_first_entry(&queue->head, struct xlink_event,
+ list);
+ list_del(&event->list);
+ queue->count--;
+ }
+ mutex_unlock(&queue->lock);
+ return event;
+}
+
+static int event_enqueue(struct event_queue *queue, struct xlink_event *event)
+{
+ int rc = -1;
+
+ mutex_lock(&queue->lock);
+ if (queue->count < ((queue->capacity / 10) * 7)) {
+ list_add_tail(&event->list, &queue->head);
+ queue->count++;
+ rc = 0;
+ }
+ mutex_unlock(&queue->lock);
+ return rc;
+}
+
+static struct xlink_event *dispatcher_event_get(struct dispatcher *disp)
+{
+ int rc = 0;
+ struct xlink_event *event = NULL;
+
+ // wait until an event is available
+ rc = down_interruptible(&disp->event_sem);
+ // dequeue and return next event to process
+ if (!rc)
+ event = event_dequeue(&disp->queue);
+ return event;
+}
+
+static int is_valid_event_header(struct xlink_event *event)
+{
+ if (event->header.magic != XLINK_EVENT_HEADER_MAGIC)
+ return 0;
+ else
+ return 1;
+}
+
+static int dispatcher_event_send(struct xlink_event *event)
+{
+ size_t event_header_size = sizeof(event->header);
+ int rc;
+
+ // write event header
+ // printk(KERN_DEBUG "Sending event: type = 0x%x, id = 0x%x\n",
+ // event->header.type, event->header.id);
+ rc = xlink_platform_write(event->interface,
+ event->handle->sw_device_id, &event->header,
+ &event_header_size, event->header.timeout, NULL);
+ if (rc || event_header_size != sizeof(event->header)) {
+ pr_err("Write header failed %d\n", rc);
+ return rc;
+ }
+ if (event->header.type == XLINK_WRITE_REQ ||
+ event->header.type == XLINK_WRITE_VOLATILE_REQ) {
+ // write event data
+ rc = xlink_platform_write(event->interface,
+ event->handle->sw_device_id, event->data,
+ &event->header.size, event->header.timeout,
+ NULL);
+ if (rc)
+ pr_err("Write data failed %d\n", rc);
+ if (event->user_data == 1) {
+ if (event->paddr != 0) {
+ xlink_platform_deallocate(xlinkd->dev,
+ event->data, event->paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_CMA_MEMORY);
+ } else {
+ xlink_platform_deallocate(xlinkd->dev,
+ event->data, event->paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ }
+ }
+ }
+ return rc;
+}
+
+static int xlink_dispatcher_rxthread(void *context)
+{
+ struct dispatcher *disp = (struct dispatcher *)context;
+ struct xlink_event *event;
+ size_t size;
+ int rc;
+
+ // printk(KERN_DEBUG "dispatcher rxthread started\n");
+ event = xlink_create_event(disp->link_id, 0, disp->handle, 0, 0, 0);
+ if (!event)
+ return -1;
+
+ allow_signal(SIGTERM); // allow thread termination while waiting on sem
+ complete(&disp->rx_done);
+ while (!kthread_should_stop()) {
+ size = sizeof(event->header);
+ rc = xlink_platform_read(disp->interface,
+ disp->handle->sw_device_id,
+ &event->header, &size,
+ DISPATCHER_RX_TIMEOUT_MSEC, NULL);
+ if (rc || size != (int)sizeof(event->header))
+ continue;
+ if (is_valid_event_header(event)) {
+ event->link_id = disp->link_id;
+ rc = xlink_multiplexer_rx(event);
+ if (!rc) {
+ event = xlink_create_event(disp->link_id, 0,
+ disp->handle, 0, 0,
+ 0);
+ if (!event)
+ return -1;
+ }
+ }
+ }
+ // printk(KERN_INFO "dispatcher rxthread stopped\n");
+ complete(&disp->rx_done);
+ do_exit(0);
+ return 0;
+}
+
+static int xlink_dispatcher_txthread(void *context)
+{
+ struct dispatcher *disp = (struct dispatcher *)context;
+ struct xlink_event *event;
+
+ // printk(KERN_DEBUG "dispatcher txthread started\n");
+ allow_signal(SIGTERM); // allow thread termination while waiting on sem
+ complete(&disp->tx_done);
+ while (!kthread_should_stop()) {
+ event = dispatcher_event_get(disp);
+ if (!event)
+ continue;
+
+ dispatcher_event_send(event);
+ xlink_destroy_event(event); // free handled event
+ }
+ // printk(KERN_INFO "dispatcher txthread stopped\n");
+ complete(&disp->tx_done);
+ do_exit(0);
+ return 0;
+}
+
+/*
+ * Dispatcher External Functions
+ *
+ */
+
+enum xlink_error xlink_dispatcher_init(void *dev)
+{
+ struct platform_device *plat_dev = (struct platform_device *)dev;
+ int i;
+
+ xlinkd = kzalloc(sizeof(*xlinkd), GFP_KERNEL);
+ if (!xlinkd)
+ return X_LINK_ERROR;
+
+ xlinkd->dev = &plat_dev->dev;
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
+ xlinkd->dispatchers[i].link_id = i;
+ sema_init(&xlinkd->dispatchers[i].event_sem, 0);
+ init_completion(&xlinkd->dispatchers[i].rx_done);
+ init_completion(&xlinkd->dispatchers[i].tx_done);
+ INIT_LIST_HEAD(&xlinkd->dispatchers[i].queue.head);
+ mutex_init(&xlinkd->dispatchers[i].queue.lock);
+ xlinkd->dispatchers[i].queue.count = 0;
+ xlinkd->dispatchers[i].queue.capacity =
+ XLINK_EVENT_QUEUE_CAPACITY;
+ xlinkd->dispatchers[i].state = XLINK_DISPATCHER_INIT;
+ }
+ mutex_init(&xlinkd->lock);
+
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_dispatcher_start(int id, struct xlink_handle *handle)
+{
+ struct dispatcher *disp;
+
+ mutex_lock(&xlinkd->lock);
+ // get dispatcher by link id
+ disp = get_dispatcher_by_id(id);
+ if (!disp)
+ goto r_error;
+
+ // cannot start a running or failed dispatcher
+ if (disp->state == XLINK_DISPATCHER_RUNNING ||
+ disp->state == XLINK_DISPATCHER_ERROR)
+ goto r_error;
+
+ // set the dispatcher context
+ disp->handle = handle;
+ disp->interface = get_interface_from_sw_device_id(handle->sw_device_id);
+
+ // run dispatcher thread to handle and write outgoing packets
+ disp->txthread = kthread_run(xlink_dispatcher_txthread,
+ (void *)disp, "txthread");
+ if (!disp->txthread) {
+ pr_err("xlink txthread creation failed\n");
+ goto r_txthread;
+ }
+ wait_for_completion(&disp->tx_done);
+ disp->state = XLINK_DISPATCHER_RUNNING;
+ // run dispatcher thread to read and handle incoming packets
+ disp->rxthread = kthread_run(xlink_dispatcher_rxthread,
+ (void *)disp, "rxthread");
+ if (!disp->rxthread) {
+ pr_err("xlink rxthread creation failed\n");
+ goto r_rxthread;
+ }
+ wait_for_completion(&disp->rx_done);
+ mutex_unlock(&xlinkd->lock);
+
+ return X_LINK_SUCCESS;
+
+r_rxthread:
+ kthread_stop(disp->txthread);
+r_txthread:
+ disp->state = XLINK_DISPATCHER_STOPPED;
+r_error:
+ mutex_unlock(&xlinkd->lock);
+ return X_LINK_ERROR;
+}
+
+enum xlink_error xlink_dispatcher_event_add(enum xlink_event_origin origin,
+ struct xlink_event *event)
+{
+ struct dispatcher *disp;
+ int rc;
+
+ // get dispatcher by handle
+ disp = get_dispatcher_by_id(event->link_id);
+ if (!disp)
+ return X_LINK_ERROR;
+
+ // only add events if the dispatcher is running
+ if (disp->state != XLINK_DISPATCHER_RUNNING)
+ return X_LINK_ERROR;
+
+ // configure event and add to queue
+ if (origin == EVENT_TX)
+ event->header.id = event_generate_id();
+ event->origin = origin;
+ rc = event_enqueue(&disp->queue, event);
+ if (rc)
+ return X_LINK_CHAN_FULL;
+
+ // notify dispatcher tx thread of new event
+ up(&disp->event_sem);
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_dispatcher_stop(int id)
+{
+ struct dispatcher *disp;
+ int rc;
+
+ mutex_lock(&xlinkd->lock);
+ // get dispatcher by link id
+ disp = get_dispatcher_by_id(id);
+ if (!disp)
+ goto r_error;
+
+ // don't stop dispatcher if not started
+ if (disp->state != XLINK_DISPATCHER_RUNNING)
+ goto r_error;
+
+ if (disp->rxthread) {
+ // stop dispatcher rx thread
+ send_sig(SIGTERM, disp->rxthread, 0);
+ rc = kthread_stop(disp->rxthread);
+ if (rc)
+ goto r_thread;
+ }
+ wait_for_completion(&disp->rx_done);
+ if (disp->txthread) {
+ // stop dispatcher tx thread
+ send_sig(SIGTERM, disp->txthread, 0);
+ rc = kthread_stop(disp->txthread);
+ if (rc)
+ goto r_thread;
+ }
+ wait_for_completion(&disp->tx_done);
+ disp->state = XLINK_DISPATCHER_STOPPED;
+ mutex_unlock(&xlinkd->lock);
+ return X_LINK_SUCCESS;
+
+r_thread:
+ // dispatcher now in error state and cannot be used
+ disp->state = XLINK_DISPATCHER_ERROR;
+r_error:
+ mutex_unlock(&xlinkd->lock);
+ return X_LINK_ERROR;
+}
+
+enum xlink_error xlink_dispatcher_destroy(void)
+{
+ enum xlink_event_type type;
+ struct xlink_event *event;
+ struct dispatcher *disp;
+ int i;
+
+ for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
+ // get dispatcher by link id
+ disp = get_dispatcher_by_id(i);
+ if (!disp)
+ continue;
+
+ // stop all running dispatchers
+ if (disp->state == XLINK_DISPATCHER_RUNNING)
+ xlink_dispatcher_stop(i);
+
+ // empty queues of all used dispatchers
+ if (disp->state == XLINK_DISPATCHER_INIT)
+ continue;
+
+ // deallocate remaining events in queue
+ while (!list_empty(&disp->queue.head)) {
+ event = event_dequeue(&disp->queue);
+ if (!event)
+ continue;
+ type = event->header.type;
+ if (type == XLINK_WRITE_REQ ||
+ type == XLINK_WRITE_VOLATILE_REQ) {
+ // deallocate event data
+ xlink_platform_deallocate(xlinkd->dev,
+ event->data,
+ event->paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ }
+ xlink_destroy_event(event);
+ }
+ // destroy dispatcher
+ mutex_destroy(&disp->queue.lock);
+ }
+ mutex_destroy(&xlinkd->lock);
+ return X_LINK_SUCCESS;
+}
diff --git a/drivers/misc/xlink-core/xlink-dispatcher.h b/drivers/misc/xlink-core/xlink-dispatcher.h
new file mode 100644
index 000000000000..d1458e7a4ab7
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-dispatcher.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink Dispatcher.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#ifndef __XLINK_DISPATCHER_H
+#define __XLINK_DISPATCHER_H
+
+#include "xlink-defs.h"
+
+enum xlink_error xlink_dispatcher_init(void *dev);
+
+enum xlink_error xlink_dispatcher_start(int id, struct xlink_handle *handle);
+
+enum xlink_error xlink_dispatcher_event_add(enum xlink_event_origin origin,
+ struct xlink_event *event);
+
+enum xlink_error xlink_dispatcher_stop(int id);
+
+enum xlink_error xlink_dispatcher_destroy(void);
+
+enum xlink_error xlink_dispatcher_ipc_passthru_event_add(struct xlink_event *event);
+
+#endif /* __XLINK_DISPATCHER_H */
diff --git a/drivers/misc/xlink-core/xlink-multiplexer.c b/drivers/misc/xlink-core/xlink-multiplexer.c
index 9b1ed008bb56..339734826f3e 100644
--- a/drivers/misc/xlink-core/xlink-multiplexer.c
+++ b/drivers/misc/xlink-core/xlink-multiplexer.c
@@ -27,6 +27,7 @@
#include <linux/xlink-ipc.h>
#endif
+#include "xlink-dispatcher.h"
#include "xlink-multiplexer.h"
#include "xlink-platform.h"
@@ -165,6 +166,32 @@ static int is_channel_for_device(u16 chan, u32 sw_device_id,
return 0;
}
+static int is_enough_space_in_channel(struct open_channel *opchan,
+ u32 size)
+{
+ if (opchan->tx_packet_level >= ((XLINK_PACKET_QUEUE_CAPACITY / 100) * THR_UPR)) {
+ pr_info("Packet queue limit reached\n");
+ return 0;
+ }
+ if (opchan->tx_up_limit == 0) {
+ if ((opchan->tx_fill_level + size)
+ > ((opchan->chan->size / 100) * THR_UPR)) {
+ opchan->tx_up_limit = 1;
+ return 0;
+ }
+ }
+ if (opchan->tx_up_limit == 1) {
+ if ((opchan->tx_fill_level + size)
+ < ((opchan->chan->size / 100) * THR_LWR)) {
+ opchan->tx_up_limit = 0;
+ return 1;
+ } else {
+ return 0;
+ }
+ }
+ return 1;
+}
+
static int is_control_channel(u16 chan)
{
if (chan == IP_CONTROL_CHANNEL || chan == VPU_CONTROL_CHANNEL)
@@ -173,6 +200,51 @@ static int is_control_channel(u16 chan)
return 0;
}
+static struct open_channel *get_channel(u32 link_id, u16 chan)
+{
+ if (!xmux->channels[link_id][chan].opchan)
+ return NULL;
+ mutex_lock(&xmux->channels[link_id][chan].opchan->lock);
+ return xmux->channels[link_id][chan].opchan;
+}
+
+static void release_channel(struct open_channel *opchan)
+{
+ if (opchan)
+ mutex_unlock(&opchan->lock);
+}
+
+static int add_packet_to_channel(struct open_channel *opchan,
+ struct packet_queue *queue,
+ void *buffer, u32 size,
+ dma_addr_t paddr)
+{
+ struct packet *pkt;
+
+ if (queue->count < queue->capacity) {
+ pkt = kzalloc(sizeof(*pkt), GFP_KERNEL);
+ if (!pkt)
+ return X_LINK_ERROR;
+ pkt->data = buffer;
+ pkt->length = size;
+ pkt->paddr = paddr;
+ list_add_tail(&pkt->list, &queue->head);
+ queue->count++;
+ opchan->rx_fill_level += pkt->length;
+ }
+ return X_LINK_SUCCESS;
+}
+
+static struct packet *get_packet_from_channel(struct packet_queue *queue)
+{
+ struct packet *pkt = NULL;
+ // get first packet in queue
+ if (!list_empty(&queue->head))
+ pkt = list_first_entry(&queue->head, struct packet, list);
+
+ return pkt;
+}
+
static int release_packet_from_channel(struct open_channel *opchan,
struct packet_queue *queue,
u8 * const addr, u32 *size)
@@ -355,15 +427,46 @@ enum xlink_error xlink_multiplexer_destroy(void)
return X_LINK_SUCCESS;
}
+static int compl_wait(struct completion *compl, struct open_channel *opchan)
+{
+ int rc;
+ unsigned long tout = msecs_to_jiffies(opchan->chan->timeout);
+
+ if (opchan->chan->timeout == 0) {
+ mutex_unlock(&opchan->lock);
+ rc = wait_for_completion_interruptible(compl);
+ mutex_lock(&opchan->lock);
+ if (rc < 0) // wait interrupted
+ rc = X_LINK_ERROR;
+ } else {
+ mutex_unlock(&opchan->lock);
+ rc = wait_for_completion_interruptible_timeout(compl, tout);
+ mutex_lock(&opchan->lock);
+ if (rc == 0)
+ rc = X_LINK_TIMEOUT;
+ else if (rc < 0) // wait interrupted
+ rc = X_LINK_ERROR;
+ else if (rc > 0)
+ rc = X_LINK_SUCCESS;
+ }
+ return rc;
+}
+
enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
int *event_queued)
{
+ struct open_channel *opchan = NULL;
+ struct packet *pkt = NULL;
int rc = X_LINK_SUCCESS;
+ u32 link_id = 0;
+ u32 size = 0;
u16 chan = 0;
+ u32 save_timeout = 0;
if (!xmux || !event)
return X_LINK_ERROR;
+ link_id = event->link_id;
chan = event->header.chan;
// verify channel ID is in range
@@ -379,9 +482,402 @@ enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
if (is_control_channel(chan))
return X_LINK_ERROR;
- if (chan < XLINK_IPC_MAX_CHANNELS && event->interface == IPC_INTERFACE)
+ if (chan < XLINK_IPC_MAX_CHANNELS && event->interface == IPC_INTERFACE) {
// event should be handled by passthrough
rc = xlink_passthrough(event);
+ return rc;
+ }
+ // event should be handled by dispatcher
+ switch (event->header.type) {
+ case XLINK_WRITE_REQ:
+ case XLINK_WRITE_VOLATILE_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan || opchan->chan->status != CHAN_OPEN) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ while (!is_enough_space_in_channel(opchan,
+ event->header.size)) {
+ if (opchan->chan->mode == RXN_TXB ||
+ opchan->chan->mode == RXB_TXB) {
+ // channel is blocking,
+ // wait for packet to be released
+ rc = compl_wait(&opchan->pkt_released, opchan);
+ } else {
+ rc = X_LINK_CHAN_FULL;
+ break;
+ }
+ }
+ if (rc == X_LINK_SUCCESS) {
+ opchan->tx_fill_level += event->header.size;
+ opchan->tx_packet_level++;
+ xlink_dispatcher_event_add(EVENT_TX, event);
+ *event_queued = 1;
+ if (opchan->chan->mode == RXN_TXB ||
+ opchan->chan->mode == RXB_TXB) {
+ // channel is blocking,
+ // wait for packet to be consumed
+ mutex_unlock(&opchan->lock);
+ rc = compl_wait(&opchan->pkt_consumed, opchan);
+ mutex_lock(&opchan->lock);
+ }
+ }
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_READ_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan || opchan->chan->status != CHAN_OPEN) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ if (opchan->chan->mode == RXB_TXN ||
+ opchan->chan->mode == RXB_TXB) {
+ // channel is blocking, wait for packet to become available
+ mutex_unlock(&opchan->lock);
+ rc = compl_wait(&opchan->pkt_available, opchan);
+ mutex_lock(&opchan->lock);
+ }
+ if (rc == X_LINK_SUCCESS) {
+ pkt = get_packet_from_channel(&opchan->rx_queue);
+ if (pkt) {
+ *(u32 **)event->pdata = (u32 *)pkt->data;
+ *event->length = pkt->length;
+ xlink_dispatcher_event_add(EVENT_TX, event);
+ *event_queued = 1;
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ }
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_READ_TO_BUFFER_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan || opchan->chan->status != CHAN_OPEN) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ if (opchan->chan->mode == RXB_TXN ||
+ opchan->chan->mode == RXB_TXB) {
+ // channel is blocking, wait for packet to become available
+ mutex_unlock(&opchan->lock);
+ rc = compl_wait(&opchan->pkt_available, opchan);
+ mutex_lock(&opchan->lock);
+ }
+ if (rc == X_LINK_SUCCESS) {
+ pkt = get_packet_from_channel(&opchan->rx_queue);
+ if (pkt) {
+ memcpy(event->data, pkt->data, pkt->length);
+ *event->length = pkt->length;
+ xlink_dispatcher_event_add(EVENT_TX, event);
+ *event_queued = 1;
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ }
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_RELEASE_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ rc = release_packet_from_channel(opchan,
+ &opchan->rx_queue,
+ event->data,
+ &size);
+ if (rc) {
+ rc = X_LINK_ERROR;
+ } else {
+ event->header.size = size;
+ xlink_dispatcher_event_add(EVENT_TX, event);
+ *event_queued = 1;
+ }
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_OPEN_CHANNEL_REQ:
+ if (xmux->channels[link_id][chan].status == CHAN_CLOSED) {
+ xmux->channels[link_id][chan].size = event->header.size;
+ xmux->channels[link_id][chan].timeout = event->header.timeout;
+ xmux->channels[link_id][chan].mode = (uintptr_t)event->data;
+ rc = multiplexer_open_channel(link_id, chan);
+ if (rc) {
+ rc = X_LINK_ERROR;
+ } else {
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ xlink_dispatcher_event_add(EVENT_TX, event);
+ *event_queued = 1;
+ mutex_unlock(&opchan->lock);
+ save_timeout = opchan->chan->timeout;
+ opchan->chan->timeout = OPEN_CHANNEL_TIMEOUT_MSEC;
+ rc = compl_wait(&opchan->opened, opchan);
+ opchan->chan->timeout = save_timeout;
+ if (rc == 0) {
+ xmux->channels[link_id][chan].status = CHAN_OPEN;
+ release_channel(opchan);
+ } else {
+ multiplexer_close_channel(opchan);
+ }
+ }
+ }
+ } else if (xmux->channels[link_id][chan].status == CHAN_OPEN_PEER) {
+ /* channel already open */
+ xmux->channels[link_id][chan].status = CHAN_OPEN; // opened locally
+ xmux->channels[link_id][chan].size = event->header.size;
+ xmux->channels[link_id][chan].timeout = event->header.timeout;
+ xmux->channels[link_id][chan].mode = (uintptr_t)event->data;
+ rc = multiplexer_open_channel(link_id, chan);
+ } else {
+ /* channel already open */
+ rc = X_LINK_ALREADY_OPEN;
+ }
+ break;
+ case XLINK_CLOSE_CHANNEL_REQ:
+ if (xmux->channels[link_id][chan].status == CHAN_OPEN) {
+ opchan = get_channel(link_id, chan);
+ if (!opchan)
+ return X_LINK_COMMUNICATION_FAIL;
+ rc = multiplexer_close_channel(opchan);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ xmux->channels[link_id][chan].status = CHAN_CLOSED;
+ } else {
+ /* can't close channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
+ case XLINK_PING_REQ:
+ break;
+ case XLINK_WRITE_RESP:
+ case XLINK_WRITE_VOLATILE_RESP:
+ case XLINK_READ_RESP:
+ case XLINK_READ_TO_BUFFER_RESP:
+ case XLINK_RELEASE_RESP:
+ case XLINK_OPEN_CHANNEL_RESP:
+ case XLINK_CLOSE_CHANNEL_RESP:
+ case XLINK_PING_RESP:
+ default:
+ rc = X_LINK_ERROR;
+ }
+return rc;
+}
+
+enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
+{
+ struct xlink_event *passthru_event = NULL;
+ struct open_channel *opchan = NULL;
+ int rc = X_LINK_SUCCESS;
+ dma_addr_t paddr = 0;
+ void *buffer = NULL;
+ size_t size = 0;
+ u32 link_id;
+ u16 chan;
+
+ if (!xmux || !event)
+ return X_LINK_ERROR;
+
+ link_id = event->link_id;
+ chan = event->header.chan;
+
+ switch (event->header.type) {
+ case XLINK_WRITE_REQ:
+ case XLINK_WRITE_VOLATILE_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ // if we receive data on a closed channel - flush/read the data
+ buffer = xlink_platform_allocate(xmux->dev, &paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ if (buffer) {
+ size = event->header.size;
+ xlink_platform_read(event->interface,
+ event->handle->sw_device_id,
+ buffer, &size, 1000, NULL);
+ xlink_platform_deallocate(xmux->dev, buffer,
+ paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ } else {
+ pr_err("Fatal error: can't allocate memory in line:%d func:%s\n", __LINE__, __func__);
+ }
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ buffer = xlink_platform_allocate(xmux->dev, &paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ if (buffer) {
+ size = event->header.size;
+ rc = xlink_platform_read(event->interface,
+ event->handle->sw_device_id,
+ buffer, &size,
+ opchan->chan->timeout,
+ NULL);
+ if (rc || event->header.size != size) {
+ xlink_platform_deallocate(xmux->dev, buffer,
+ paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ rc = X_LINK_ERROR;
+ release_channel(opchan);
+ break;
+ }
+ event->paddr = paddr;
+ event->data = buffer;
+ if (add_packet_to_channel(opchan, &opchan->rx_queue,
+ event->data,
+ event->header.size,
+ paddr)) {
+ xlink_platform_deallocate(xmux->dev,
+ buffer, paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ rc = X_LINK_ERROR;
+ release_channel(opchan);
+ break;
+ }
+ event->header.type = XLINK_WRITE_VOLATILE_RESP;
+ xlink_dispatcher_event_add(EVENT_RX, event);
+ //complete regardless of mode/timeout
+ complete(&opchan->pkt_available);
+ } else {
+ // failed to allocate buffer
+ rc = X_LINK_ERROR;
+ }
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_READ_REQ:
+ case XLINK_READ_TO_BUFFER_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ break;
+ }
+ event->header.timeout = opchan->chan->timeout;
+ event->header.type = XLINK_READ_TO_BUFFER_RESP;
+ xlink_dispatcher_event_add(EVENT_RX, event);
+ //complete regardless of mode/timeout
+ complete(&opchan->pkt_consumed);
+ release_channel(opchan);
+ break;
+ case XLINK_RELEASE_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ opchan->tx_fill_level -= event->header.size;
+ opchan->tx_packet_level--;
+ event->header.type = XLINK_RELEASE_RESP;
+ xlink_dispatcher_event_add(EVENT_RX, event);
+ //complete regardless of mode/timeout
+ complete(&opchan->pkt_released);
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_OPEN_CHANNEL_REQ:
+ if (xmux->channels[link_id][chan].status == CHAN_CLOSED) {
+ xmux->channels[link_id][chan].size = event->header.size;
+ xmux->channels[link_id][chan].timeout = event->header.timeout;
+ //xmux->channels[link_id][chan].mode = *(enum xlink_opmode *)event->data;
+ rc = multiplexer_open_channel(link_id, chan);
+ if (rc) {
+ rc = X_LINK_ERROR;
+ } else {
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ xmux->channels[link_id][chan].status = CHAN_OPEN_PEER;
+ complete(&opchan->opened);
+ passthru_event = xlink_create_event(link_id,
+ XLINK_OPEN_CHANNEL_RESP,
+ event->handle,
+ chan,
+ 0,
+ opchan->chan->timeout);
+ if (!passthru_event) {
+ rc = X_LINK_ERROR;
+ release_channel(opchan);
+ break;
+ }
+ xlink_dispatcher_event_add(EVENT_RX,
+ passthru_event);
+ }
+ release_channel(opchan);
+ }
+ } else {
+ /* channel already open */
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ passthru_event = xlink_create_event(link_id,
+ XLINK_OPEN_CHANNEL_RESP,
+ event->handle,
+ chan, 0, 0);
+ if (!passthru_event) {
+ release_channel(opchan);
+ rc = X_LINK_ERROR;
+ break;
+ }
+ xlink_dispatcher_event_add(EVENT_RX,
+ passthru_event);
+ }
+ release_channel(opchan);
+ }
+ rc = xlink_passthrough(event);
+ if (rc == 0)
+ xlink_destroy_event(event); // event is handled and can now be freed
+ break;
+ case XLINK_CLOSE_CHANNEL_REQ:
+ case XLINK_PING_REQ:
+ break;
+ case XLINK_WRITE_RESP:
+ case XLINK_WRITE_VOLATILE_RESP:
+ opchan = get_channel(link_id, chan);
+ if (!opchan)
+ rc = X_LINK_COMMUNICATION_FAIL;
+ else
+ xlink_destroy_event(event); // event is handled and can now be freed
+ release_channel(opchan);
+ break;
+ case XLINK_READ_RESP:
+ case XLINK_READ_TO_BUFFER_RESP:
+ case XLINK_RELEASE_RESP:
+ xlink_destroy_event(event); // event is handled and can now be freed
+ break;
+ case XLINK_OPEN_CHANNEL_RESP:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ xlink_destroy_event(event); // event is handled and can now be freed
+ complete(&opchan->opened);
+ }
+ release_channel(opchan);
+ break;
+ case XLINK_CLOSE_CHANNEL_RESP:
+ case XLINK_PING_RESP:
+ xlink_destroy_event(event); // event is handled and can now be freed
+ break;
+ default:
+ rc = X_LINK_ERROR;
+ }
+
return rc;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 20/22] xlink-core: Enable VPU IP management and runtime control
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (18 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 19/22] xlink-core: Enable xlink protocol over pcie mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 21/22] xlink-core: add async channel and events mgross
` (2 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Arnd Bergmann, Greg Kroah-Hartman
From: Seamus Kelly <seamus.kelly@intel.com>
Enable VPU management including, enumeration, boot and runtime control.
Add APIs:
write control data:
used to transmit small, local data
start vpu:
calls boot_device API ( soon to be deprecated )
stop vpu
calls reset_device API ( soon to be deprecated )
reset vpu
calls reset_device API ( soon to be deprecated )
get device name:
Returns the device name for the input device id
This could be a char device path, for example "/dev/ttyUSB0"
for a serial device; or it could be a device string
description, for example, for PCIE "00:00.0 Host bridge: Intel
Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)"
get device list:
Returns the list of software device IDs for all connected
physical devices
get device status:
returns the current state of the input device
OFF - The device is off (D3cold/Slot power removed).
BUSY - device is busy and not available (device is booting)
READY - device is available for use
ERROR - device HW failure is detected
RECOVERY - device is in recovery mode, waiting for recovery operations
boot device:
When used on the remote host - starts the SOC device by calling
corresponding function from VPU Driver.
Takes firmware's 'binary_name' as input.
For Linux, the firmware image is expected to be located in
'/lib/firmware' folder or its subfolders.
For Linux, 'binary_name' is not a path but an image name that
will be searched in the default Linux search paths ('/lib/firmware').
When used on the local host - triggers the booting of VPUIP device.
reset device:
When used on the remote host - resets the device by calling
corresponding VPU Driver function.
When used on the local host - resets the VPUIP device
get device mode:
query and returns the current device power mode
set device mode:
used for device throttling or entering various power modes
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
---
drivers/misc/xlink-core/xlink-core.c | 408 +++++++++++++++++++-
drivers/misc/xlink-core/xlink-defs.h | 2 +
drivers/misc/xlink-core/xlink-multiplexer.c | 56 +++
drivers/misc/xlink-core/xlink-platform.c | 86 +++++
include/linux/xlink.h | 27 ++
5 files changed, 573 insertions(+), 6 deletions(-)
diff --git a/drivers/misc/xlink-core/xlink-core.c b/drivers/misc/xlink-core/xlink-core.c
index e7d5f68a5ba1..a600068840d3 100644
--- a/drivers/misc/xlink-core/xlink-core.c
+++ b/drivers/misc/xlink-core/xlink-core.c
@@ -81,6 +81,8 @@ struct keembay_xlink_dev {
struct mutex lock; // protect access to xlink_dev
};
+static u8 volbuf[XLINK_MAX_BUF_SIZE]; // buffer for volatile transactions
+
/*
* global variable pointing to our xlink device.
*
@@ -260,14 +262,27 @@ static int kmb_xlink_remove(struct platform_device *pdev)
static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
- int interface = NULL_INTERFACE;
struct xlink_handle devh = {0};
struct xlinkopenchannel op = {0};
struct xlinkwritedata wr = {0};
struct xlinkreaddata rd = {0};
+ struct xlinkreadtobuffer rdtobuf = {0};
struct xlinkconnect con = {0};
struct xlinkrelease rel = {0};
- u8 volbuf[XLINK_MAX_BUF_SIZE];
+ struct xlinkstartvpu startvpu = {0};
+ struct xlinkgetdevicename devn = {0};
+ struct xlinkgetdevicelist devl = {0};
+ struct xlinkgetdevicestatus devs = {0};
+ struct xlinkbootdevice boot = {0};
+ struct xlinkresetdevice res = {0};
+ struct xlinkdevmode devm = {0};
+ u32 sw_device_id_list[XLINK_MAX_DEVICE_LIST_SIZE];
+ char name[XLINK_MAX_DEVICE_NAME_SIZE];
+ int interface = NULL_INTERFACE;
+ u32 device_status = 0;
+ u32 num_devices = 0;
+ u32 device_mode = 0;
+ char filename[64];
u8 reladdr;
u8 *rdaddr;
u32 size;
@@ -328,6 +343,26 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
if (copy_to_user((void __user *)rd.return_code, (void *)&rc, sizeof(rc)))
return -EFAULT;
break;
+ case XL_READ_TO_BUFFER:
+ if (copy_from_user(&rdtobuf, (void __user *)arg,
+ sizeof(struct xlinkreadtobuffer)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rdtobuf.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_read_data_to_buffer(&devh, rdtobuf.chan,
+ (u8 *)volbuf, &size);
+ if (!rc) {
+ if (copy_to_user((void __user *)rdtobuf.pmessage, (void *)volbuf,
+ size))
+ return -EFAULT;
+ if (copy_to_user((void __user *)rdtobuf.size, (void *)&size,
+ sizeof(size)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rdtobuf.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
case XL_WRITE_DATA:
if (copy_from_user(&wr, (void __user *)arg,
sizeof(struct xlinkwritedata)))
@@ -346,16 +381,40 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
}
break;
case XL_WRITE_VOLATILE:
- if (copy_from_user(&wr, (void __user *)arg, sizeof(struct xlinkwritedata)))
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
return -EFAULT;
if (copy_from_user(&devh, (void __user *)wr.handle,
sizeof(struct xlink_handle)))
return -EFAULT;
if (wr.size <= XLINK_MAX_BUF_SIZE) {
- if (copy_from_user(volbuf, (void __user *)wr.pmessage, wr.size))
+ if (copy_from_user(volbuf, (void __user *)wr.pmessage,
+ wr.size))
return -EFAULT;
- rc = xlink_write_volatile_user(&devh, wr.chan, volbuf, wr.size);
- if (copy_to_user((void __user *)wr.return_code, (void *)&rc, sizeof(rc)))
+ rc = xlink_write_volatile_user(&devh, wr.chan, volbuf,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ break;
+ case XL_WRITE_CONTROL_DATA:
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_CONTROL_DATA_SIZE) {
+ if (copy_from_user(volbuf, (void __user *)wr.pmessage,
+ wr.size))
+ return -EFAULT;
+ rc = xlink_write_control_data(&devh, wr.chan, volbuf,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code,
+ (void *)&rc, sizeof(rc)))
return -EFAULT;
} else {
return -EFAULT;
@@ -390,6 +449,26 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
return -EFAULT;
break;
+ case XL_START_VPU:
+ if (copy_from_user(&startvpu, (void __user *)arg,
+ sizeof(struct xlinkstartvpu)))
+ return -EFAULT;
+ if (startvpu.namesize > sizeof(filename))
+ return -EINVAL;
+ memset(filename, 0, sizeof(filename));
+ if (copy_from_user(filename, (void __user *)startvpu.filename,
+ startvpu.namesize))
+ return -EFAULT;
+ rc = xlink_start_vpu(filename);
+ if (copy_to_user((void __user *)startvpu.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_STOP_VPU:
+ rc = xlink_stop_vpu();
+ break;
+ case XL_RESET_VPU:
+ rc = xlink_stop_vpu();
+ break;
case XL_DISCONNECT:
if (copy_from_user(&con, (void __user *)arg,
sizeof(struct xlinkconnect)))
@@ -401,6 +480,124 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
if (copy_to_user((void __user *)con.return_code, (void *)&rc, sizeof(rc)))
return -EFAULT;
break;
+ case XL_GET_DEVICE_NAME:
+ if (copy_from_user(&devn, (void __user *)arg,
+ sizeof(struct xlinkgetdevicename)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devn.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (devn.name_size <= XLINK_MAX_DEVICE_NAME_SIZE) {
+ rc = xlink_get_device_name(&devh, name, devn.name_size);
+ if (!rc) {
+ if (copy_to_user((void __user *)devn.name, (void *)name,
+ devn.name_size))
+ return -EFAULT;
+ }
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)devn.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_GET_DEVICE_LIST:
+ if (copy_from_user(&devl, (void __user *)arg,
+ sizeof(struct xlinkgetdevicelist)))
+ return -EFAULT;
+ rc = xlink_get_device_list(sw_device_id_list, &num_devices);
+ if (!rc && num_devices <= XLINK_MAX_DEVICE_LIST_SIZE) {
+ /* TODO: this next copy is dangerous! we have no idea
+ * how large the devl.sw_device_id_list buffer is
+ * provided by the user. if num_devices is too large,
+ * the copy will overflow the buffer.
+ */
+ if (copy_to_user((void __user *)devl.sw_device_id_list,
+ (void *)sw_device_id_list,
+ (sizeof(*sw_device_id_list)
+ * num_devices)))
+ return -EFAULT;
+ if (copy_to_user((void __user *)devl.num_devices, (void *)&num_devices,
+ (sizeof(num_devices))))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devl.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_GET_DEVICE_STATUS:
+ if (copy_from_user(&devs, (void __user *)arg,
+ sizeof(struct xlinkgetdevicestatus)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devs.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_get_device_status(&devh, &device_status);
+ if (!rc) {
+ if (copy_to_user((void __user *)devs.device_status,
+ (void *)&device_status,
+ sizeof(device_status)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devs.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_BOOT_DEVICE:
+ if (copy_from_user(&boot, (void __user *)arg,
+ sizeof(struct xlinkbootdevice)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)boot.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (boot.binary_name_size > sizeof(filename))
+ return -EINVAL;
+ memset(filename, 0, sizeof(filename));
+ if (copy_from_user(filename, (void __user *)boot.binary_name,
+ boot.binary_name_size))
+ return -EFAULT;
+ rc = xlink_boot_device(&devh, filename);
+ if (copy_to_user((void __user *)boot.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_RESET_DEVICE:
+ if (copy_from_user(&res, (void __user *)arg,
+ sizeof(struct xlinkresetdevice)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)res.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_reset_device(&devh);
+ if (copy_to_user((void __user *)res.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_GET_DEVICE_MODE:
+ if (copy_from_user(&devm, (void __user *)arg,
+ sizeof(struct xlinkdevmode)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devm.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_get_device_mode(&devh, &device_mode);
+ if (!rc) {
+ if (copy_to_user((void __user *)devm.device_mode, (void *)&device_mode,
+ sizeof(device_mode)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_SET_DEVICE_MODE:
+ if (copy_from_user(&devm, (void __user *)arg,
+ sizeof(struct xlinkdevmode)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devm.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (copy_from_user(&device_mode, (void __user *)devm.device_mode,
+ sizeof(device_mode)))
+ return -EFAULT;
+ rc = xlink_set_device_mode(&devh, device_mode);
+ if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
}
if (rc)
return -EIO;
@@ -411,6 +608,30 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
/*
* xlink Kernel API.
*/
+enum xlink_error xlink_stop_vpu(void)
+{
+#ifdef CONFIG_XLINK_LOCAL_HOST
+ int rc;
+
+ rc = xlink_ipc_reset_device(0x0); // stop vpu slice 0
+ if (rc)
+ return X_LINK_ERROR;
+#endif
+ return X_LINK_SUCCESS;
+}
+EXPORT_SYMBOL(xlink_stop_vpu);
+enum xlink_error xlink_start_vpu(char *filename)
+{
+#ifdef CONFIG_XLINK_LOCAL_HOST
+ int rc;
+
+ rc = xlink_ipc_boot_device(0x0, filename); // start vpu slice 0
+ if (rc)
+ return X_LINK_ERROR;
+#endif
+ return X_LINK_SUCCESS;
+}
+EXPORT_SYMBOL(xlink_start_vpu);
enum xlink_error xlink_initialize(void)
{
@@ -649,6 +870,33 @@ static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
return rc;
}
+enum xlink_error xlink_write_control_data(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage,
+ u32 size)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ int event_queued = 0;
+ enum xlink_error rc;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ if (size > XLINK_MAX_CONTROL_DATA_SIZE)
+ return X_LINK_ERROR; // TODO: XLink Parameter Error
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+ event = xlink_create_event(link->id, XLINK_WRITE_CONTROL_REQ,
+ &link->handle, chan, size, 0);
+ if (!event)
+ return X_LINK_ERROR;
+ memcpy(event->header.control_data, pmessage, size);
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_write_control_data);
static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
u16 chan, u8 const *message,
u32 size)
@@ -844,6 +1092,154 @@ enum xlink_error xlink_disconnect(struct xlink_handle *handle)
}
EXPORT_SYMBOL(xlink_disconnect);
+enum xlink_error xlink_get_device_list(u32 *sw_device_id_list,
+ u32 *num_devices)
+{
+ u32 interface_nmb_devices = 0;
+ enum xlink_error rc;
+ int i;
+
+ if (!xlink)
+ return X_LINK_ERROR;
+ if (!sw_device_id_list || !num_devices)
+ return X_LINK_ERROR;
+ /* loop through each interface and combine the lists */
+ for (i = 0; i < NMB_OF_INTERFACES; i++) {
+ rc = xlink_platform_get_device_list(i, sw_device_id_list,
+ &interface_nmb_devices);
+ if (!rc) {
+ *num_devices += interface_nmb_devices;
+ sw_device_id_list += interface_nmb_devices;
+ }
+ interface_nmb_devices = 0;
+ }
+ return X_LINK_SUCCESS;
+}
+EXPORT_SYMBOL(xlink_get_device_list);
+enum xlink_error xlink_get_device_name(struct xlink_handle *handle, char *name,
+ size_t name_size)
+{
+ enum xlink_error rc;
+ int interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ if (!name || !name_size)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_get_device_name(interface, handle->sw_device_id,
+ name, name_size);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_get_device_name);
+enum xlink_error xlink_get_device_status(struct xlink_handle *handle,
+ u32 *device_status)
+{
+ enum xlink_error rc;
+ u32 interface;
+
+ if (!xlink)
+ return X_LINK_ERROR;
+ if (!device_status)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_get_device_status(interface, handle->sw_device_id,
+ device_status);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_get_device_status);
+enum xlink_error xlink_boot_device(struct xlink_handle *handle,
+ const char *binary_name)
+{
+ enum xlink_error rc;
+ u32 interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ if (!binary_name)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_boot_device(interface, handle->sw_device_id,
+ binary_name);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_boot_device);
+enum xlink_error xlink_reset_device(struct xlink_handle *handle)
+{
+ enum xlink_error rc;
+ u32 interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_reset_device(interface, handle->sw_device_id);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_reset_device);
+enum xlink_error xlink_set_device_mode(struct xlink_handle *handle,
+ enum xlink_device_power_mode power_mode)
+{
+ enum xlink_error rc;
+ u32 interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_set_device_mode(interface, handle->sw_device_id,
+ power_mode);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_set_device_mode);
+enum xlink_error xlink_get_device_mode(struct xlink_handle *handle,
+ enum xlink_device_power_mode *power_mode)
+{
+ enum xlink_error rc;
+ u32 interface;
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ rc = xlink_platform_get_device_mode(interface, handle->sw_device_id,
+ power_mode);
+ if (rc)
+ rc = X_LINK_ERROR;
+ else
+ rc = X_LINK_SUCCESS;
+ return rc;
+}
+EXPORT_SYMBOL(xlink_get_device_mode);
/* Device tree driver match. */
static const struct of_device_id kmb_xlink_of_match[] = {
{
diff --git a/drivers/misc/xlink-core/xlink-defs.h b/drivers/misc/xlink-core/xlink-defs.h
index 09aee36d5542..8985f6631175 100644
--- a/drivers/misc/xlink-core/xlink-defs.h
+++ b/drivers/misc/xlink-core/xlink-defs.h
@@ -101,6 +101,7 @@ enum xlink_event_type {
XLINK_OPEN_CHANNEL_REQ,
XLINK_CLOSE_CHANNEL_REQ,
XLINK_PING_REQ,
+ XLINK_WRITE_CONTROL_REQ,
XLINK_REQ_LAST,
// response events
XLINK_WRITE_RESP = 0x10,
@@ -111,6 +112,7 @@ enum xlink_event_type {
XLINK_OPEN_CHANNEL_RESP,
XLINK_CLOSE_CHANNEL_RESP,
XLINK_PING_RESP,
+ XLINK_WRITE_CONTROL_RESP,
XLINK_RESP_LAST,
};
diff --git a/drivers/misc/xlink-core/xlink-multiplexer.c b/drivers/misc/xlink-core/xlink-multiplexer.c
index 339734826f3e..48451dc30712 100644
--- a/drivers/misc/xlink-core/xlink-multiplexer.c
+++ b/drivers/misc/xlink-core/xlink-multiplexer.c
@@ -491,6 +491,7 @@ enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
switch (event->header.type) {
case XLINK_WRITE_REQ:
case XLINK_WRITE_VOLATILE_REQ:
+ case XLINK_WRITE_CONTROL_REQ:
opchan = get_channel(link_id, chan);
if (!opchan || opchan->chan->status != CHAN_OPEN) {
rc = X_LINK_COMMUNICATION_FAIL;
@@ -657,6 +658,7 @@ enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
break;
case XLINK_WRITE_RESP:
case XLINK_WRITE_VOLATILE_RESP:
+ case XLINK_WRITE_CONTROL_RESP:
case XLINK_READ_RESP:
case XLINK_READ_TO_BUFFER_RESP:
case XLINK_RELEASE_RESP:
@@ -759,6 +761,46 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
}
release_channel(opchan);
break;
+ case XLINK_WRITE_CONTROL_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ event->header.timeout = opchan->chan->timeout;
+ buffer = xlink_platform_allocate(xmux->dev, &paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ if (buffer) {
+ size = event->header.size;
+ memcpy(buffer, event->header.control_data, size);
+ event->paddr = paddr;
+ event->data = buffer;
+ if (add_packet_to_channel(opchan,
+ &opchan->rx_queue,
+ event->data,
+ event->header.size,
+ paddr)) {
+ xlink_platform_deallocate(xmux->dev,
+ buffer, paddr,
+ event->header.size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ rc = X_LINK_ERROR;
+ release_channel(opchan);
+ break;
+ }
+ event->header.type = XLINK_WRITE_CONTROL_RESP;
+ xlink_dispatcher_event_add(EVENT_RX, event);
+ // channel blocking, notify waiting threads of available packet
+ complete(&opchan->pkt_available);
+ } else {
+ // failed to allocate buffer
+ rc = X_LINK_ERROR;
+ }
+ }
+ release_channel(opchan);
+ break;
case XLINK_READ_REQ:
case XLINK_READ_TO_BUFFER_REQ:
opchan = get_channel(link_id, chan);
@@ -848,6 +890,7 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
break;
case XLINK_WRITE_RESP:
case XLINK_WRITE_VOLATILE_RESP:
+ case XLINK_WRITE_CONTROL_RESP:
opchan = get_channel(link_id, chan);
if (!opchan)
rc = X_LINK_COMMUNICATION_FAIL;
@@ -929,6 +972,18 @@ enum xlink_error xlink_passthrough(struct xlink_event *event)
rc = X_LINK_ERROR;
}
break;
+ case XLINK_WRITE_CONTROL_REQ:
+ if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
+ ipc.is_volatile = 1;
+ rc = xlink_platform_write(IPC_INTERFACE,
+ event->handle->sw_device_id,
+ event->header.control_data,
+ &event->header.size, 0, &ipc);
+ } else {
+ /* channel not open */
+ rc = X_LINK_ERROR;
+ }
+ break;
case XLINK_READ_REQ:
if (xmux->channels[link_id][chan].ipc_status == CHAN_OPEN) {
/* if channel has receive blocking set,
@@ -1013,6 +1068,7 @@ enum xlink_error xlink_passthrough(struct xlink_event *event)
case XLINK_PING_REQ:
case XLINK_WRITE_RESP:
case XLINK_WRITE_VOLATILE_RESP:
+ case XLINK_WRITE_CONTROL_RESP:
case XLINK_READ_RESP:
case XLINK_READ_TO_BUFFER_RESP:
case XLINK_RELEASE_RESP:
diff --git a/drivers/misc/xlink-core/xlink-platform.c b/drivers/misc/xlink-core/xlink-platform.c
index c34b69ee206b..56eb8da28a5f 100644
--- a/drivers/misc/xlink-core/xlink-platform.c
+++ b/drivers/misc/xlink-core/xlink-platform.c
@@ -34,6 +34,20 @@ static inline int xlink_ipc_read(u32 sw_device_id, void *data,
size_t * const size, u32 timeout, void *context)
{ return -1; }
+static inline int xlink_ipc_get_device_list(u32 *sw_device_id_list,
+ u32 *num_devices)
+{ return -1; }
+static inline int xlink_ipc_get_device_name(u32 sw_device_id,
+ char *device_name, size_t name_size)
+{ return -1; }
+static inline int xlink_ipc_get_device_status(u32 sw_device_id,
+ u32 *device_status)
+{ return -1; }
+static inline int xlink_ipc_boot_device(u32 sw_device_id,
+ const char *binary_path)
+{ return -1; }
+static inline int xlink_ipc_reset_device(u32 sw_device_id)
+{ return -1; }
static inline int xlink_ipc_open_channel(u32 sw_device_id,
u32 channel)
{ return -1; }
@@ -59,6 +73,23 @@ static int (*write_fcts[NMB_OF_INTERFACES])(u32, void *, size_t * const, u32) =
static int (*read_fcts[NMB_OF_INTERFACES])(u32, void *, size_t * const, u32) = {
NULL, xlink_pcie_read, NULL, NULL};
+static int (*reset_fcts[NMB_OF_INTERFACES])(u32) = {
+ xlink_ipc_reset_device, xlink_pcie_reset_device, NULL, NULL};
+static int (*boot_fcts[NMB_OF_INTERFACES])(u32, const char *) = {
+ xlink_ipc_boot_device, xlink_pcie_boot_device, NULL, NULL};
+static int (*dev_name_fcts[NMB_OF_INTERFACES])(u32, char *, size_t) = {
+ xlink_ipc_get_device_name, xlink_pcie_get_device_name,
+ NULL, NULL};
+static int (*dev_list_fcts[NMB_OF_INTERFACES])(u32 *, u32 *) = {
+ xlink_ipc_get_device_list, xlink_pcie_get_device_list,
+ NULL, NULL};
+static int (*dev_status_fcts[NMB_OF_INTERFACES])(u32, u32 *) = {
+ xlink_ipc_get_device_status, xlink_pcie_get_device_status,
+ NULL, NULL};
+static int (*dev_set_mode_fcts[NMB_OF_INTERFACES])(u32, u32) = {
+ NULL, NULL, NULL, NULL};
+static int (*dev_get_mode_fcts[NMB_OF_INTERFACES])(u32, u32 *) = {
+ NULL, NULL, NULL, NULL};
static int (*open_chan_fcts[NMB_OF_INTERFACES])(u32, u32) = {
xlink_ipc_open_channel, NULL, NULL, NULL};
@@ -103,6 +134,61 @@ int xlink_platform_read(u32 interface, u32 sw_device_id, void *data,
return read_fcts[interface](sw_device_id, data, size, timeout);
}
+int xlink_platform_reset_device(u32 interface, u32 sw_device_id)
+{
+ if (interface >= NMB_OF_INTERFACES || !reset_fcts[interface])
+ return -1;
+ return reset_fcts[interface](sw_device_id);
+}
+
+int xlink_platform_boot_device(u32 interface, u32 sw_device_id,
+ const char *binary_name)
+{
+ if (interface >= NMB_OF_INTERFACES || !boot_fcts[interface])
+ return -1;
+ return boot_fcts[interface](sw_device_id, binary_name);
+}
+
+int xlink_platform_get_device_name(u32 interface, u32 sw_device_id,
+ char *device_name, size_t name_size)
+{
+ if (interface >= NMB_OF_INTERFACES || !dev_name_fcts[interface])
+ return -1;
+ return dev_name_fcts[interface](sw_device_id, device_name, name_size);
+}
+
+int xlink_platform_get_device_list(u32 interface,
+ u32 *sw_device_id_list, u32 *num_devices)
+{
+ if (interface >= NMB_OF_INTERFACES || !dev_list_fcts[interface])
+ return -1;
+ return dev_list_fcts[interface](sw_device_id_list, num_devices);
+}
+
+int xlink_platform_get_device_status(u32 interface, u32 sw_device_id,
+ u32 *device_status)
+{
+ if (interface >= NMB_OF_INTERFACES || !dev_status_fcts[interface])
+ return -1;
+ return dev_status_fcts[interface](sw_device_id, device_status);
+}
+
+int xlink_platform_set_device_mode(u32 interface, u32 sw_device_id,
+ u32 power_mode)
+{
+ if (interface >= NMB_OF_INTERFACES || !dev_set_mode_fcts[interface])
+ return -1;
+ return dev_set_mode_fcts[interface](sw_device_id, power_mode);
+}
+
+int xlink_platform_get_device_mode(u32 interface, u32 sw_device_id,
+ u32 *power_mode)
+{
+ if (interface >= NMB_OF_INTERFACES || !dev_get_mode_fcts[interface])
+ return -1;
+ return dev_get_mode_fcts[interface](sw_device_id, power_mode);
+}
+
int xlink_platform_open_channel(u32 interface, u32 sw_device_id,
u32 channel)
{
diff --git a/include/linux/xlink.h b/include/linux/xlink.h
index c22439d5aade..b00dbc719530 100644
--- a/include/linux/xlink.h
+++ b/include/linux/xlink.h
@@ -78,6 +78,10 @@ enum xlink_error xlink_write_data(struct xlink_handle *handle,
enum xlink_error xlink_write_volatile(struct xlink_handle *handle,
u16 chan, u8 const *message, u32 size);
+enum xlink_error xlink_write_control_data(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size);
+
enum xlink_error xlink_read_data(struct xlink_handle *handle,
u16 chan, u8 **message, u32 *size);
@@ -90,6 +94,29 @@ enum xlink_error xlink_release_data(struct xlink_handle *handle,
enum xlink_error xlink_disconnect(struct xlink_handle *handle);
+enum xlink_error xlink_get_device_list(u32 *sw_device_id_list, u32 *num_devices);
+
+enum xlink_error xlink_get_device_name(struct xlink_handle *handle, char *name,
+ size_t name_size);
+
+enum xlink_error xlink_get_device_status(struct xlink_handle *handle,
+ u32 *device_status);
+
+enum xlink_error xlink_boot_device(struct xlink_handle *handle,
+ const char *binary_name);
+
+enum xlink_error xlink_reset_device(struct xlink_handle *handle);
+
+enum xlink_error xlink_set_device_mode(struct xlink_handle *handle,
+ enum xlink_device_power_mode power_mode);
+
+enum xlink_error xlink_get_device_mode(struct xlink_handle *handle,
+ enum xlink_device_power_mode *power_mode);
+
+enum xlink_error xlink_start_vpu(char *filename); /* depreciated */
+
+enum xlink_error xlink_stop_vpu(void); /* depreciated */
+
/* API functions to be implemented
*
* enum xlink_error xlink_write_crc_data(struct xlink_handle *handle,
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 21/22] xlink-core: add async channel and events
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (19 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 20/22] xlink-core: Enable VPU IP management and runtime control mgross
@ 2020-11-30 23:07 ` mgross
2020-11-30 23:07 ` [PATCH 22/22] xlink-core: factorize xlink_ioctl function by creating sub-functions for each ioctl command mgross
2020-12-01 10:14 ` [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 Greg KH
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Arnd Bergmann, Greg Kroah-Hartman
From: Seamus Kelly <seamus.kelly@intel.com>
Enable asynchronous channel and event communication.
Add APIs:
data ready callback:
The xLink Data Ready Callback function is used to
register a callback function that is invoked when data
is ready to be read from a channel
data consumed callback:
The xLink Data Consumed Callback function is used to
register a callback function that is invoked when data
is consumed by the peer node on a channel
Add event notification handling including APIs:
register device event:
The xLink Register Device Event function is used to
register a callback for notification of certain system
events. Currently XLink supports 4 such events [0-3]
whose meaning is system dependent. Registering for an
event means that the callback will be called when the
event occurs with 2 parameters the sw_device_id of the
device that triggered the event and the event number [0-3]
unregister device event
The xLink Unregister Device Event function is used to
unregister events that have previously been registered
by register device event API
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
---
drivers/misc/xlink-core/xlink-core.c | 422 +++++++++++++++++++-
drivers/misc/xlink-core/xlink-defs.h | 4 +
drivers/misc/xlink-core/xlink-dispatcher.c | 53 ++-
drivers/misc/xlink-core/xlink-multiplexer.c | 176 +++++---
drivers/misc/xlink-core/xlink-platform.c | 27 ++
include/linux/xlink.h | 15 +-
6 files changed, 609 insertions(+), 88 deletions(-)
diff --git a/drivers/misc/xlink-core/xlink-core.c b/drivers/misc/xlink-core/xlink-core.c
index a600068840d3..63f65def8aa9 100644
--- a/drivers/misc/xlink-core/xlink-core.c
+++ b/drivers/misc/xlink-core/xlink-core.c
@@ -62,6 +62,16 @@ static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
u16 chan, u8 const *message,
u32 size, u32 user_flag);
+static enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
+ u32 *event_list,
+ u32 num_events,
+ xlink_device_event_cb event_notif_fn);
+static enum xlink_error do_xlink_register_device_event(struct xlink_handle *handle,
+ u32 *event_list,
+ u32 num_events,
+ xlink_device_event_cb event_notif_fn,
+ u32 user_flag);
+static struct mutex dev_event_lock;
static const struct file_operations fops = {
.owner = THIS_MODULE,
@@ -74,14 +84,81 @@ struct xlink_link {
struct kref refcount;
};
+struct xlink_attr {
+ unsigned long value;
+ u32 sw_dev_id;
+};
+
struct keembay_xlink_dev {
struct platform_device *pdev;
struct xlink_link links[XLINK_MAX_CONNECTIONS];
u32 nmb_connected_links;
struct mutex lock; // protect access to xlink_dev
+ struct xlink_attr eventx[4];
+};
+
+struct event_info {
+ struct list_head list;
+ u32 sw_device_id;
+ u32 event_type;
+ u32 user_flag;
+ xlink_device_event_cb event_notif_fn;
};
static u8 volbuf[XLINK_MAX_BUF_SIZE]; // buffer for volatile transactions
+#define NUM_REG_EVENTS 4
+
+// sysfs attribute functions
+
+static ssize_t event0_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct keembay_xlink_dev *xlink_dev = dev_get_drvdata(dev);
+ struct xlink_attr *a = &xlink_dev->eventx[0];
+
+ return scnprintf(buf, PAGE_SIZE, "0x%x 0x%lx\n", a->sw_dev_id, a->value);
+}
+
+static ssize_t event1_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct keembay_xlink_dev *xlink_dev = dev_get_drvdata(dev);
+ struct xlink_attr *a = &xlink_dev->eventx[1];
+
+ return scnprintf(buf, PAGE_SIZE, "0x%x 0x%lx\n", a->sw_dev_id, a->value);
+}
+
+static ssize_t event2_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct keembay_xlink_dev *xlink_dev = dev_get_drvdata(dev);
+ struct xlink_attr *a = &xlink_dev->eventx[2];
+
+ return scnprintf(buf, PAGE_SIZE, "0x%x 0x%lx\n", a->sw_dev_id, a->value);
+}
+
+static ssize_t event3_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct keembay_xlink_dev *xlink_dev = dev_get_drvdata(dev);
+ struct xlink_attr *a = &xlink_dev->eventx[3];
+
+ return scnprintf(buf, PAGE_SIZE, "0x%x 0x%lx\n", a->sw_dev_id, a->value);
+}
+
+static DEVICE_ATTR_RO(event0);
+static DEVICE_ATTR_RO(event1);
+static DEVICE_ATTR_RO(event2);
+static DEVICE_ATTR_RO(event3);
+static struct attribute *xlink_sysfs_entries[] = {
+ &dev_attr_event0.attr,
+ &dev_attr_event1.attr,
+ &dev_attr_event2.attr,
+ &dev_attr_event3.attr,
+ NULL,
+};
+
+static const struct attribute_group xlink_sysfs_group = {
+ .attrs = xlink_sysfs_entries,
+};
+
+static struct event_info ev_info;
/*
* global variable pointing to our xlink device.
@@ -215,7 +292,14 @@ static int kmb_xlink_probe(struct platform_device *pdev)
dev_info(&pdev->dev, "Cannot add the device to the system\n");
goto r_class;
}
+ INIT_LIST_HEAD(&ev_info.list);
+ rc = devm_device_add_group(&pdev->dev, &xlink_sysfs_group);
+ if (rc) {
+ dev_err(&pdev->dev, "failed to create sysfs entries: %d\n", rc);
+ return rc;
+ }
+ mutex_init(&dev_event_lock);
return 0;
r_device:
@@ -239,7 +323,6 @@ static int kmb_xlink_remove(struct platform_device *pdev)
rc = xlink_multiplexer_destroy();
if (rc != X_LINK_SUCCESS)
pr_err("Multiplexer destroy failed\n");
- // stop dispatchers and destroy
rc = xlink_dispatcher_destroy();
if (rc != X_LINK_SUCCESS)
pr_err("Dispatcher destroy failed\n");
@@ -270,19 +353,23 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
struct xlinkconnect con = {0};
struct xlinkrelease rel = {0};
struct xlinkstartvpu startvpu = {0};
+ struct xlinkcallback cb = {0};
struct xlinkgetdevicename devn = {0};
struct xlinkgetdevicelist devl = {0};
struct xlinkgetdevicestatus devs = {0};
struct xlinkbootdevice boot = {0};
struct xlinkresetdevice res = {0};
struct xlinkdevmode devm = {0};
+ struct xlinkregdevevent regdevevent = {0};
u32 sw_device_id_list[XLINK_MAX_DEVICE_LIST_SIZE];
char name[XLINK_MAX_DEVICE_NAME_SIZE];
int interface = NULL_INTERFACE;
u32 device_status = 0;
u32 num_devices = 0;
u32 device_mode = 0;
+ u32 num_events = 0;
char filename[64];
+ u32 *ev_list;
u8 reladdr;
u8 *rdaddr;
u32 size;
@@ -318,6 +405,30 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
return -EFAULT;
break;
+ case XL_DATA_READY_CALLBACK:
+ if (copy_from_user(&cb, (void __user *)arg,
+ sizeof(struct xlinkcallback)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)cb.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
+ rc = xlink_data_available_event(&devh, cb.chan, cb.callback);
+ if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_DATA_CONSUMED_CALLBACK:
+ if (copy_from_user(&cb, (void __user *)arg,
+ sizeof(struct xlinkcallback)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)cb.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
+ rc = xlink_data_consumed_event(&devh, cb.chan, cb.callback);
+ if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
case XL_READ_DATA:
if (copy_from_user(&rd, (void __user *)arg,
sizeof(struct xlinkreaddata)))
@@ -598,6 +709,61 @@ static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
return -EFAULT;
break;
+ case XL_REGISTER_DEV_EVENT:
+ if (copy_from_user(®devevent, (void __user *)arg,
+ sizeof(struct xlinkregdevevent)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)regdevevent.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ num_events = regdevevent.num_events;
+ if (num_events > 0 && num_events <= NUM_REG_EVENTS) {
+ ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
+ if (ev_list) {
+ if (copy_from_user(ev_list,
+ (void __user *)regdevevent.event_list,
+ (num_events * sizeof(u32)))) {
+ kfree(ev_list);
+ return -EFAULT;
+ }
+ rc = xlink_register_device_event_user(&devh,
+ ev_list,
+ num_events,
+ NULL);
+ kfree(ev_list);
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
+ case XL_UNREGISTER_DEV_EVENT:
+ if (copy_from_user(®devevent, (void __user *)arg,
+ sizeof(struct xlinkregdevevent)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)regdevevent.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ num_events = regdevevent.num_events;
+ if (num_events <= NUM_REG_EVENTS) {
+ ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
+ if (copy_from_user(ev_list,
+ (void __user *)regdevevent.event_list,
+ (num_events * sizeof(u32)))) {
+ kfree(ev_list);
+ return -EFAULT;
+ }
+ rc = xlink_unregister_device_event(&devh, ev_list, num_events);
+ kfree(ev_list);
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ break;
}
if (rc)
return -EIO;
@@ -671,14 +837,12 @@ enum xlink_error xlink_connect(struct xlink_handle *handle)
xlink->nmb_connected_links++;
kref_init(&link->refcount);
if (interface != IPC_INTERFACE) {
- // start dispatcher
rc = xlink_dispatcher_start(link->id, &link->handle);
if (rc) {
pr_err("dispatcher start failed\n");
goto r_cleanup;
}
}
- // initialize multiplexer connection
rc = xlink_multiplexer_connect(link->id);
if (rc) {
pr_err("multiplexer connect failed\n");
@@ -689,7 +853,6 @@ enum xlink_error xlink_connect(struct xlink_handle *handle)
link->handle.dev_type,
xlink->nmb_connected_links);
} else {
- // already connected
pr_info("dev 0x%x ALREADY connected - dev_type %d\n",
link->handle.sw_device_id,
link->handle.dev_type);
@@ -697,7 +860,6 @@ enum xlink_error xlink_connect(struct xlink_handle *handle)
*handle = link->handle;
}
mutex_unlock(&xlink->lock);
- // TODO: implement ping
return X_LINK_SUCCESS;
r_cleanup:
@@ -707,6 +869,78 @@ enum xlink_error xlink_connect(struct xlink_handle *handle)
}
EXPORT_SYMBOL(xlink_connect);
+enum xlink_error xlink_data_available_event(struct xlink_handle *handle,
+ u16 chan,
+ xlink_event data_available_event)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int event_queued = 0;
+ char origin = 'K';
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ if (CHANNEL_USER_BIT_IS_SET(chan))
+ origin = 'U'; // function called from user space
+ CHANNEL_CLEAR_USER_BIT(chan); // restore proper channel value
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+ event = xlink_create_event(link->id, XLINK_DATA_READY_CALLBACK_REQ,
+ &link->handle, chan, 0, 0);
+ if (!event)
+ return X_LINK_ERROR;
+ event->data = data_available_event;
+ event->callback_origin = origin;
+ if (!data_available_event)
+ event->calling_pid = NULL; // disable callbacks on this channel
+ else
+ event->calling_pid = current;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_data_available_event);
+enum xlink_error xlink_data_consumed_event(struct xlink_handle *handle,
+ u16 chan,
+ xlink_event data_consumed_event)
+{
+ struct xlink_event *event;
+ struct xlink_link *link;
+ enum xlink_error rc;
+ int event_queued = 0;
+ char origin = 'K';
+
+ if (!xlink || !handle)
+ return X_LINK_ERROR;
+
+ if (CHANNEL_USER_BIT_IS_SET(chan))
+ origin = 'U'; // function called from user space
+ CHANNEL_CLEAR_USER_BIT(chan); // restore proper channel value
+
+ link = get_link_by_sw_device_id(handle->sw_device_id);
+ if (!link)
+ return X_LINK_ERROR;
+ event = xlink_create_event(link->id, XLINK_DATA_CONSUMED_CALLBACK_REQ,
+ &link->handle, chan, 0, 0);
+ if (!event)
+ return X_LINK_ERROR;
+ event->data = data_consumed_event;
+ event->callback_origin = origin;
+ if (!data_consumed_event)
+ event->calling_pid = NULL; // disable callbacks on this channel
+ else
+ event->calling_pid = current;
+ rc = xlink_multiplexer_tx(event, &event_queued);
+ if (!event_queued)
+ xlink_destroy_event(event);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_data_consumed_event);
enum xlink_error xlink_open_channel(struct xlink_handle *handle,
u16 chan, enum xlink_opmode mode,
u32 data_size, u32 timeout)
@@ -1051,8 +1285,8 @@ EXPORT_SYMBOL(xlink_release_data);
enum xlink_error xlink_disconnect(struct xlink_handle *handle)
{
struct xlink_link *link;
- int interface = NULL_INTERFACE;
- enum xlink_error rc = X_LINK_ERROR;
+ int interface;
+ enum xlink_error rc = 0;
if (!xlink || !handle)
return X_LINK_ERROR;
@@ -1061,7 +1295,6 @@ enum xlink_error xlink_disconnect(struct xlink_handle *handle)
if (!link)
return X_LINK_ERROR;
- // decrement refcount, if count is 0 lock mutex and disconnect
if (kref_put_mutex(&link->refcount, release_after_kref_put,
&xlink->lock)) {
// stop dispatcher
@@ -1240,6 +1473,179 @@ enum xlink_error xlink_get_device_mode(struct xlink_handle *handle,
return rc;
}
EXPORT_SYMBOL(xlink_get_device_mode);
+
+static int xlink_device_event_handler(u32 sw_device_id, u32 event_type)
+{
+ struct event_info *events = NULL;
+ xlink_device_event_cb event_cb;
+ bool found = false;
+ char event_attr[7];
+
+ mutex_lock(&dev_event_lock);
+ // find sw_device_id, event_type in list
+ list_for_each_entry(events, &ev_info.list, list) {
+ if (events) {
+ if (events->sw_device_id == sw_device_id &&
+ events->event_type == event_type) {
+ event_cb = events->event_notif_fn;
+ found = true;
+ break;
+ }
+ }
+ }
+ if (found) {
+ if (events->user_flag) {
+ xlink->eventx[events->event_type].value = events->event_type;
+ xlink->eventx[events->event_type].sw_dev_id = sw_device_id;
+ sprintf(event_attr, "event%d", events->event_type);
+ sysfs_notify(&xlink->pdev->dev.kobj, NULL, event_attr);
+ } else {
+ if (event_cb) {
+ event_cb(sw_device_id, event_type);
+ } else {
+ pr_info("No callback found for sw_device_id : 0x%x event type %d\n",
+ sw_device_id, event_type);
+ mutex_unlock(&dev_event_lock);
+ return X_LINK_ERROR;
+ }
+ }
+ pr_info("sysfs_notify event %d swdev_id %xs\n",
+ events->event_type, sw_device_id);
+ }
+ mutex_unlock(&dev_event_lock);
+return X_LINK_SUCCESS;
+}
+
+static bool event_registered(u32 sw_dev_id, u32 event)
+{
+ struct event_info *events = NULL;
+
+ list_for_each_entry(events, &ev_info.list, list) {
+ if (events) {
+ if (events->sw_device_id == sw_dev_id &&
+ events->event_type == event) {
+ return true;
+ }
+ }
+ }
+return false;
+}
+
+static enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn)
+{
+ enum xlink_error rc;
+
+ rc = do_xlink_register_device_event(handle, event_list, num_events,
+ event_notif_fn, 1);
+ return rc;
+}
+
+enum xlink_error xlink_register_device_event(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn)
+{
+ enum xlink_error rc;
+
+ rc = do_xlink_register_device_event(handle, event_list, num_events,
+ event_notif_fn, 0);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_register_device_event);
+
+static enum xlink_error do_xlink_register_device_event(struct xlink_handle *handle,
+ u32 *event_list,
+ u32 num_events,
+ xlink_device_event_cb event_notif_fn,
+ u32 user_flag)
+{
+ struct event_info *events;
+ u32 interface;
+ u32 event;
+ int i;
+
+ if (num_events < 0 || num_events >= NUM_REG_EVENTS)
+ return X_LINK_ERROR;
+ for (i = 0; i < num_events; i++) {
+ events = kzalloc(sizeof(*events), GFP_KERNEL);
+ if (!events)
+ return X_LINK_ERROR;
+ event = *event_list;
+ events->sw_device_id = handle->sw_device_id;
+ events->event_notif_fn = event_notif_fn;
+ events->event_type = *event_list++;
+ events->user_flag = user_flag;
+ if (user_flag) {
+ /* only add to list once if userspace */
+ /* xlink userspace handles multi process callbacks */
+ if (event_registered(handle->sw_device_id, event)) {
+ pr_info("xlink-core: Event 0x%x - %d already registered\n",
+ handle->sw_device_id, event);
+ kfree(events);
+ continue;
+ }
+ }
+ pr_info("xlink-core:Events: sw_device_id 0x%x event %d fn %p user_flag %d\n",
+ events->sw_device_id, events->event_type,
+ events->event_notif_fn, events->user_flag);
+ list_add_tail(&events->list, &ev_info.list);
+ }
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ xlink_platform_register_for_events(interface, handle->sw_device_id,
+ xlink_device_event_handler);
+ return X_LINK_SUCCESS;
+}
+
+enum xlink_error xlink_unregister_device_event(struct xlink_handle *handle,
+ u32 *event_list,
+ u32 num_events)
+{
+ struct event_info *events = NULL;
+ u32 interface;
+ int found = 0;
+ int count = 0;
+ int i;
+
+ if (num_events < 0 || num_events >= NUM_REG_EVENTS)
+ return X_LINK_ERROR;
+ for (i = 0; i < num_events; i++) {
+ list_for_each_entry(events, &ev_info.list, list) {
+ if (events->sw_device_id == handle->sw_device_id &&
+ events->event_type == event_list[i]) {
+ found = 1;
+ break;
+ }
+ }
+ if (!events || !found)
+ return X_LINK_ERROR;
+ pr_info("removing event %d for sw_device_id 0x%x\n",
+ events->event_type, events->sw_device_id);
+ list_del(&events->list);
+ kfree(events);
+ }
+ // check if any events left for this sw_device_id
+ // are still registered ( in list )
+ list_for_each_entry(events, &ev_info.list, list) {
+ if (events) {
+ if (events->sw_device_id == handle->sw_device_id) {
+ count++;
+ break;
+ }
+ }
+ }
+ if (count == 0) {
+ interface = get_interface_from_sw_device_id(handle->sw_device_id);
+ if (interface == NULL_INTERFACE)
+ return X_LINK_ERROR;
+ xlink_platform_unregister_for_events(interface, handle->sw_device_id);
+ }
+ return X_LINK_SUCCESS;
+}
+EXPORT_SYMBOL(xlink_unregister_device_event);
+
/* Device tree driver match. */
static const struct of_device_id kmb_xlink_of_match[] = {
{
diff --git a/drivers/misc/xlink-core/xlink-defs.h b/drivers/misc/xlink-core/xlink-defs.h
index 8985f6631175..df686e5185c5 100644
--- a/drivers/misc/xlink-core/xlink-defs.h
+++ b/drivers/misc/xlink-core/xlink-defs.h
@@ -102,6 +102,8 @@ enum xlink_event_type {
XLINK_CLOSE_CHANNEL_REQ,
XLINK_PING_REQ,
XLINK_WRITE_CONTROL_REQ,
+ XLINK_DATA_READY_CALLBACK_REQ,
+ XLINK_DATA_CONSUMED_CALLBACK_REQ,
XLINK_REQ_LAST,
// response events
XLINK_WRITE_RESP = 0x10,
@@ -113,6 +115,8 @@ enum xlink_event_type {
XLINK_CLOSE_CHANNEL_RESP,
XLINK_PING_RESP,
XLINK_WRITE_CONTROL_RESP,
+ XLINK_DATA_READY_CALLBACK_RESP,
+ XLINK_DATA_CONSUMED_CALLBACK_RESP,
XLINK_RESP_LAST,
};
diff --git a/drivers/misc/xlink-core/xlink-dispatcher.c b/drivers/misc/xlink-core/xlink-dispatcher.c
index 11ef8e4110ca..bc2f184488ac 100644
--- a/drivers/misc/xlink-core/xlink-dispatcher.c
+++ b/drivers/misc/xlink-core/xlink-dispatcher.c
@@ -5,18 +5,18 @@
* Copyright (C) 2018-2019 Intel Corporation
*
*/
-#include <linux/init.h>
-#include <linux/module.h>
+#include <linux/completion.h>
#include <linux/device.h>
+#include <linux/init.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/list.h>
-#include <linux/semaphore.h>
+#include <linux/module.h>
#include <linux/mutex.h>
-#include <linux/completion.h>
-#include <linux/sched/signal.h>
#include <linux/platform_device.h>
+#include <linux/sched/signal.h>
+#include <linux/semaphore.h>
+#include <linux/slab.h>
#include "xlink-dispatcher.h"
#include "xlink-multiplexer.h"
@@ -34,18 +34,18 @@ enum dispatcher_state {
/* queue for dispatcher tx thread event handling */
struct event_queue {
+ struct list_head head; /* head of event linked list */
u32 count; /* number of events in the queue */
u32 capacity; /* capacity of events in the queue */
- struct list_head head; /* head of event linked list */
struct mutex lock; /* locks queue while accessing */
};
/* dispatcher servicing a single link to a device */
struct dispatcher {
u32 link_id; /* id of link being serviced */
+ int interface; /* underlying interface of link */
enum dispatcher_state state; /* state of the dispatcher */
struct xlink_handle *handle; /* xlink device handle */
- int interface; /* underlying interface of link */
struct task_struct *rxthread; /* kthread servicing rx */
struct task_struct *txthread; /* kthread servicing tx */
struct event_queue queue; /* xlink event queue */
@@ -82,7 +82,7 @@ static struct dispatcher *get_dispatcher_by_id(u32 id)
static u32 event_generate_id(void)
{
- static u32 id = 0xa; // TODO: temporary solution
+ static u32 id = 0xa;
return id++;
}
@@ -142,9 +142,6 @@ static int dispatcher_event_send(struct xlink_event *event)
size_t event_header_size = sizeof(event->header);
int rc;
- // write event header
- // printk(KERN_DEBUG "Sending event: type = 0x%x, id = 0x%x\n",
- // event->header.type, event->header.id);
rc = xlink_platform_write(event->interface,
event->handle->sw_device_id, &event->header,
&event_header_size, event->header.timeout, NULL);
@@ -159,8 +156,10 @@ static int dispatcher_event_send(struct xlink_event *event)
event->handle->sw_device_id, event->data,
&event->header.size, event->header.timeout,
NULL);
- if (rc)
+ if (rc) {
pr_err("Write data failed %d\n", rc);
+ return rc;
+ }
if (event->user_data == 1) {
if (event->paddr != 0) {
xlink_platform_deallocate(xlinkd->dev,
@@ -187,7 +186,6 @@ static int xlink_dispatcher_rxthread(void *context)
size_t size;
int rc;
- // printk(KERN_DEBUG "dispatcher rxthread started\n");
event = xlink_create_event(disp->link_id, 0, disp->handle, 0, 0, 0);
if (!event)
return -1;
@@ -214,7 +212,6 @@ static int xlink_dispatcher_rxthread(void *context)
}
}
}
- // printk(KERN_INFO "dispatcher rxthread stopped\n");
complete(&disp->rx_done);
do_exit(0);
return 0;
@@ -225,7 +222,6 @@ static int xlink_dispatcher_txthread(void *context)
struct dispatcher *disp = (struct dispatcher *)context;
struct xlink_event *event;
- // printk(KERN_DEBUG "dispatcher txthread started\n");
allow_signal(SIGTERM); // allow thread termination while waiting on sem
complete(&disp->tx_done);
while (!kthread_should_stop()) {
@@ -236,7 +232,6 @@ static int xlink_dispatcher_txthread(void *context)
dispatcher_event_send(event);
xlink_destroy_event(event); // free handled event
}
- // printk(KERN_INFO "dispatcher txthread stopped\n");
complete(&disp->tx_done);
do_exit(0);
return 0;
@@ -250,6 +245,7 @@ static int xlink_dispatcher_txthread(void *context)
enum xlink_error xlink_dispatcher_init(void *dev)
{
struct platform_device *plat_dev = (struct platform_device *)dev;
+ struct dispatcher *dsp;
int i;
xlinkd = kzalloc(sizeof(*xlinkd), GFP_KERNEL);
@@ -258,16 +254,16 @@ enum xlink_error xlink_dispatcher_init(void *dev)
xlinkd->dev = &plat_dev->dev;
for (i = 0; i < XLINK_MAX_CONNECTIONS; i++) {
- xlinkd->dispatchers[i].link_id = i;
- sema_init(&xlinkd->dispatchers[i].event_sem, 0);
- init_completion(&xlinkd->dispatchers[i].rx_done);
- init_completion(&xlinkd->dispatchers[i].tx_done);
- INIT_LIST_HEAD(&xlinkd->dispatchers[i].queue.head);
- mutex_init(&xlinkd->dispatchers[i].queue.lock);
- xlinkd->dispatchers[i].queue.count = 0;
- xlinkd->dispatchers[i].queue.capacity =
- XLINK_EVENT_QUEUE_CAPACITY;
- xlinkd->dispatchers[i].state = XLINK_DISPATCHER_INIT;
+ dsp = &xlinkd->dispatchers[i];
+ dsp->link_id = i;
+ sema_init(&dsp->event_sem, 0);
+ init_completion(&dsp->rx_done);
+ init_completion(&dsp->tx_done);
+ INIT_LIST_HEAD(&dsp->queue.head);
+ mutex_init(&dsp->queue.lock);
+ dsp->queue.count = 0;
+ dsp->queue.capacity = XLINK_EVENT_QUEUE_CAPACITY;
+ dsp->state = XLINK_DISPATCHER_INIT;
}
mutex_init(&xlinkd->lock);
@@ -329,7 +325,7 @@ enum xlink_error xlink_dispatcher_event_add(enum xlink_event_origin origin,
struct dispatcher *disp;
int rc;
- // get dispatcher by handle
+ // get dispatcher by link id
disp = get_dispatcher_by_id(event->link_id);
if (!disp)
return X_LINK_ERROR;
@@ -433,7 +429,6 @@ enum xlink_error xlink_dispatcher_destroy(void)
}
xlink_destroy_event(event);
}
- // destroy dispatcher
mutex_destroy(&disp->queue.lock);
}
mutex_destroy(&xlinkd->lock);
diff --git a/drivers/misc/xlink-core/xlink-multiplexer.c b/drivers/misc/xlink-core/xlink-multiplexer.c
index 48451dc30712..e09458b62c45 100644
--- a/drivers/misc/xlink-core/xlink-multiplexer.c
+++ b/drivers/misc/xlink-core/xlink-multiplexer.c
@@ -115,6 +115,38 @@ static struct xlink_multiplexer *xmux;
*
*/
+static enum xlink_error run_callback(struct open_channel *opchan,
+ void *callback, struct task_struct *pid)
+{
+ enum xlink_error rc = X_LINK_SUCCESS;
+ struct kernel_siginfo info;
+ void (*func)(int chan);
+ int ret;
+
+ if (opchan->callback_origin == 'U') { // user-space origin
+ if (pid) {
+ memset(&info, 0, sizeof(struct kernel_siginfo));
+ info.si_signo = SIGXLNK;
+ info.si_code = SI_QUEUE;
+ info.si_errno = opchan->id;
+ info.si_ptr = (void __user *)callback;
+ ret = send_sig_info(SIGXLNK, &info, pid);
+ if (ret < 0) {
+ pr_err("Unable to send signal %d\n", ret);
+ rc = X_LINK_ERROR;
+ }
+ } else {
+ pr_err("CHAN 0x%x -- calling_pid == NULL\n",
+ opchan->id);
+ rc = X_LINK_ERROR;
+ }
+ } else { // kernel origin
+ func = callback;
+ func(opchan->id);
+ }
+ return rc;
+}
+
static inline int chan_is_non_blocking_read(struct open_channel *opchan)
{
if (opchan->chan->mode == RXN_TXN || opchan->chan->mode == RXN_TXB)
@@ -151,7 +183,7 @@ static int is_channel_for_device(u16 chan, u32 sw_device_id,
enum xlink_dev_type dev_type)
{
struct xlink_channel_type const *chan_type = get_channel_type(chan);
- int interface = NULL_INTERFACE;
+ int interface;
if (chan_type) {
interface = get_interface_from_sw_device_id(sw_device_id);
@@ -181,13 +213,9 @@ static int is_enough_space_in_channel(struct open_channel *opchan,
}
}
if (opchan->tx_up_limit == 1) {
- if ((opchan->tx_fill_level + size)
- < ((opchan->chan->size / 100) * THR_LWR)) {
- opchan->tx_up_limit = 0;
- return 1;
- } else {
+ if ((opchan->tx_fill_level + size) >=
+ ((opchan->chan->size / 100) * THR_LWR))
return 0;
- }
}
return 1;
}
@@ -231,6 +259,8 @@ static int add_packet_to_channel(struct open_channel *opchan,
list_add_tail(&pkt->list, &queue->head);
queue->count++;
opchan->rx_fill_level += pkt->length;
+ } else {
+ return X_LINK_ERROR;
}
return X_LINK_SUCCESS;
}
@@ -262,9 +292,11 @@ static int release_packet_from_channel(struct open_channel *opchan,
} else {
// find packet in channel rx queue
list_for_each_entry(pkt, &queue->head, list) {
- if (pkt->data == addr) {
- packet_found = 1;
- break;
+ if (pkt) {
+ if (pkt->data == addr) {
+ packet_found = 1;
+ break;
+ }
}
}
}
@@ -629,16 +661,46 @@ enum xlink_error xlink_multiplexer_tx(struct xlink_event *event,
}
} else if (xmux->channels[link_id][chan].status == CHAN_OPEN_PEER) {
/* channel already open */
- xmux->channels[link_id][chan].status = CHAN_OPEN; // opened locally
- xmux->channels[link_id][chan].size = event->header.size;
- xmux->channels[link_id][chan].timeout = event->header.timeout;
- xmux->channels[link_id][chan].mode = (uintptr_t)event->data;
rc = multiplexer_open_channel(link_id, chan);
+ if (rc == X_LINK_SUCCESS) {
+ struct channel *xchan = &xmux->channels[link_id][chan];
+
+ xchan->status = CHAN_OPEN; // opened locally
+ xchan->size = event->header.size;
+ xchan->timeout = event->header.timeout;
+ xchan->mode = (uintptr_t)event->data;
+ }
} else {
/* channel already open */
rc = X_LINK_ALREADY_OPEN;
}
break;
+ case XLINK_DATA_READY_CALLBACK_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ opchan->ready_callback = event->data;
+ opchan->ready_calling_pid = event->calling_pid;
+ opchan->callback_origin = event->callback_origin;
+ pr_info("xlink ready callback process registered - %lx chan %d\n",
+ (uintptr_t)event->calling_pid, chan);
+ release_channel(opchan);
+ }
+ break;
+ case XLINK_DATA_CONSUMED_CALLBACK_REQ:
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ } else {
+ opchan->consumed_callback = event->data;
+ opchan->consumed_calling_pid = event->calling_pid;
+ opchan->callback_origin = event->callback_origin;
+ pr_info("xlink consumed callback process registered - %lx chan %d\n",
+ (uintptr_t)event->calling_pid, chan);
+ release_channel(opchan);
+ }
+ break;
case XLINK_CLOSE_CHANNEL_REQ:
if (xmux->channels[link_id][chan].status == CHAN_OPEN) {
opchan = get_channel(link_id, chan);
@@ -709,7 +771,8 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
XLINK_PACKET_ALIGNMENT,
XLINK_NORMAL_MEMORY);
} else {
- pr_err("Fatal error: can't allocate memory in line:%d func:%s\n", __LINE__, __func__);
+ pr_err("Fatal error: can't allocate memory in line:%d func:%s\n",
+ __LINE__, __func__);
}
rc = X_LINK_COMMUNICATION_FAIL;
} else {
@@ -754,6 +817,14 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
xlink_dispatcher_event_add(EVENT_RX, event);
//complete regardless of mode/timeout
complete(&opchan->pkt_available);
+ // run callback
+ if (xmux->channels[link_id][chan].status == CHAN_OPEN &&
+ chan_is_non_blocking_read(opchan) &&
+ opchan->ready_callback) {
+ rc = run_callback(opchan, opchan->ready_callback,
+ opchan->ready_calling_pid);
+ break;
+ }
} else {
// failed to allocate buffer
rc = X_LINK_ERROR;
@@ -813,6 +884,13 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
xlink_dispatcher_event_add(EVENT_RX, event);
//complete regardless of mode/timeout
complete(&opchan->pkt_consumed);
+ // run callback
+ if (xmux->channels[link_id][chan].status == CHAN_OPEN &&
+ chan_is_non_blocking_write(opchan) &&
+ opchan->consumed_callback) {
+ rc = run_callback(opchan, opchan->consumed_callback,
+ opchan->consumed_calling_pid);
+ }
release_channel(opchan);
break;
case XLINK_RELEASE_REQ:
@@ -838,47 +916,47 @@ enum xlink_error xlink_multiplexer_rx(struct xlink_event *event)
rc = multiplexer_open_channel(link_id, chan);
if (rc) {
rc = X_LINK_ERROR;
- } else {
- opchan = get_channel(link_id, chan);
- if (!opchan) {
- rc = X_LINK_COMMUNICATION_FAIL;
- } else {
- xmux->channels[link_id][chan].status = CHAN_OPEN_PEER;
- complete(&opchan->opened);
- passthru_event = xlink_create_event(link_id,
- XLINK_OPEN_CHANNEL_RESP,
- event->handle,
- chan,
- 0,
- opchan->chan->timeout);
- if (!passthru_event) {
- rc = X_LINK_ERROR;
- release_channel(opchan);
- break;
- }
- xlink_dispatcher_event_add(EVENT_RX,
- passthru_event);
- }
+ break;
+ }
+ opchan = get_channel(link_id, chan);
+ if (!opchan) {
+ rc = X_LINK_COMMUNICATION_FAIL;
+ break;
+ }
+ xmux->channels[link_id][chan].status = CHAN_OPEN_PEER;
+ complete(&opchan->opened);
+ passthru_event = xlink_create_event(link_id,
+ XLINK_OPEN_CHANNEL_RESP,
+ event->handle,
+ chan,
+ 0,
+ opchan->chan->timeout);
+ if (!passthru_event) {
+ rc = X_LINK_ERROR;
release_channel(opchan);
+ break;
}
+ xlink_dispatcher_event_add(EVENT_RX,
+ passthru_event);
+ release_channel(opchan);
} else {
/* channel already open */
opchan = get_channel(link_id, chan);
if (!opchan) {
rc = X_LINK_COMMUNICATION_FAIL;
- } else {
- passthru_event = xlink_create_event(link_id,
- XLINK_OPEN_CHANNEL_RESP,
- event->handle,
- chan, 0, 0);
- if (!passthru_event) {
- release_channel(opchan);
- rc = X_LINK_ERROR;
- break;
- }
- xlink_dispatcher_event_add(EVENT_RX,
- passthru_event);
+ break;
+ }
+ passthru_event = xlink_create_event(link_id,
+ XLINK_OPEN_CHANNEL_RESP,
+ event->handle,
+ chan, 0, 0);
+ if (!passthru_event) {
+ release_channel(opchan);
+ rc = X_LINK_ERROR;
+ break;
}
+ xlink_dispatcher_event_add(EVENT_RX,
+ passthru_event);
release_channel(opchan);
}
rc = xlink_passthrough(event);
@@ -930,7 +1008,7 @@ enum xlink_error xlink_passthrough(struct xlink_event *event)
#ifdef CONFIG_XLINK_LOCAL_HOST
struct xlink_ipc_context ipc = {0};
phys_addr_t physaddr = 0;
- dma_addr_t vpuaddr = 0;
+ static dma_addr_t vpuaddr;
u32 timeout = 0;
u32 link_id;
u16 chan;
diff --git a/drivers/misc/xlink-core/xlink-platform.c b/drivers/misc/xlink-core/xlink-platform.c
index 56eb8da28a5f..b0076cb3671d 100644
--- a/drivers/misc/xlink-core/xlink-platform.c
+++ b/drivers/misc/xlink-core/xlink-platform.c
@@ -56,6 +56,11 @@ static inline int xlink_ipc_close_channel(u32 sw_device_id,
u32 channel)
{ return -1; }
+static inline int xlink_ipc_register_for_events(u32 sw_device_id,
+ int (*callback)(u32 sw_device_id, u32 event))
+{ return -1; }
+static inline int xlink_ipc_unregister_for_events(u32 sw_device_id)
+{ return -1; }
#endif /* CONFIG_XLINK_LOCAL_HOST */
/*
@@ -95,6 +100,13 @@ static int (*open_chan_fcts[NMB_OF_INTERFACES])(u32, u32) = {
static int (*close_chan_fcts[NMB_OF_INTERFACES])(u32, u32) = {
xlink_ipc_close_channel, NULL, NULL, NULL};
+static int (*register_for_events_fcts[NMB_OF_INTERFACES])(u32,
+ int (*callback)(u32 sw_device_id, u32 event)) = {
+ xlink_ipc_register_for_events,
+ xlink_pcie_register_device_event,
+ NULL, NULL};
+static int (*unregister_for_events_fcts[NMB_OF_INTERFACES])(u32) = {
+ xlink_ipc_unregister_for_events, xlink_pcie_unregister_device_event, NULL, NULL};
/*
* xlink low-level driver interface
@@ -207,6 +219,21 @@ int xlink_platform_close_channel(u32 interface, u32 sw_device_id,
return close_chan_fcts[interface](sw_device_id, channel);
}
+int xlink_platform_register_for_events(u32 interface, u32 sw_device_id,
+ xlink_device_event_cb event_notif_fn)
+{
+ if (interface >= NMB_OF_INTERFACES || !register_for_events_fcts[interface])
+ return -1;
+ return register_for_events_fcts[interface](sw_device_id, event_notif_fn);
+}
+
+int xlink_platform_unregister_for_events(u32 interface, u32 sw_device_id)
+{
+ if (interface >= NMB_OF_INTERFACES || !unregister_for_events_fcts[interface])
+ return -1;
+ return unregister_for_events_fcts[interface](sw_device_id);
+}
+
void *xlink_platform_allocate(struct device *dev, dma_addr_t *handle,
u32 size, u32 alignment,
enum xlink_memory_region region)
diff --git a/include/linux/xlink.h b/include/linux/xlink.h
index b00dbc719530..ac196ff85469 100644
--- a/include/linux/xlink.h
+++ b/include/linux/xlink.h
@@ -70,6 +70,12 @@ enum xlink_error xlink_open_channel(struct xlink_handle *handle,
u16 chan, enum xlink_opmode mode,
u32 data_size, u32 timeout);
+enum xlink_error xlink_data_available_event(struct xlink_handle *handle,
+ u16 chan,
+ xlink_event data_available_event);
+enum xlink_error xlink_data_consumed_event(struct xlink_handle *handle,
+ u16 chan,
+ xlink_event data_consumed_event);
enum xlink_error xlink_close_channel(struct xlink_handle *handle, u16 chan);
enum xlink_error xlink_write_data(struct xlink_handle *handle,
@@ -113,9 +119,14 @@ enum xlink_error xlink_set_device_mode(struct xlink_handle *handle,
enum xlink_error xlink_get_device_mode(struct xlink_handle *handle,
enum xlink_device_power_mode *power_mode);
-enum xlink_error xlink_start_vpu(char *filename); /* depreciated */
+enum xlink_error xlink_register_device_event(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn);
+enum xlink_error xlink_unregister_device_event(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events);
+enum xlink_error xlink_start_vpu(char *filename); /* deprecated */
-enum xlink_error xlink_stop_vpu(void); /* depreciated */
+enum xlink_error xlink_stop_vpu(void); /* deprecated */
/* API functions to be implemented
*
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 22/22] xlink-core: factorize xlink_ioctl function by creating sub-functions for each ioctl command
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (20 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 21/22] xlink-core: add async channel and events mgross
@ 2020-11-30 23:07 ` mgross
2020-12-01 10:14 ` [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 Greg KH
22 siblings, 0 replies; 34+ messages in thread
From: mgross @ 2020-11-30 23:07 UTC (permalink / raw)
To: linux-kernel
Cc: markgross, mgross, adam.r.gretzinger, Seamus Kelly,
Arnd Bergmann, Greg Kroah-Hartman
From: Seamus Kelly <seamus.kelly@intel.com>
Refactor the too large IOCTL function to call helper functions.
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Mark Gross <mgross@linux.intel.com>
Signed-off-by: Seamus Kelly <seamus.kelly@intel.com>
---
drivers/misc/xlink-core/Makefile | 2 +-
drivers/misc/xlink-core/xlink-core.c | 625 ++++++--------------------
drivers/misc/xlink-core/xlink-core.h | 24 +
drivers/misc/xlink-core/xlink-defs.h | 2 +-
drivers/misc/xlink-core/xlink-ioctl.c | 584 ++++++++++++++++++++++++
drivers/misc/xlink-core/xlink-ioctl.h | 36 ++
6 files changed, 773 insertions(+), 500 deletions(-)
create mode 100644 drivers/misc/xlink-core/xlink-core.h
create mode 100644 drivers/misc/xlink-core/xlink-ioctl.c
create mode 100644 drivers/misc/xlink-core/xlink-ioctl.h
diff --git a/drivers/misc/xlink-core/Makefile b/drivers/misc/xlink-core/Makefile
index 6e604c0b8962..2f64703301d6 100644
--- a/drivers/misc/xlink-core/Makefile
+++ b/drivers/misc/xlink-core/Makefile
@@ -2,4 +2,4 @@
# Makefile for KeemBay xlink Linux driver
#
obj-$(CONFIG_XLINK_CORE) += xlink.o
-xlink-objs += xlink-core.o xlink-multiplexer.o xlink-dispatcher.o xlink-platform.o
+xlink-objs += xlink-core.o xlink-multiplexer.o xlink-dispatcher.o xlink-platform.o xlink-ioctl.o
diff --git a/drivers/misc/xlink-core/xlink-core.c b/drivers/misc/xlink-core/xlink-core.c
index 63f65def8aa9..ed2b37085851 100644
--- a/drivers/misc/xlink-core/xlink-core.c
+++ b/drivers/misc/xlink-core/xlink-core.c
@@ -20,9 +20,11 @@
#endif
#include "xlink-defs.h"
+#include "xlink-core.h"
#include "xlink-dispatcher.h"
#include "xlink-multiplexer.h"
#include "xlink-platform.h"
+#include "xlink-ioctl.h"
// xlink version number
#define XLINK_VERSION_MAJOR 0
@@ -52,25 +54,7 @@ static struct class *dev_class;
static struct cdev xlink_cdev;
static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
-static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
- u16 chan, u8 const *pmessage,
- u32 size);
-static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
- u16 chan, u8 const *message,
- u32 size);
-static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
- u16 chan, u8 const *message,
- u32 size, u32 user_flag);
-static enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
- u32 *event_list,
- u32 num_events,
- xlink_device_event_cb event_notif_fn);
-static enum xlink_error do_xlink_register_device_event(struct xlink_handle *handle,
- u32 *event_list,
- u32 num_events,
- xlink_device_event_cb event_notif_fn,
- u32 user_flag);
static struct mutex dev_event_lock;
static const struct file_operations fops = {
@@ -105,9 +89,6 @@ struct event_info {
xlink_device_event_cb event_notif_fn;
};
-static u8 volbuf[XLINK_MAX_BUF_SIZE]; // buffer for volatile transactions
-#define NUM_REG_EVENTS 4
-
// sysfs attribute functions
static ssize_t event0_show(struct device *dev, struct device_attribute *attr, char *buf)
@@ -342,427 +323,84 @@ static int kmb_xlink_remove(struct platform_device *pdev)
* IOCTL function for User Space access to xlink kernel functions
*
*/
+int ioctl_connect(unsigned long arg);
static long xlink_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
- struct xlink_handle devh = {0};
- struct xlinkopenchannel op = {0};
- struct xlinkwritedata wr = {0};
- struct xlinkreaddata rd = {0};
- struct xlinkreadtobuffer rdtobuf = {0};
- struct xlinkconnect con = {0};
- struct xlinkrelease rel = {0};
- struct xlinkstartvpu startvpu = {0};
- struct xlinkcallback cb = {0};
- struct xlinkgetdevicename devn = {0};
- struct xlinkgetdevicelist devl = {0};
- struct xlinkgetdevicestatus devs = {0};
- struct xlinkbootdevice boot = {0};
- struct xlinkresetdevice res = {0};
- struct xlinkdevmode devm = {0};
- struct xlinkregdevevent regdevevent = {0};
- u32 sw_device_id_list[XLINK_MAX_DEVICE_LIST_SIZE];
- char name[XLINK_MAX_DEVICE_NAME_SIZE];
- int interface = NULL_INTERFACE;
- u32 device_status = 0;
- u32 num_devices = 0;
- u32 device_mode = 0;
- u32 num_events = 0;
- char filename[64];
- u32 *ev_list;
- u8 reladdr;
- u8 *rdaddr;
- u32 size;
int rc;
switch (cmd) {
case XL_CONNECT:
- if (copy_from_user(&con, (void __user *)arg,
- sizeof(struct xlinkconnect)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)con.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_connect(&devh);
- if (rc == X_LINK_SUCCESS) {
- if (copy_to_user((void __user *)con.handle,
- &devh, sizeof(struct xlink_handle)))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)con.return_code, (void *)&rc,
- sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_connect(arg);
break;
case XL_OPEN_CHANNEL:
- if (copy_from_user(&op, (void __user *)arg,
- sizeof(struct xlinkopenchannel)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)op.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_open_channel(&devh, op.chan, op.mode, op.data_size,
- op.timeout);
- if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_open_channel(arg);
break;
case XL_DATA_READY_CALLBACK:
- if (copy_from_user(&cb, (void __user *)arg,
- sizeof(struct xlinkcallback)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)cb.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
- rc = xlink_data_available_event(&devh, cb.chan, cb.callback);
- if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_data_ready_callback(arg);
break;
case XL_DATA_CONSUMED_CALLBACK:
- if (copy_from_user(&cb, (void __user *)arg,
- sizeof(struct xlinkcallback)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)cb.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
- rc = xlink_data_consumed_event(&devh, cb.chan, cb.callback);
- if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_data_consumed_callback(arg);
break;
case XL_READ_DATA:
- if (copy_from_user(&rd, (void __user *)arg,
- sizeof(struct xlinkreaddata)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)rd.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_read_data(&devh, rd.chan, &rdaddr, &size);
- if (!rc) {
- interface = get_interface_from_sw_device_id(devh.sw_device_id);
- if (interface == IPC_INTERFACE) {
- if (copy_to_user((void __user *)rd.pmessage, (void *)&rdaddr,
- sizeof(u32)))
- return -EFAULT;
- } else {
- if (copy_to_user((void __user *)rd.pmessage, (void *)rdaddr,
- size))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)rd.size, (void *)&size, sizeof(size)))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)rd.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_read_data(arg);
break;
case XL_READ_TO_BUFFER:
- if (copy_from_user(&rdtobuf, (void __user *)arg,
- sizeof(struct xlinkreadtobuffer)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)rdtobuf.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_read_data_to_buffer(&devh, rdtobuf.chan,
- (u8 *)volbuf, &size);
- if (!rc) {
- if (copy_to_user((void __user *)rdtobuf.pmessage, (void *)volbuf,
- size))
- return -EFAULT;
- if (copy_to_user((void __user *)rdtobuf.size, (void *)&size,
- sizeof(size)))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)rdtobuf.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_read_to_buffer(arg);
break;
case XL_WRITE_DATA:
- if (copy_from_user(&wr, (void __user *)arg,
- sizeof(struct xlinkwritedata)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)wr.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (wr.size <= XLINK_MAX_DATA_SIZE) {
- rc = xlink_write_data_user(&devh, wr.chan, wr.pmessage,
- wr.size);
- if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
- sizeof(rc)))
- return -EFAULT;
- } else {
- return -EFAULT;
- }
+ rc = ioctl_write_data(arg);
break;
case XL_WRITE_VOLATILE:
- if (copy_from_user(&wr, (void __user *)arg,
- sizeof(struct xlinkwritedata)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)wr.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (wr.size <= XLINK_MAX_BUF_SIZE) {
- if (copy_from_user(volbuf, (void __user *)wr.pmessage,
- wr.size))
- return -EFAULT;
- rc = xlink_write_volatile_user(&devh, wr.chan, volbuf,
- wr.size);
- if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
- sizeof(rc)))
- return -EFAULT;
- } else {
- return -EFAULT;
- }
+ rc = ioctl_write_volatile_data(arg);
break;
case XL_WRITE_CONTROL_DATA:
- if (copy_from_user(&wr, (void __user *)arg,
- sizeof(struct xlinkwritedata)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)wr.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (wr.size <= XLINK_MAX_CONTROL_DATA_SIZE) {
- if (copy_from_user(volbuf, (void __user *)wr.pmessage,
- wr.size))
- return -EFAULT;
- rc = xlink_write_control_data(&devh, wr.chan, volbuf,
- wr.size);
- if (copy_to_user((void __user *)wr.return_code,
- (void *)&rc, sizeof(rc)))
- return -EFAULT;
- } else {
- return -EFAULT;
- }
+ rc = ioctl_write_control_data(arg);
break;
case XL_RELEASE_DATA:
- if (copy_from_user(&rel, (void __user *)arg,
- sizeof(struct xlinkrelease)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)rel.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (rel.addr) {
- if (get_user(reladdr, (u32 __user *const)rel.addr))
- return -EFAULT;
- rc = xlink_release_data(&devh, rel.chan,
- (u8 *)&reladdr);
- } else {
- rc = xlink_release_data(&devh, rel.chan, NULL);
- }
- if (copy_to_user((void __user *)rel.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_release_data(arg);
break;
case XL_CLOSE_CHANNEL:
- if (copy_from_user(&op, (void __user *)arg,
- sizeof(struct xlinkopenchannel)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)op.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_close_channel(&devh, op.chan);
- if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_close_channel(arg);
break;
case XL_START_VPU:
- if (copy_from_user(&startvpu, (void __user *)arg,
- sizeof(struct xlinkstartvpu)))
- return -EFAULT;
- if (startvpu.namesize > sizeof(filename))
- return -EINVAL;
- memset(filename, 0, sizeof(filename));
- if (copy_from_user(filename, (void __user *)startvpu.filename,
- startvpu.namesize))
- return -EFAULT;
- rc = xlink_start_vpu(filename);
- if (copy_to_user((void __user *)startvpu.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_start_vpu(arg);
break;
case XL_STOP_VPU:
- rc = xlink_stop_vpu();
+ rc = ioctl_stop_vpu();
break;
case XL_RESET_VPU:
- rc = xlink_stop_vpu();
+ rc = ioctl_stop_vpu();
break;
case XL_DISCONNECT:
- if (copy_from_user(&con, (void __user *)arg,
- sizeof(struct xlinkconnect)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)con.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_disconnect(&devh);
- if (copy_to_user((void __user *)con.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_disconnect(arg);
break;
case XL_GET_DEVICE_NAME:
- if (copy_from_user(&devn, (void __user *)arg,
- sizeof(struct xlinkgetdevicename)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)devn.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (devn.name_size <= XLINK_MAX_DEVICE_NAME_SIZE) {
- rc = xlink_get_device_name(&devh, name, devn.name_size);
- if (!rc) {
- if (copy_to_user((void __user *)devn.name, (void *)name,
- devn.name_size))
- return -EFAULT;
- }
- } else {
- rc = X_LINK_ERROR;
- }
- if (copy_to_user((void __user *)devn.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_get_device_name(arg);
break;
case XL_GET_DEVICE_LIST:
- if (copy_from_user(&devl, (void __user *)arg,
- sizeof(struct xlinkgetdevicelist)))
- return -EFAULT;
- rc = xlink_get_device_list(sw_device_id_list, &num_devices);
- if (!rc && num_devices <= XLINK_MAX_DEVICE_LIST_SIZE) {
- /* TODO: this next copy is dangerous! we have no idea
- * how large the devl.sw_device_id_list buffer is
- * provided by the user. if num_devices is too large,
- * the copy will overflow the buffer.
- */
- if (copy_to_user((void __user *)devl.sw_device_id_list,
- (void *)sw_device_id_list,
- (sizeof(*sw_device_id_list)
- * num_devices)))
- return -EFAULT;
- if (copy_to_user((void __user *)devl.num_devices, (void *)&num_devices,
- (sizeof(num_devices))))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)devl.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_get_device_list(arg);
break;
case XL_GET_DEVICE_STATUS:
- if (copy_from_user(&devs, (void __user *)arg,
- sizeof(struct xlinkgetdevicestatus)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)devs.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_get_device_status(&devh, &device_status);
- if (!rc) {
- if (copy_to_user((void __user *)devs.device_status,
- (void *)&device_status,
- sizeof(device_status)))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)devs.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_get_device_status(arg);
break;
case XL_BOOT_DEVICE:
- if (copy_from_user(&boot, (void __user *)arg,
- sizeof(struct xlinkbootdevice)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)boot.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (boot.binary_name_size > sizeof(filename))
- return -EINVAL;
- memset(filename, 0, sizeof(filename));
- if (copy_from_user(filename, (void __user *)boot.binary_name,
- boot.binary_name_size))
- return -EFAULT;
- rc = xlink_boot_device(&devh, filename);
- if (copy_to_user((void __user *)boot.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_boot_device(arg);
break;
case XL_RESET_DEVICE:
- if (copy_from_user(&res, (void __user *)arg,
- sizeof(struct xlinkresetdevice)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)res.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_reset_device(&devh);
- if (copy_to_user((void __user *)res.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_reset_device(arg);
break;
case XL_GET_DEVICE_MODE:
- if (copy_from_user(&devm, (void __user *)arg,
- sizeof(struct xlinkdevmode)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)devm.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- rc = xlink_get_device_mode(&devh, &device_mode);
- if (!rc) {
- if (copy_to_user((void __user *)devm.device_mode, (void *)&device_mode,
- sizeof(device_mode)))
- return -EFAULT;
- }
- if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_get_device_mode(arg);
break;
case XL_SET_DEVICE_MODE:
- if (copy_from_user(&devm, (void __user *)arg,
- sizeof(struct xlinkdevmode)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)devm.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- if (copy_from_user(&device_mode, (void __user *)devm.device_mode,
- sizeof(device_mode)))
- return -EFAULT;
- rc = xlink_set_device_mode(&devh, device_mode);
- if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_set_device_mode(arg);
break;
case XL_REGISTER_DEV_EVENT:
- if (copy_from_user(®devevent, (void __user *)arg,
- sizeof(struct xlinkregdevevent)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)regdevevent.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- num_events = regdevevent.num_events;
- if (num_events > 0 && num_events <= NUM_REG_EVENTS) {
- ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
- if (ev_list) {
- if (copy_from_user(ev_list,
- (void __user *)regdevevent.event_list,
- (num_events * sizeof(u32)))) {
- kfree(ev_list);
- return -EFAULT;
- }
- rc = xlink_register_device_event_user(&devh,
- ev_list,
- num_events,
- NULL);
- kfree(ev_list);
- } else {
- rc = X_LINK_ERROR;
- }
- } else {
- rc = X_LINK_ERROR;
- }
- if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_register_device_event(arg);
break;
case XL_UNREGISTER_DEV_EVENT:
- if (copy_from_user(®devevent, (void __user *)arg,
- sizeof(struct xlinkregdevevent)))
- return -EFAULT;
- if (copy_from_user(&devh, (void __user *)regdevevent.handle,
- sizeof(struct xlink_handle)))
- return -EFAULT;
- num_events = regdevevent.num_events;
- if (num_events <= NUM_REG_EVENTS) {
- ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
- if (copy_from_user(ev_list,
- (void __user *)regdevevent.event_list,
- (num_events * sizeof(u32)))) {
- kfree(ev_list);
- return -EFAULT;
- }
- rc = xlink_unregister_device_event(&devh, ev_list, num_events);
- kfree(ev_list);
- } else {
- rc = X_LINK_ERROR;
- }
- if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
- return -EFAULT;
+ rc = ioctl_unregister_device_event(arg);
break;
}
if (rc)
@@ -997,13 +635,16 @@ enum xlink_error xlink_close_channel(struct xlink_handle *handle,
}
EXPORT_SYMBOL(xlink_close_channel);
-enum xlink_error xlink_write_data(struct xlink_handle *handle,
- u16 chan, u8 const *pmessage, u32 size)
+static enum xlink_error do_xlink_write_data(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage,
+ u32 size, u32 user_flag)
{
struct xlink_event *event;
struct xlink_link *link;
enum xlink_error rc;
int event_queued = 0;
+ dma_addr_t paddr = 0;
+ u32 addr;
if (!xlink || !handle)
return X_LINK_ERROR;
@@ -1019,88 +660,75 @@ enum xlink_error xlink_write_data(struct xlink_handle *handle,
chan, size, 0);
if (!event)
return X_LINK_ERROR;
+ event->user_data = user_flag;
if (chan < XLINK_IPC_MAX_CHANNELS &&
event->interface == IPC_INTERFACE) {
/* only passing message address across IPC interface */
- event->data = &pmessage;
+ if (user_flag) {
+ if (get_user(addr, (u32 __user *)pmessage)) {
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ event->data = &addr;
+ } else {
+ event->data = &pmessage;
+ }
rc = xlink_multiplexer_tx(event, &event_queued);
xlink_destroy_event(event);
} else {
- event->data = (u8 *)pmessage;
- event->paddr = 0;
+ if (user_flag) {
+ event->data = xlink_platform_allocate(&xlink->pdev->dev, &paddr,
+ size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ if (!event->data) {
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ if (copy_from_user(event->data, (void __user *)pmessage, size)) {
+ xlink_platform_deallocate(&xlink->pdev->dev,
+ event->data, paddr, size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ xlink_destroy_event(event);
+ return X_LINK_ERROR;
+ }
+ event->paddr = paddr;
+ } else {
+ event->data = (u8 *)pmessage;
+ event->paddr = 0;
+ }
rc = xlink_multiplexer_tx(event, &event_queued);
- if (!event_queued)
+ if (!event_queued) {
+ if (user_flag) {
+ xlink_platform_deallocate(&xlink->pdev->dev,
+ event->data, paddr, size,
+ XLINK_PACKET_ALIGNMENT,
+ XLINK_NORMAL_MEMORY);
+ }
xlink_destroy_event(event);
+ }
}
return rc;
}
-EXPORT_SYMBOL(xlink_write_data);
-static enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
- u16 chan, u8 const *pmessage,
- u32 size)
+enum xlink_error xlink_write_data(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage, u32 size)
{
- struct xlink_event *event;
- struct xlink_link *link;
- enum xlink_error rc;
- int event_queued = 0;
- dma_addr_t paddr = 0;
- u32 addr;
-
- if (!xlink || !handle)
- return X_LINK_ERROR;
-
- if (size > XLINK_MAX_DATA_SIZE)
- return X_LINK_ERROR;
+ enum xlink_error rc = 0;
- link = get_link_by_sw_device_id(handle->sw_device_id);
- if (!link)
- return X_LINK_ERROR;
+ rc = do_xlink_write_data(handle, chan, pmessage, size, 0);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_write_data);
- event = xlink_create_event(link->id, XLINK_WRITE_REQ, &link->handle,
- chan, size, 0);
- if (!event)
- return X_LINK_ERROR;
- event->user_data = 1;
+enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage, u32 size)
+{
+ enum xlink_error rc = 0;
- if (chan < XLINK_IPC_MAX_CHANNELS &&
- event->interface == IPC_INTERFACE) {
- /* only passing message address across IPC interface */
- if (get_user(addr, (u32 __user *)pmessage)) {
- xlink_destroy_event(event);
- return X_LINK_ERROR;
- }
- event->data = &addr;
- rc = xlink_multiplexer_tx(event, &event_queued);
- xlink_destroy_event(event);
- } else {
- event->data = xlink_platform_allocate(&xlink->pdev->dev, &paddr,
- size,
- XLINK_PACKET_ALIGNMENT,
- XLINK_NORMAL_MEMORY);
- if (!event->data) {
- xlink_destroy_event(event);
- return X_LINK_ERROR;
- }
- if (copy_from_user(event->data, (void __user *)pmessage, size)) {
- xlink_platform_deallocate(&xlink->pdev->dev,
- event->data, paddr, size,
- XLINK_PACKET_ALIGNMENT,
- XLINK_NORMAL_MEMORY);
- xlink_destroy_event(event);
- return X_LINK_ERROR;
- }
- event->paddr = paddr;
- rc = xlink_multiplexer_tx(event, &event_queued);
- if (!event_queued) {
- xlink_platform_deallocate(&xlink->pdev->dev,
- event->data, paddr, size,
- XLINK_PACKET_ALIGNMENT,
- XLINK_NORMAL_MEMORY);
- xlink_destroy_event(event);
- }
- }
+ rc = do_xlink_write_data(handle, chan, pmessage, size, 1);
return rc;
}
@@ -1131,25 +759,6 @@ enum xlink_error xlink_write_control_data(struct xlink_handle *handle,
return rc;
}
EXPORT_SYMBOL(xlink_write_control_data);
-static enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
- u16 chan, u8 const *message,
- u32 size)
-{
- enum xlink_error rc = 0;
-
- rc = do_xlink_write_volatile(handle, chan, message, size, 1);
- return rc;
-}
-
-enum xlink_error xlink_write_volatile(struct xlink_handle *handle,
- u16 chan, u8 const *message, u32 size)
-{
- enum xlink_error rc = 0;
-
- rc = do_xlink_write_volatile(handle, chan, message, size, 0);
- return rc;
-}
-EXPORT_SYMBOL(xlink_write_volatile);
static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
u16 chan, u8 const *message,
@@ -1196,6 +805,26 @@ static enum xlink_error do_xlink_write_volatile(struct xlink_handle *handle,
return rc;
}
+enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size)
+{
+ enum xlink_error rc = 0;
+
+ rc = do_xlink_write_volatile(handle, chan, message, size, 1);
+ return rc;
+}
+
+enum xlink_error xlink_write_volatile(struct xlink_handle *handle,
+ u16 chan, u8 const *message, u32 size)
+{
+ enum xlink_error rc = 0;
+
+ rc = do_xlink_write_volatile(handle, chan, message, size, 0);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_write_volatile);
+
enum xlink_error xlink_read_data(struct xlink_handle *handle,
u16 chan, u8 **pmessage, u32 *size)
{
@@ -1531,29 +1160,6 @@ static bool event_registered(u32 sw_dev_id, u32 event)
return false;
}
-static enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
- u32 *event_list, u32 num_events,
- xlink_device_event_cb event_notif_fn)
-{
- enum xlink_error rc;
-
- rc = do_xlink_register_device_event(handle, event_list, num_events,
- event_notif_fn, 1);
- return rc;
-}
-
-enum xlink_error xlink_register_device_event(struct xlink_handle *handle,
- u32 *event_list, u32 num_events,
- xlink_device_event_cb event_notif_fn)
-{
- enum xlink_error rc;
-
- rc = do_xlink_register_device_event(handle, event_list, num_events,
- event_notif_fn, 0);
- return rc;
-}
-EXPORT_SYMBOL(xlink_register_device_event);
-
static enum xlink_error do_xlink_register_device_event(struct xlink_handle *handle,
u32 *event_list,
u32 num_events,
@@ -1599,6 +1205,29 @@ static enum xlink_error do_xlink_register_device_event(struct xlink_handle *hand
return X_LINK_SUCCESS;
}
+enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn)
+{
+ enum xlink_error rc;
+
+ rc = do_xlink_register_device_event(handle, event_list, num_events,
+ event_notif_fn, 1);
+ return rc;
+}
+
+enum xlink_error xlink_register_device_event(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn)
+{
+ enum xlink_error rc;
+
+ rc = do_xlink_register_device_event(handle, event_list, num_events,
+ event_notif_fn, 0);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_register_device_event);
+
enum xlink_error xlink_unregister_device_event(struct xlink_handle *handle,
u32 *event_list,
u32 num_events)
diff --git a/drivers/misc/xlink-core/xlink-core.h b/drivers/misc/xlink-core/xlink-core.h
new file mode 100644
index 000000000000..c3d100a75c44
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-core.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink core header file.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+
+#ifndef XLINK_CORE_H_
+#define XLINK_CORE_H_
+#include "xlink-defs.h"
+
+#define NUM_REG_EVENTS 4
+
+enum xlink_error xlink_write_data_user(struct xlink_handle *handle,
+ u16 chan, u8 const *pmessage,
+ u32 size);
+enum xlink_error xlink_register_device_event_user(struct xlink_handle *handle,
+ u32 *event_list, u32 num_events,
+ xlink_device_event_cb event_notif_fn);
+enum xlink_error xlink_write_volatile_user(struct xlink_handle *handle,
+ u16 chan, u8 const *message,
+ u32 size);
+#endif /* XLINK_CORE_H_ */
diff --git a/drivers/misc/xlink-core/xlink-defs.h b/drivers/misc/xlink-core/xlink-defs.h
index df686e5185c5..81aa3bfffcd3 100644
--- a/drivers/misc/xlink-core/xlink-defs.h
+++ b/drivers/misc/xlink-core/xlink-defs.h
@@ -35,7 +35,7 @@
#define CONTROL_CHANNEL_TIMEOUT_MS 0U // wait indefinitely
#define SIGXLNK 44 // signal XLink uses for callback signalling
-#define UNUSED(x) (void)(x)
+#define UNUSED(x) ((void)(x))
// the end of the IPC channel range (starting at zero)
#define XLINK_IPC_MAX_CHANNELS 1024
diff --git a/drivers/misc/xlink-core/xlink-ioctl.c b/drivers/misc/xlink-core/xlink-ioctl.c
new file mode 100644
index 000000000000..9c1a8c896945
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-ioctl.c
@@ -0,0 +1,584 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Core Driver.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/fs.h>
+#include <linux/cdev.h>
+#include <linux/platform_device.h>
+#include <linux/mod_devicetable.h>
+#include <linux/uaccess.h>
+#include <linux/slab.h>
+#include <linux/kref.h>
+
+#include "xlink-defs.h"
+#include "xlink-core.h"
+#include "xlink-ioctl.h"
+
+#define CHANNEL_SET_USER_BIT(chan) ((chan) |= (1 << 15))
+
+int ioctl_connect(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkconnect con = {0};
+ int rc = 0;
+
+ if (copy_from_user(&con, (void __user *)arg,
+ sizeof(struct xlinkconnect)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)con.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_connect(&devh);
+ if (rc == X_LINK_SUCCESS) {
+ if (copy_to_user((void __user *)con.handle,
+ &devh, sizeof(struct xlink_handle)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)con.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_open_channel(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkopenchannel op = {0};
+ int rc = 0;
+
+ if (copy_from_user(&op, (void __user *)arg,
+ sizeof(struct xlinkopenchannel)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)op.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_open_channel(&devh, op.chan, op.mode, op.data_size,
+ op.timeout);
+ if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_read_data(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkreaddata rd = {0};
+ int rc = 0;
+ u8 *rdaddr;
+ u32 size;
+ int interface;
+
+ if (copy_from_user(&rd, (void __user *)arg,
+ sizeof(struct xlinkreaddata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rd.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_read_data(&devh, rd.chan, &rdaddr, &size);
+ if (!rc) {
+ interface = get_interface_from_sw_device_id(devh.sw_device_id);
+ if (interface == IPC_INTERFACE) {
+ if (copy_to_user((void __user *)rd.pmessage, (void *)&rdaddr,
+ sizeof(u32)))
+ return -EFAULT;
+ } else {
+ if (copy_to_user((void __user *)rd.pmessage, (void *)rdaddr,
+ size))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rd.size, (void *)&size, sizeof(size)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rd.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_read_to_buffer(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkreadtobuffer rdtobuf = {0};
+ int rc = 0;
+ u32 size;
+ u8 volbuf[XLINK_MAX_BUF_SIZE]; // buffer for volatile transactions
+
+ if (copy_from_user(&rdtobuf, (void __user *)arg,
+ sizeof(struct xlinkreadtobuffer)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rdtobuf.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_read_data_to_buffer(&devh, rdtobuf.chan,
+ (u8 *)volbuf, &size);
+ if (!rc) {
+ if (copy_to_user((void __user *)rdtobuf.pmessage, (void *)volbuf,
+ size))
+ return -EFAULT;
+ if (copy_to_user((void __user *)rdtobuf.size, (void *)&size,
+ sizeof(size)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)rdtobuf.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_write_data(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkwritedata wr = {0};
+ int rc = 0;
+
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_DATA_SIZE) {
+ rc = xlink_write_data_user(&devh, wr.chan, wr.pmessage,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ return rc;
+}
+
+int ioctl_write_control_data(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkwritedata wr = {0};
+ u8 volbuf[XLINK_MAX_BUF_SIZE];
+ int rc = 0;
+
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_CONTROL_DATA_SIZE) {
+ if (copy_from_user(volbuf, (void __user *)wr.pmessage,
+ wr.size))
+ return -EFAULT;
+ rc = xlink_write_control_data(&devh, wr.chan, volbuf,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code,
+ (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ return rc;
+}
+
+int ioctl_write_volatile_data(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkwritedata wr = {0};
+ int rc = 0;
+ u8 volbuf[XLINK_MAX_BUF_SIZE]; // buffer for volatile transactions
+
+ if (copy_from_user(&wr, (void __user *)arg,
+ sizeof(struct xlinkwritedata)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)wr.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (wr.size <= XLINK_MAX_BUF_SIZE) {
+ if (copy_from_user(volbuf, (void __user *)wr.pmessage,
+ wr.size))
+ return -EFAULT;
+ rc = xlink_write_volatile_user(&devh, wr.chan, volbuf,
+ wr.size);
+ if (copy_to_user((void __user *)wr.return_code, (void *)&rc,
+ sizeof(rc)))
+ return -EFAULT;
+ } else {
+ return -EFAULT;
+ }
+ return rc;
+}
+
+int ioctl_release_data(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkrelease rel = {0};
+ int rc = 0;
+ u8 reladdr;
+
+ if (copy_from_user(&rel, (void __user *)arg,
+ sizeof(struct xlinkrelease)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)rel.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (rel.addr) {
+ if (get_user(reladdr, (u32 __user *const)rel.addr))
+ return -EFAULT;
+ rc = xlink_release_data(&devh, rel.chan,
+ (u8 *)&reladdr);
+ } else {
+ rc = xlink_release_data(&devh, rel.chan, NULL);
+ }
+ if (copy_to_user((void __user *)rel.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_close_channel(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkopenchannel op = {0};
+ int rc = 0;
+
+ if (copy_from_user(&op, (void __user *)arg,
+ sizeof(struct xlinkopenchannel)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)op.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_close_channel(&devh, op.chan);
+ if (copy_to_user((void __user *)op.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_start_vpu(unsigned long arg)
+{
+ struct xlinkstartvpu startvpu = {0};
+ char filename[64];
+ int rc = 0;
+
+ if (copy_from_user(&startvpu, (void __user *)arg,
+ sizeof(struct xlinkstartvpu)))
+ return -EFAULT;
+ if (startvpu.namesize > sizeof(filename))
+ return -EINVAL;
+ memset(filename, 0, sizeof(filename));
+ if (copy_from_user(filename, (void __user *)startvpu.filename,
+ startvpu.namesize))
+ return -EFAULT;
+ rc = xlink_start_vpu(filename);
+ if (copy_to_user((void __user *)startvpu.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_stop_vpu(void)
+{
+ int rc = 0;
+
+ rc = xlink_stop_vpu();
+ return rc;
+}
+
+int ioctl_disconnect(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkconnect con = {0};
+ int rc = 0;
+
+ if (copy_from_user(&con, (void __user *)arg,
+ sizeof(struct xlinkconnect)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)con.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_disconnect(&devh);
+ if (copy_to_user((void __user *)con.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_get_device_name(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkgetdevicename devn = {0};
+ char name[XLINK_MAX_DEVICE_NAME_SIZE];
+ int rc = 0;
+
+ if (copy_from_user(&devn, (void __user *)arg,
+ sizeof(struct xlinkgetdevicename)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devn.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (devn.name_size <= XLINK_MAX_DEVICE_NAME_SIZE) {
+ rc = xlink_get_device_name(&devh, name, devn.name_size);
+ if (!rc) {
+ if (copy_to_user((void __user *)devn.name, (void *)name,
+ devn.name_size))
+ return -EFAULT;
+ }
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)devn.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_get_device_list(unsigned long arg)
+{
+ struct xlinkgetdevicelist devl = {0};
+ u32 sw_device_id_list[XLINK_MAX_DEVICE_LIST_SIZE];
+ u32 num_devices = 0;
+ int rc = 0;
+
+ if (copy_from_user(&devl, (void __user *)arg,
+ sizeof(struct xlinkgetdevicelist)))
+ return -EFAULT;
+ rc = xlink_get_device_list(sw_device_id_list, &num_devices);
+ if (!rc && num_devices <= XLINK_MAX_DEVICE_LIST_SIZE) {
+ /* TODO: this next copy is dangerous! we have no idea
+ * how large the devl.sw_device_id_list buffer is
+ * provided by the user. if num_devices is too large,
+ * the copy will overflow the buffer.
+ */
+ if (copy_to_user((void __user *)devl.sw_device_id_list,
+ (void *)sw_device_id_list,
+ (sizeof(*sw_device_id_list)
+ * num_devices)))
+ return -EFAULT;
+ if (copy_to_user((void __user *)devl.num_devices, (void *)&num_devices,
+ (sizeof(num_devices))))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devl.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_get_device_status(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkgetdevicestatus devs = {0};
+ u32 device_status = 0;
+ int rc = 0;
+
+ if (copy_from_user(&devs, (void __user *)arg,
+ sizeof(struct xlinkgetdevicestatus)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devs.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_get_device_status(&devh, &device_status);
+ if (!rc) {
+ if (copy_to_user((void __user *)devs.device_status,
+ (void *)&device_status,
+ sizeof(device_status)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devs.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_boot_device(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkbootdevice boot = {0};
+ char filename[64];
+ int rc = 0;
+
+ if (copy_from_user(&boot, (void __user *)arg,
+ sizeof(struct xlinkbootdevice)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)boot.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (boot.binary_name_size > sizeof(filename))
+ return -EINVAL;
+ memset(filename, 0, sizeof(filename));
+ if (copy_from_user(filename, (void __user *)boot.binary_name,
+ boot.binary_name_size))
+ return -EFAULT;
+ rc = xlink_boot_device(&devh, filename);
+ if (copy_to_user((void __user *)boot.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_reset_device(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkresetdevice res = {0};
+ int rc = 0;
+
+ if (copy_from_user(&res, (void __user *)arg,
+ sizeof(struct xlinkresetdevice)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)res.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_reset_device(&devh);
+ if (copy_to_user((void __user *)res.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_get_device_mode(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkdevmode devm = {0};
+ u32 device_mode = 0;
+ int rc = 0;
+
+ if (copy_from_user(&devm, (void __user *)arg,
+ sizeof(struct xlinkdevmode)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devm.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ rc = xlink_get_device_mode(&devh, &device_mode);
+ if (!rc) {
+ if (copy_to_user((void __user *)devm.device_mode, (void *)&device_mode,
+ sizeof(device_mode)))
+ return -EFAULT;
+ }
+ if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_set_device_mode(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkdevmode devm = {0};
+ u32 device_mode = 0;
+ int rc = 0;
+
+ if (copy_from_user(&devm, (void __user *)arg,
+ sizeof(struct xlinkdevmode)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)devm.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ if (copy_from_user(&device_mode, (void __user *)devm.device_mode,
+ sizeof(device_mode)))
+ return -EFAULT;
+ rc = xlink_set_device_mode(&devh, device_mode);
+ if (copy_to_user((void __user *)devm.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_register_device_event(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkregdevevent regdevevent = {0};
+ u32 num_events = 0;
+ u32 *ev_list;
+ int rc = 0;
+
+ if (copy_from_user(®devevent, (void __user *)arg,
+ sizeof(struct xlinkregdevevent)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)regdevevent.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ num_events = regdevevent.num_events;
+ if (num_events > 0 && num_events <= NUM_REG_EVENTS) {
+ ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
+ if (ev_list) {
+ if (copy_from_user(ev_list,
+ (void __user *)regdevevent.event_list,
+ (num_events * sizeof(u32)))) {
+ kfree(ev_list);
+ return -EFAULT;
+ }
+ rc = xlink_register_device_event_user(&devh,
+ ev_list,
+ num_events,
+ NULL);
+ kfree(ev_list);
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_unregister_device_event(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkregdevevent regdevevent = {0};
+ u32 num_events = 0;
+ u32 *ev_list;
+ int rc = 0;
+
+ if (copy_from_user(®devevent, (void __user *)arg,
+ sizeof(struct xlinkregdevevent)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)regdevevent.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ num_events = regdevevent.num_events;
+ if (num_events <= NUM_REG_EVENTS) {
+ ev_list = kzalloc((num_events * sizeof(u32)), GFP_KERNEL);
+ if (copy_from_user(ev_list,
+ (void __user *)regdevevent.event_list,
+ (num_events * sizeof(u32)))) {
+ kfree(ev_list);
+ return -EFAULT;
+ }
+ rc = xlink_unregister_device_event(&devh, ev_list, num_events);
+ kfree(ev_list);
+ } else {
+ rc = X_LINK_ERROR;
+ }
+ if (copy_to_user((void __user *)regdevevent.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_data_ready_callback(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkcallback cb = {0};
+ int rc = 0;
+
+ if (copy_from_user(&cb, (void __user *)arg,
+ sizeof(struct xlinkcallback)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)cb.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
+ rc = xlink_data_available_event(&devh, cb.chan, cb.callback);
+ if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
+
+int ioctl_data_consumed_callback(unsigned long arg)
+{
+ struct xlink_handle devh = {0};
+ struct xlinkcallback cb = {0};
+ int rc = 0;
+
+ if (copy_from_user(&cb, (void __user *)arg,
+ sizeof(struct xlinkcallback)))
+ return -EFAULT;
+ if (copy_from_user(&devh, (void __user *)cb.handle,
+ sizeof(struct xlink_handle)))
+ return -EFAULT;
+ CHANNEL_SET_USER_BIT(cb.chan); // set MSbit for user space call
+ rc = xlink_data_consumed_event(&devh, cb.chan, cb.callback);
+ if (copy_to_user((void __user *)cb.return_code, (void *)&rc, sizeof(rc)))
+ return -EFAULT;
+ return rc;
+}
diff --git a/drivers/misc/xlink-core/xlink-ioctl.h b/drivers/misc/xlink-core/xlink-ioctl.h
new file mode 100644
index 000000000000..7818b676d488
--- /dev/null
+++ b/drivers/misc/xlink-core/xlink-ioctl.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * xlink ioctl header files.
+ *
+ * Copyright (C) 2018-2019 Intel Corporation
+ *
+ */
+
+#ifndef XLINK_IOCTL_H_
+#define XLINK_IOCTL_H_
+
+int ioctl_connect(unsigned long arg);
+int ioctl_open_channel(unsigned long arg);
+int ioctl_read_data(unsigned long arg);
+int ioctl_read_to_buffer(unsigned long arg);
+int ioctl_write_data(unsigned long arg);
+int ioctl_write_control_data(unsigned long arg);
+int ioctl_write_volatile_data(unsigned long arg);
+int ioctl_release_data(unsigned long arg);
+int ioctl_close_channel(unsigned long arg);
+int ioctl_start_vpu(unsigned long arg);
+int ioctl_stop_vpu(void);
+int ioctl_disconnect(unsigned long arg);
+int ioctl_get_device_name(unsigned long arg);
+int ioctl_get_device_list(unsigned long arg);
+int ioctl_get_device_status(unsigned long arg);
+int ioctl_boot_device(unsigned long arg);
+int ioctl_reset_device(unsigned long arg);
+int ioctl_get_device_mode(unsigned long arg);
+int ioctl_set_device_mode(unsigned long arg);
+int ioctl_register_device_event(unsigned long arg);
+int ioctl_unregister_device_event(unsigned long arg);
+int ioctl_data_ready_callback(unsigned long arg);
+int ioctl_data_consumed_callback(unsigned long arg);
+
+#endif /* XLINK_IOCTL_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 00/22] Intel Vision Processing Unit base enabling part 1
2020-11-30 23:06 [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 mgross
` (21 preceding siblings ...)
2020-11-30 23:07 ` [PATCH 22/22] xlink-core: factorize xlink_ioctl function by creating sub-functions for each ioctl command mgross
@ 2020-12-01 10:14 ` Greg KH
2020-12-01 17:53 ` mark gross
22 siblings, 1 reply; 34+ messages in thread
From: Greg KH @ 2020-12-01 10:14 UTC (permalink / raw)
To: mgross; +Cc: linux-kernel, markgross, adam.r.gretzinger
On Mon, Nov 30, 2020 at 03:06:45PM -0800, mgross@linux.intel.com wrote:
> From: mark gross <mgross@linux.intel.com>
>
> The Intel Vision Processing Unit (VPU) is an IP block that is showing up for
> the first time as part of the Keem Bay SOC. Keem Bay is a quad core A53 Arm
> SOC. It is designed to be used as a stand alone SOC as well as in an PCIe
> Vision Processing accelerator add in card.
>
> This part 1 of the patches make up the base or core of the stack needed to
> enable both use cases for the VPU.
>
> Part 2 includes 11 more patches that depend on part 1. Those should be ready
> in a couple of weeks or less.
>
> I am trying something a bit new with this sequence where I've been working with
> the driver developers as a "pre-maintainer" reviewing and enforcing the kernel
> expectations as I understand them. Its taken a couple of months to get this
> code to the point I feel its ready for public posting. My goal is to make sure
> it meets expectations for quality and compliance with kernel expectations and
> there will be mostly technical / design issues to talk about.
>
> Thanks for looking at these and providing feedback.
>
> --mark
> p.s. I have had a problem my MTA configuration between mutt and git send-email
> where I was using msmpt to send from mutt (because 15+ years ago its the first
> way I got to work and never changed) while my worstation MTA that git
> send-email uses was un-configured resulting in my return-path naming my
> workstion withing the firewall. I suck at email administration.
>
> I appologies for the multiple copies.
Ah, here's the full set of patches...
But, you didn't cc: everyone on them, some of us just got a partial set
of patches, why?
still confused,
greg k-h
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [PATCH 00/22] Intel Vision Processing Unit base enabling part 1
2020-12-01 10:14 ` [PATCH 00/22] Intel Vision Processing Unit base enabling part 1 Greg KH
@ 2020-12-01 17:53 ` mark gross
0 siblings, 0 replies; 34+ messages in thread
From: mark gross @ 2020-12-01 17:53 UTC (permalink / raw)
To: Greg KH; +Cc: mgross, linux-kernel, markgross, adam.r.gretzinger
On Tue, Dec 01, 2020 at 11:14:12AM +0100, Greg KH wrote:
> On Mon, Nov 30, 2020 at 03:06:45PM -0800, mgross@linux.intel.com wrote:
> > From: mark gross <mgross@linux.intel.com>
> >
> > The Intel Vision Processing Unit (VPU) is an IP block that is showing up for
> > the first time as part of the Keem Bay SOC. Keem Bay is a quad core A53 Arm
> > SOC. It is designed to be used as a stand alone SOC as well as in an PCIe
> > Vision Processing accelerator add in card.
> >
> > This part 1 of the patches make up the base or core of the stack needed to
> > enable both use cases for the VPU.
> >
> > Part 2 includes 11 more patches that depend on part 1. Those should be ready
> > in a couple of weeks or less.
> >
> > I am trying something a bit new with this sequence where I've been working with
> > the driver developers as a "pre-maintainer" reviewing and enforcing the kernel
> > expectations as I understand them. Its taken a couple of months to get this
> > code to the point I feel its ready for public posting. My goal is to make sure
> > it meets expectations for quality and compliance with kernel expectations and
> > there will be mostly technical / design issues to talk about.
> >
> > Thanks for looking at these and providing feedback.
> >
> > --mark
> > p.s. I have had a problem my MTA configuration between mutt and git send-email
> > where I was using msmpt to send from mutt (because 15+ years ago its the first
> > way I got to work and never changed) while my worstation MTA that git
> > send-email uses was un-configured resulting in my return-path naming my
> > workstion withing the firewall. I suck at email administration.
> >
> > I appologies for the multiple copies.
>
> Ah, here's the full set of patches...
>
> But, you didn't cc: everyone on them, some of us just got a partial set
> of patches, why?
Because I thought ccing everyone on all the changes was not what was expected.
Does the DT guy want to see all the non-DT changes?
this is my first time sending a big patch set that crosses subsystems like
this. I'm not sure what the preferd way to do things so I used
get_maintainer.pl on each patch to add thime in the cc: tags just above the
signed off's.
Should I have simply concatinated the list of mainttainers and CC them all on
all the patches?
>
> still confused,
its my fault.
--mark
^ permalink raw reply [flat|nested] 34+ messages in thread