[PATCH v13 0/4] userspace MHI client interface driver

[PATCH v13 0/4] userspace MHI client interface driver

[Hemant Kumar]

This patch series adds support for UCI driver. UCI driver enables userspace
clients to communicate to external MHI devices like modem and WLAN. UCI driver
probe creates standard character device file nodes for userspace clients to
perform open, read, write, poll and release file operations. These file
operations call MHI core layer APIs to perform data transfer using MHI bus
to communicate with MHI device. Patch is tested using arm64 based platform.

V13:
- Removed LOOPBACK channel from mhi_device_id table from this patch series.
Pushing a new patch series to add support for LOOPBACK channel and the user
space test application. Also removed the description from kernel documentation.
- Added QMI channel to mhi_device_id table. QMI channel has existing libqmi
support from user space.
- Updated kernel Documentation for QMI channel and provided external reference
for libqmi.
- Updated device file node name by appending mhi device name only, which already
includes mhi controller device name.

V12:
- Added loopback test driver under selftest/drivers/mhi. Updated kernel
  documentation for the usage of the loopback test application.
- Addressed review comments for renaming variable names, updated inline
  comments and removed two redundant dev_dbg.

V11:
- Fixed review comments for UCI documentation by expanding TLAs and rewording
  some sentences.

V10:
- Replaced mutex_lock with mutex_lock_interruptible in read() and write() file
  ops call back.

V9:
- Renamed dl_lock to dl_pending _lock and pending list to dl_pending for
  clarity.
- Used read lock to protect cur_buf.
- Change transfer status check logic and only consider 0 and -EOVERFLOW as
  only success.
- Added __int to module init function.
- Print channel name instead of minor number upon successful probe.

V8:
- Fixed kernel test robot compilation error by changing %lu to %zu for
  size_t.
- Replaced uci with UCI in Kconfig, commit text, and comments in driver
  code.
- Fixed minor style related comments.

V7:
- Decoupled uci device and uci channel objects. uci device is
  associated with device file node. uci channel is associated
  with MHI channels. uci device refers to uci channel to perform
  MHI channel operations for device file operations like read()
  and write(). uci device increments its reference count for
  every open(). uci device calls mhi_uci_dev_start_chan() to start
  the MHI channel. uci channel object is tracking number of times
  MHI channel is referred. This allows to keep the MHI channel in
  start state until last release() is called. After that uci channel
  reference count goes to 0 and uci channel clean up is performed
  which stops the MHI channel. After the last call to release() if
  driver is removed uci reference count becomes 0 and uci object is
  cleaned up.
- Use separate uci channel read and write lock to fine grain locking
  between reader and writer.
- Use uci device lock to synchronize open, release and driver remove.
- Optimize for downlink only or uplink only UCI device.

V6:
- Moved uci.c to mhi directory.
- Updated Kconfig to add module information.
- Updated Makefile to rename uci object file name as mhi_uci
- Removed kref for open count

V5:
- Removed mhi_uci_drv structure.
- Used idr instead of creating global list of uci devices.
- Used kref instead of local ref counting for uci device and
  open count.
- Removed unlikely macro.

V4:
- Fix locking to protect proper struct members.
- Updated documentation describing uci client driver use cases.
- Fixed uci ref counting in mhi_uci_open for error case.
- Addressed style related review comments.

V3: Added documentation for MHI UCI driver.

V2:
- Added mutex lock to prevent multiple readers to access same
- mhi buffer which can result into use after free.

Hemant Kumar (4):
  bus: mhi: core: Add helper API to return number of free TREs
  bus: mhi: core: Move MHI_MAX_MTU to external header file
  docs: Add documentation for userspace client interface
  bus: mhi: Add userspace client interface driver

 Documentation/mhi/index.rst     |   1 +
 Documentation/mhi/uci.rst       |  94 ++++++
 drivers/bus/mhi/Kconfig         |  13 +
 drivers/bus/mhi/Makefile        |   3 +
 drivers/bus/mhi/core/internal.h |   1 -
 drivers/bus/mhi/core/main.c     |  12 +
 drivers/bus/mhi/uci.c           | 665 ++++++++++++++++++++++++++++++++++++++++
 include/linux/mhi.h             |  12 +
 8 files changed, 800 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/mhi/uci.rst
 create mode 100644 drivers/bus/mhi/uci.c

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH v13 1/4] bus: mhi: core: Add helper API to return number of free TREs

[Hemant Kumar]

Introduce mhi_get_free_desc_count() API to return number
of TREs available to queue buffer. MHI clients can use this
API to know before hand if ring is full without calling queue
API.

Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
Reviewed-by: Jeffrey Hugo <jhugo@codeaurora.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/bus/mhi/core/main.c | 12 ++++++++++++
 include/linux/mhi.h         |  9 +++++++++
 2 files changed, 21 insertions(+)

diff --git a/drivers/bus/mhi/core/main.c b/drivers/bus/mhi/core/main.c
index 4eb93d8..defd579a 100644
--- a/drivers/bus/mhi/core/main.c
+++ b/drivers/bus/mhi/core/main.c
@@ -260,6 +260,18 @@ int mhi_destroy_device(struct device *dev, void *data)
 	return 0;
 }
 
+int mhi_get_free_desc_count(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir)
+{
+	struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+	struct mhi_chan *mhi_chan = (dir == DMA_TO_DEVICE) ?
+		mhi_dev->ul_chan : mhi_dev->dl_chan;
+	struct mhi_ring *tre_ring = &mhi_chan->tre_ring;
+
+	return get_nr_avail_ring_elements(mhi_cntrl, tre_ring);
+}
+EXPORT_SYMBOL_GPL(mhi_get_free_desc_count);
+
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason)
 {
 	struct mhi_driver *mhi_drv;
diff --git a/include/linux/mhi.h b/include/linux/mhi.h
index d31efcf..b3bc966 100644
--- a/include/linux/mhi.h
+++ b/include/linux/mhi.h
@@ -597,6 +597,15 @@ void mhi_set_mhi_state(struct mhi_controller *mhi_cntrl,
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason);
 
 /**
+ * mhi_get_free_desc_count - Get transfer ring length
+ * Get # of TD available to queue buffers
+ * @mhi_dev: Device associated with the channels
+ * @dir: Direction of the channel
+ */
+int mhi_get_free_desc_count(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir);
+
+/**
  * mhi_prepare_for_power_up - Do pre-initialization before power up.
  *                            This is optional, call this before power up if
  *                            the controller does not want bus framework to
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH v13 2/4] bus: mhi: core: Move MHI_MAX_MTU to external header file

[Hemant Kumar]

Currently this macro is defined in internal MHI header as
a TRE length mask. Moving it to external header allows MHI
client drivers to set this upper bound for the transmit
buffer size.

Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
Reviewed-by: Jeffrey Hugo <jhugo@codeaurora.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/bus/mhi/core/internal.h | 1 -
 include/linux/mhi.h             | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/bus/mhi/core/internal.h b/drivers/bus/mhi/core/internal.h
index 6f80ec3..2b9c063 100644
--- a/drivers/bus/mhi/core/internal.h
+++ b/drivers/bus/mhi/core/internal.h
@@ -453,7 +453,6 @@ enum mhi_pm_state {
 #define CMD_EL_PER_RING			128
 #define PRIMARY_CMD_RING		0
 #define MHI_DEV_WAKE_DB			127
-#define MHI_MAX_MTU			0xffff
 #define MHI_RANDOM_U32_NONZERO(bmsk)	(prandom_u32_max(bmsk) + 1)
 
 enum mhi_er_type {
diff --git a/include/linux/mhi.h b/include/linux/mhi.h
index b3bc966..fc903b2 100644
--- a/include/linux/mhi.h
+++ b/include/linux/mhi.h
@@ -15,6 +15,9 @@
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 
+/* MHI client drivers to set this upper bound for tx buffer */
+#define MHI_MAX_MTU 0xffff
+
 #define MHI_MAX_OEM_PK_HASH_SEGMENTS 16
 
 struct mhi_chan;
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH v13 3/4] docs: Add documentation for userspace client interface

[Hemant Kumar]

MHI userspace client driver is creating device file node
for user application to perform file operations. File
operations are handled by MHI core driver. Currently
QMI MHI channel is supported by this driver.

Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
---
 Documentation/mhi/index.rst |  1 +
 Documentation/mhi/uci.rst   | 94 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+)
 create mode 100644 Documentation/mhi/uci.rst

diff --git a/Documentation/mhi/index.rst b/Documentation/mhi/index.rst
index 1d8dec3..c75a371 100644
--- a/Documentation/mhi/index.rst
+++ b/Documentation/mhi/index.rst
@@ -9,6 +9,7 @@ MHI
 
    mhi
    topology
+   uci
 
 .. only::  subproject and html
 
diff --git a/Documentation/mhi/uci.rst b/Documentation/mhi/uci.rst
new file mode 100644
index 0000000..ad22650
--- /dev/null
+++ b/Documentation/mhi/uci.rst
@@ -0,0 +1,94 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================================
+Userspace Client Interface (UCI)
+=================================
+
+UCI driver enables userspace clients to communicate to external MHI devices
+like modem and WLAN. UCI driver probe creates standard character device file
+nodes for userspace clients to perform open, read, write, poll and release file
+operations. UCI device object represents UCI device file node which gets
+instantiated as part of MHI UCI driver probe. UCI channel object represents
+MHI uplink or downlink channel.
+
+Operations
+==========
+
+open
+----
+
+Instantiates UCI channel object and starts MHI channels to move it to running
+state. Inbound buffers are queued to downlink channel transfer ring. Every
+subsequent open() increments UCI device reference count as well as UCI channel
+reference count.
+
+read
+----
+
+When data transfer is completed on downlink channel, transfer ring element
+buffer is copied to pending list. Reader is unblocked and data is copied to
+userspace buffer. Transfer ring element buffer is queued back to downlink
+channel transfer ring.
+
+write
+-----
+
+Write buffer is queued to uplink channel transfer ring if ring is not full. Upon
+uplink transfer completion buffer is freed.
+
+poll
+----
+
+Returns EPOLLIN | EPOLLRDNORM mask if pending list has buffers to be read by
+userspace. Returns EPOLLOUT | EPOLLWRNORM mask if MHI uplink channel transfer
+ring is not empty. Returns EPOLLERR when UCI driver is removed.
+
+release
+-------
+
+Decrements UCI device reference count and UCI channel reference count upon last
+release(). UCI channel clean up is performed. MHI channel moves to disable
+state and inbound buffers are freed.
+
+Usage
+=====
+
+Device file node is created with format:-
+
+/dev/mhi_<mhi_device_name>
+
+mhi_device_name includes mhi controller name and the name of the MHI channel
+being used by MHI client in userspace to send or receive data using MHI
+protocol.
+
+There is a separate character device file node created for each channel
+specified in MHI device id table. MHI channels are statically defined by MHI
+specification. The list of supported channels is in the channel list variable
+of mhi_device_id table in UCI driver.
+
+Qualcomm MSM Interface(QMI) Channel
+-----------------------------------
+
+Qualcomm MSM Interface(QMI) is a modem control messaging protocol used to
+communicate between software components in the modem and other peripheral
+subsystems. QMI communication is of request/response type or an unsolicited
+event type. libqmi is userspace MHI client which communicates to a QMI service
+using UCI device. It sends a QMI request to a QMI service using MHI channel 14
+or 16. QMI response is received using MHI channel 15 or 17 respectively. libqmi
+is a glib-based library for talking to WWAN modems and devices which speaks QMI
+protocol. For more information about libqmi please refer
+https://www.freedesktop.org/wiki/Software/libqmi/
+
+Usage Example
+~~~~~~~~~~~~~
+
+QMI command to retrieve device mode
+$ sudo qmicli -d /dev/mhi_0000\:02\:00.0_QMI --dms-get-model
+[/dev/mhi_0000:02:00.0_QMI] Device model retrieved:
+    Model: 'FN980m'
+
+Other Use Cases
+---------------
+
+Getting MHI device specific diagnostics information to userspace MHI diagnostic
+client using DIAG channel 4 (Host to device) and 5 (Device to Host).
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Hemant Kumar]

This MHI client driver allows userspace clients to transfer
raw data between MHI device and host using standard file operations.
Driver instantiates UCI device object which is associated to device
file node. UCI device object instantiates UCI channel object when device
file node is opened. UCI channel object is used to manage MHI channels
by calling MHI core APIs for read and write operations. MHI channels
are started as part of device open(). MHI channels remain in start
state until last release() is called on UCI device file node. Device
file node is created with format

/dev/mhi_<mhi_device_name>

Currently it supports QMI channel.

Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
---
 drivers/bus/mhi/Kconfig  |  13 +
 drivers/bus/mhi/Makefile |   3 +
 drivers/bus/mhi/uci.c    | 665 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 681 insertions(+)
 create mode 100644 drivers/bus/mhi/uci.c

diff --git a/drivers/bus/mhi/Kconfig b/drivers/bus/mhi/Kconfig
index da5cd0c..5194e8e 100644
--- a/drivers/bus/mhi/Kconfig
+++ b/drivers/bus/mhi/Kconfig
@@ -29,3 +29,16 @@ config MHI_BUS_PCI_GENERIC
 	  This driver provides MHI PCI controller driver for devices such as
 	  Qualcomm SDX55 based PCIe modems.
 
+config MHI_UCI
+	tristate "MHI UCI"
+	depends on MHI_BUS
+	help
+	  MHI based Userspace Client Interface (UCI) driver is used for
+	  transferring raw data between host and device using standard file
+	  operations from userspace. Open, read, write, poll and close
+	  operations are supported by this driver. Please check
+	  mhi_uci_match_table for all supported channels that are exposed to
+	  userspace.
+
+	  To compile this driver as a module, choose M here: the module will be
+	  called mhi_uci.
diff --git a/drivers/bus/mhi/Makefile b/drivers/bus/mhi/Makefile
index 0a2d778..69f2111 100644
--- a/drivers/bus/mhi/Makefile
+++ b/drivers/bus/mhi/Makefile
@@ -4,3 +4,6 @@ obj-y += core/
 obj-$(CONFIG_MHI_BUS_PCI_GENERIC) += mhi_pci_generic.o
 mhi_pci_generic-y += pci_generic.o
 
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
diff --git a/drivers/bus/mhi/uci.c b/drivers/bus/mhi/uci.c
new file mode 100644
index 0000000..fb9c183
--- /dev/null
+++ b/drivers/bus/mhi/uci.c
@@ -0,0 +1,665 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
+
+#include <linux/kernel.h>
+#include <linux/mhi.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#define MHI_DEVICE_NAME "mhi"
+#define MHI_UCI_DRIVER_NAME "mhi_uci"
+#define MHI_MAX_UCI_MINORS 128
+
+static DEFINE_IDR(uci_idr);
+static DEFINE_MUTEX(uci_drv_mutex);
+static struct class *uci_dev_class;
+static int uci_dev_major;
+
+/**
+ * struct uci_chan - MHI channel for a UCI device
+ * @udev: associated UCI device object
+ * @ul_wq: wait queue for writer
+ * @write_lock: mutex write lock for ul channel
+ * @dl_wq: wait queue for reader
+ * @read_lock: mutex read lock for dl channel
+ * @dl_pending_lock: spin lock for dl_pending list
+ * @dl_pending: list of dl buffers userspace is waiting to read
+ * @cur_buf: current buffer userspace is reading
+ * @dl_size: size of the current dl buffer userspace is reading
+ * @ref_count: uci_chan reference count
+ */
+struct uci_chan {
+	struct uci_dev *udev;
+	wait_queue_head_t ul_wq;
+
+	/* ul channel lock to synchronize multiple writes */
+	struct mutex write_lock;
+
+	wait_queue_head_t dl_wq;
+
+	/* dl channel lock to synchronize multiple reads */
+	struct mutex read_lock;
+
+	/*
+	 * protects pending list in bh context, channel release, read and
+	 * poll
+	 */
+	spinlock_t dl_pending_lock;
+
+	struct list_head dl_pending;
+	struct uci_buf *cur_buf;
+	size_t dl_size;
+	struct kref ref_count;
+};
+
+/**
+ * struct uci_buf - UCI buffer
+ * @data: data buffer
+ * @len: length of data buffer
+ * @node: list node of the UCI buffer
+ */
+struct uci_buf {
+	void *data;
+	size_t len;
+	struct list_head node;
+};
+
+/**
+ * struct uci_dev - MHI UCI device
+ * @minor: UCI device node minor number
+ * @mhi_dev: associated mhi device object
+ * @uchan: UCI uplink and downlink channel object
+ * @mtu: max TRE buffer length
+ * @enabled: Flag to track the state of the UCI device
+ * @lock: mutex lock to manage uchan object
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+	unsigned int minor;
+	struct mhi_device *mhi_dev;
+	struct uci_chan *uchan;
+	size_t mtu;
+	bool enabled;
+
+	/* synchronize open, release and driver remove */
+	struct mutex lock;
+	struct kref ref_count;
+};
+
+static void mhi_uci_dev_chan_release(struct kref *ref)
+{
+	struct uci_buf *buf_itr, *tmp;
+	struct uci_chan *uchan =
+		container_of(ref, struct uci_chan, ref_count);
+
+	if (uchan->udev->enabled)
+		mhi_unprepare_from_transfer(uchan->udev->mhi_dev);
+
+	spin_lock_bh(&uchan->dl_pending_lock);
+	list_for_each_entry_safe(buf_itr, tmp, &uchan->dl_pending, node) {
+		list_del(&buf_itr->node);
+		kfree(buf_itr->data);
+	}
+	spin_unlock_bh(&uchan->dl_pending_lock);
+
+	wake_up(&uchan->ul_wq);
+	wake_up(&uchan->dl_wq);
+
+	mutex_lock(&uchan->read_lock);
+	if (uchan->cur_buf)
+		kfree(uchan->cur_buf->data);
+
+	uchan->cur_buf = NULL;
+	mutex_unlock(&uchan->read_lock);
+
+	mutex_destroy(&uchan->write_lock);
+	mutex_destroy(&uchan->read_lock);
+
+	uchan->udev->uchan = NULL;
+	kfree(uchan);
+}
+
+static int mhi_queue_inbound(struct uci_dev *udev)
+{
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	int nr_desc, i, ret = -EIO;
+	size_t dl_buf_size;
+	void *buf;
+	struct uci_buf *ubuf;
+
+	/*
+	 * skip queuing without error if dl channel is not supported. This
+	 * allows open to succeed for udev, supporting ul only channel.
+	 */
+	if (!udev->mhi_dev->dl_chan)
+		return 0;
+
+	nr_desc = mhi_get_free_desc_count(mhi_dev, DMA_FROM_DEVICE);
+
+	for (i = 0; i < nr_desc; i++) {
+		buf = kmalloc(udev->mtu, GFP_KERNEL);
+		if (!buf)
+			return -ENOMEM;
+
+		dl_buf_size = udev->mtu - sizeof(*ubuf);
+
+		/* save uci_buf info at the end of buf */
+		ubuf = buf + dl_buf_size;
+		ubuf->data = buf;
+
+		dev_dbg(dev, "Allocated buf %d of %d size %zu\n", i, nr_desc,
+			dl_buf_size);
+
+		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, dl_buf_size,
+				    MHI_EOT);
+		if (ret) {
+			kfree(buf);
+			dev_err(dev, "Failed to queue buffer %d\n", i);
+			return ret;
+		}
+	}
+
+	return ret;
+}
+
+static int mhi_uci_dev_start_chan(struct uci_dev *udev)
+{
+	int ret = 0;
+	struct uci_chan *uchan;
+
+	mutex_lock(&udev->lock);
+	if (!udev->uchan || !kref_get_unless_zero(&udev->uchan->ref_count)) {
+		uchan = kzalloc(sizeof(*uchan), GFP_KERNEL);
+		if (!uchan) {
+			ret = -ENOMEM;
+			goto error_chan_start;
+		}
+
+		udev->uchan = uchan;
+		uchan->udev = udev;
+		init_waitqueue_head(&uchan->ul_wq);
+		init_waitqueue_head(&uchan->dl_wq);
+		mutex_init(&uchan->write_lock);
+		mutex_init(&uchan->read_lock);
+		spin_lock_init(&uchan->dl_pending_lock);
+		INIT_LIST_HEAD(&uchan->dl_pending);
+
+		ret = mhi_prepare_for_transfer(udev->mhi_dev);
+		if (ret) {
+			dev_err(&udev->mhi_dev->dev, "Error starting transfer channels\n");
+			goto error_chan_cleanup;
+		}
+
+		ret = mhi_queue_inbound(udev);
+		if (ret)
+			goto error_chan_cleanup;
+
+		kref_init(&uchan->ref_count);
+	}
+
+	mutex_unlock(&udev->lock);
+
+	return 0;
+
+error_chan_cleanup:
+	mhi_uci_dev_chan_release(&uchan->ref_count);
+error_chan_start:
+	mutex_unlock(&udev->lock);
+	return ret;
+}
+
+static void mhi_uci_dev_release(struct kref *ref)
+{
+	struct uci_dev *udev =
+		container_of(ref, struct uci_dev, ref_count);
+
+	mutex_destroy(&udev->lock);
+
+	kfree(udev);
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+	unsigned int minor = iminor(inode);
+	struct uci_dev *udev = NULL;
+	int ret;
+
+	mutex_lock(&uci_drv_mutex);
+	udev = idr_find(&uci_idr, minor);
+	if (!udev) {
+		pr_debug("uci dev: minor %d not found\n", minor);
+		mutex_unlock(&uci_drv_mutex);
+		return -ENODEV;
+	}
+
+	kref_get(&udev->ref_count);
+	mutex_unlock(&uci_drv_mutex);
+
+	ret = mhi_uci_dev_start_chan(udev);
+	if (ret) {
+		kref_put(&udev->ref_count, mhi_uci_dev_release);
+		return ret;
+	}
+
+	filp->private_data = udev;
+
+	return 0;
+}
+
+static int mhi_uci_release(struct inode *inode, struct file *file)
+{
+	struct uci_dev *udev = file->private_data;
+
+	mutex_lock(&udev->lock);
+	kref_put(&udev->uchan->ref_count, mhi_uci_dev_chan_release);
+	mutex_unlock(&udev->lock);
+
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+
+	return 0;
+}
+
+static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan = udev->uchan;
+	__poll_t mask = 0;
+
+	poll_wait(file, &udev->uchan->ul_wq, wait);
+	poll_wait(file, &udev->uchan->dl_wq, wait);
+
+	if (!udev->enabled) {
+		mask = EPOLLERR;
+		goto done;
+	}
+
+	spin_lock_bh(&uchan->dl_pending_lock);
+	if (!list_empty(&uchan->dl_pending) || uchan->cur_buf)
+		mask |= EPOLLIN | EPOLLRDNORM;
+	spin_unlock_bh(&uchan->dl_pending_lock);
+
+	if (mhi_get_free_desc_count(mhi_dev, DMA_TO_DEVICE) > 0)
+		mask |= EPOLLOUT | EPOLLWRNORM;
+
+	dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
+
+done:
+	return mask;
+}
+
+static ssize_t mhi_uci_write(struct file *file,
+			     const char __user *buf,
+			     size_t count,
+			     loff_t *offp)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan = udev->uchan;
+	size_t bytes_xfered = 0;
+	int ret, nr_desc = 0;
+
+	/* if ul channel is not supported return error */
+	if (!mhi_dev->ul_chan)
+		return -EOPNOTSUPP;
+
+	if (!buf || !count)
+		return -EINVAL;
+
+	dev_dbg(dev, "%s: to xfer: %zu bytes\n", __func__, count);
+
+	if (mutex_lock_interruptible(&uchan->write_lock))
+		return -EINTR;
+
+	while (count) {
+		size_t xfer_size;
+		void *kbuf;
+		enum mhi_flags flags;
+
+		/* wait for free descriptors */
+		ret = wait_event_interruptible(uchan->ul_wq,
+					       (!udev->enabled) ||
+				(nr_desc = mhi_get_free_desc_count(mhi_dev,
+					       DMA_TO_DEVICE)) > 0);
+
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Interrupted by a signal in %s, exiting\n",
+				__func__);
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+
+		xfer_size = min_t(size_t, count, udev->mtu);
+		kbuf = kmalloc(xfer_size, GFP_KERNEL);
+		if (!kbuf) {
+			ret = -ENOMEM;
+			goto err_mtx_unlock;
+		}
+
+		ret = copy_from_user(kbuf, buf, xfer_size);
+		if (ret) {
+			kfree(kbuf);
+			ret = -EFAULT;
+			goto err_mtx_unlock;
+		}
+
+		/* if ring is full after this force EOT */
+		if (nr_desc > 1 && (count - xfer_size))
+			flags = MHI_CHAIN;
+		else
+			flags = MHI_EOT;
+
+		ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf, xfer_size,
+				    flags);
+		if (ret) {
+			kfree(kbuf);
+			goto err_mtx_unlock;
+		}
+
+		bytes_xfered += xfer_size;
+		count -= xfer_size;
+		buf += xfer_size;
+	}
+
+	mutex_unlock(&uchan->write_lock);
+	dev_dbg(dev, "%s: bytes xferred: %zu\n", __func__, bytes_xfered);
+
+	return bytes_xfered;
+
+err_mtx_unlock:
+	mutex_unlock(&uchan->write_lock);
+
+	return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+			    char __user *buf,
+			    size_t count,
+			    loff_t *ppos)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct uci_chan *uchan = udev->uchan;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_buf *ubuf;
+	size_t rx_buf_size;
+	char *ptr;
+	size_t to_copy;
+	int ret = 0;
+
+	/* if dl channel is not supported return error */
+	if (!mhi_dev->dl_chan)
+		return -EOPNOTSUPP;
+
+	if (!buf)
+		return -EINVAL;
+
+	if (mutex_lock_interruptible(&uchan->read_lock))
+		return -EINTR;
+
+	spin_lock_bh(&uchan->dl_pending_lock);
+	/* No data available to read, wait */
+	if (!uchan->cur_buf && list_empty(&uchan->dl_pending)) {
+		dev_dbg(dev, "No data available to read, waiting\n");
+
+		spin_unlock_bh(&uchan->dl_pending_lock);
+		ret = wait_event_interruptible(uchan->dl_wq,
+					       (!udev->enabled ||
+					      !list_empty(&uchan->dl_pending)));
+
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Interrupted by a signal in %s, exiting\n",
+				__func__);
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+		spin_lock_bh(&uchan->dl_pending_lock);
+	}
+
+	/* new read, get the next descriptor from the list */
+	if (!uchan->cur_buf) {
+		ubuf = list_first_entry_or_null(&uchan->dl_pending,
+						struct uci_buf, node);
+		if (!ubuf) {
+			ret = -EIO;
+			goto err_spin_unlock;
+		}
+
+		list_del(&ubuf->node);
+		uchan->cur_buf = ubuf;
+		uchan->dl_size = ubuf->len;
+		dev_dbg(dev, "Got pkt of size: %zu\n", uchan->dl_size);
+	}
+	spin_unlock_bh(&uchan->dl_pending_lock);
+
+	ubuf = uchan->cur_buf;
+
+	/* Copy the buffer to user space */
+	to_copy = min_t(size_t, count, uchan->dl_size);
+	ptr = ubuf->data + (ubuf->len - uchan->dl_size);
+
+	ret = copy_to_user(buf, ptr, to_copy);
+	if (ret) {
+		ret = -EFAULT;
+		goto err_mtx_unlock;
+	}
+
+	dev_dbg(dev, "Copied %zu of %zu bytes\n", to_copy, uchan->dl_size);
+	uchan->dl_size -= to_copy;
+
+	/* we finished with this buffer, queue it back to hardware */
+	if (!uchan->dl_size) {
+		uchan->cur_buf = NULL;
+
+		rx_buf_size = udev->mtu - sizeof(*ubuf);
+		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, ubuf->data,
+				    rx_buf_size, MHI_EOT);
+		if (ret) {
+			dev_err(dev, "Failed to recycle element: %d\n", ret);
+			kfree(ubuf->data);
+			goto err_mtx_unlock;
+		}
+	}
+	mutex_unlock(&uchan->read_lock);
+
+	dev_dbg(dev, "%s: Returning %zu bytes\n", __func__, to_copy);
+
+	return to_copy;
+
+err_spin_unlock:
+	spin_unlock_bh(&uchan->dl_pending_lock);
+err_mtx_unlock:
+	mutex_unlock(&uchan->read_lock);
+	return ret;
+}
+
+static const struct file_operations mhidev_fops = {
+	.owner = THIS_MODULE,
+	.open = mhi_uci_open,
+	.release = mhi_uci_release,
+	.read = mhi_uci_read,
+	.write = mhi_uci_write,
+	.poll = mhi_uci_poll,
+};
+
+static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
+			   struct mhi_result *mhi_result)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+	struct uci_chan *uchan = udev->uchan;
+	struct device *dev = &mhi_dev->dev;
+
+	dev_dbg(dev, "%s: status: %d xfer_len: %zu\n", __func__,
+		mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+	kfree(mhi_result->buf_addr);
+
+	if (!mhi_result->transaction_status)
+		wake_up(&uchan->ul_wq);
+}
+
+static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
+			   struct mhi_result *mhi_result)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+	struct uci_chan *uchan = udev->uchan;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_buf *ubuf;
+	size_t dl_buf_size = udev->mtu - sizeof(*ubuf);
+
+	dev_dbg(dev, "%s: status: %d receive_len: %zu\n", __func__,
+		mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+	if (mhi_result->transaction_status &&
+	    mhi_result->transaction_status != -EOVERFLOW) {
+		kfree(mhi_result->buf_addr);
+		return;
+	}
+
+	ubuf = mhi_result->buf_addr + dl_buf_size;
+	ubuf->data = mhi_result->buf_addr;
+	ubuf->len = mhi_result->bytes_xferd;
+	spin_lock_bh(&uchan->dl_pending_lock);
+	list_add_tail(&ubuf->node, &uchan->dl_pending);
+	spin_unlock_bh(&uchan->dl_pending_lock);
+
+	wake_up(&uchan->dl_wq);
+}
+
+static int mhi_uci_probe(struct mhi_device *mhi_dev,
+			 const struct mhi_device_id *id)
+{
+	struct uci_dev *udev;
+	struct device *dev;
+	int index;
+
+	udev = kzalloc(sizeof(*udev), GFP_KERNEL);
+	if (!udev)
+		return -ENOMEM;
+
+	kref_init(&udev->ref_count);
+	mutex_init(&udev->lock);
+	udev->mhi_dev = mhi_dev;
+
+	mutex_lock(&uci_drv_mutex);
+	index = idr_alloc(&uci_idr, udev, 0, MHI_MAX_UCI_MINORS, GFP_KERNEL);
+	mutex_unlock(&uci_drv_mutex);
+	if (index < 0) {
+		kfree(udev);
+		return index;
+	}
+
+	udev->minor = index;
+
+	udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU);
+	dev_set_drvdata(&mhi_dev->dev, udev);
+	udev->enabled = true;
+
+	/* create device file node /dev/mhi_<mhi_dev_name> */
+	dev = device_create(uci_dev_class, &mhi_dev->dev,
+			    MKDEV(uci_dev_major, index), udev,
+			    MHI_DEVICE_NAME "_%s", dev_name(&mhi_dev->dev));
+	if (IS_ERR(dev)) {
+		mutex_lock(&uci_drv_mutex);
+		idr_remove(&uci_idr, udev->minor);
+		mutex_unlock(&uci_drv_mutex);
+		dev_set_drvdata(&mhi_dev->dev, NULL);
+		kfree(udev);
+		return PTR_ERR(dev);
+	}
+
+	dev_dbg(&mhi_dev->dev, "probed uci dev: %s\n", id->chan);
+
+	return 0;
+};
+
+static void mhi_uci_remove(struct mhi_device *mhi_dev)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+
+	/* disable the node */
+	mutex_lock(&udev->lock);
+	udev->enabled = false;
+
+	/* delete the node to prevent new opens */
+	device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
+
+	/* return error for any blocked read or write */
+	if (udev->uchan) {
+		wake_up(&udev->uchan->ul_wq);
+		wake_up(&udev->uchan->dl_wq);
+	}
+	mutex_unlock(&udev->lock);
+
+	mutex_lock(&uci_drv_mutex);
+	idr_remove(&uci_idr, udev->minor);
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+	mutex_unlock(&uci_drv_mutex);
+}
+
+/* .driver_data stores max mtu */
+static const struct mhi_device_id mhi_uci_match_table[] = {
+	{ .chan = "QMI", .driver_data = 0x1000},
+	{},
+};
+MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
+
+static struct mhi_driver mhi_uci_driver = {
+	.id_table = mhi_uci_match_table,
+	.remove = mhi_uci_remove,
+	.probe = mhi_uci_probe,
+	.ul_xfer_cb = mhi_ul_xfer_cb,
+	.dl_xfer_cb = mhi_dl_xfer_cb,
+	.driver = {
+		.name = MHI_UCI_DRIVER_NAME,
+	},
+};
+
+static int __init mhi_uci_init(void)
+{
+	int ret;
+
+	ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
+	if (ret < 0)
+		return ret;
+
+	uci_dev_major = ret;
+	uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
+	if (IS_ERR(uci_dev_class)) {
+		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+		return PTR_ERR(uci_dev_class);
+	}
+
+	ret = mhi_driver_register(&mhi_uci_driver);
+	if (ret) {
+		class_destroy(uci_dev_class);
+		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+	}
+
+	return ret;
+}
+
+static void __exit mhi_uci_exit(void)
+{
+	mhi_driver_unregister(&mhi_uci_driver);
+	class_destroy(uci_dev_class);
+	unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+	idr_destroy(&uci_idr);
+}
+
+module_init(mhi_uci_init);
+module_exit(mhi_uci_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MHI UCI Driver");
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Manivannan Sadhasivam]

Hi Hemant,

On Fri, Nov 27, 2020 at 07:26:06PM -0800, Hemant Kumar wrote:
> This MHI client driver allows userspace clients to transfer
> raw data between MHI device and host using standard file operations.
> Driver instantiates UCI device object which is associated to device
> file node. UCI device object instantiates UCI channel object when device
> file node is opened. UCI channel object is used to manage MHI channels
> by calling MHI core APIs for read and write operations. MHI channels
> are started as part of device open(). MHI channels remain in start
> state until last release() is called on UCI device file node. Device
> file node is created with format
> 
> /dev/mhi_<mhi_device_name>
> 
> Currently it supports QMI channel.
> 

Thanks for the update. This patch looks good to me. But as I'm going to
apply Loic's "bus: mhi: core: Indexed MHI controller name" patch, you
need to update the documentation accordingly.

> Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>

Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Thanks,
Mani

> ---
>  drivers/bus/mhi/Kconfig  |  13 +
>  drivers/bus/mhi/Makefile |   3 +
>  drivers/bus/mhi/uci.c    | 665 +++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 681 insertions(+)
>  create mode 100644 drivers/bus/mhi/uci.c
> 
> diff --git a/drivers/bus/mhi/Kconfig b/drivers/bus/mhi/Kconfig
> index da5cd0c..5194e8e 100644
> --- a/drivers/bus/mhi/Kconfig
> +++ b/drivers/bus/mhi/Kconfig
> @@ -29,3 +29,16 @@ config MHI_BUS_PCI_GENERIC
>  	  This driver provides MHI PCI controller driver for devices such as
>  	  Qualcomm SDX55 based PCIe modems.
>  
> +config MHI_UCI
> +	tristate "MHI UCI"
> +	depends on MHI_BUS
> +	help
> +	  MHI based Userspace Client Interface (UCI) driver is used for
> +	  transferring raw data between host and device using standard file
> +	  operations from userspace. Open, read, write, poll and close
> +	  operations are supported by this driver. Please check
> +	  mhi_uci_match_table for all supported channels that are exposed to
> +	  userspace.
> +
> +	  To compile this driver as a module, choose M here: the module will be
> +	  called mhi_uci.
> diff --git a/drivers/bus/mhi/Makefile b/drivers/bus/mhi/Makefile
> index 0a2d778..69f2111 100644
> --- a/drivers/bus/mhi/Makefile
> +++ b/drivers/bus/mhi/Makefile
> @@ -4,3 +4,6 @@ obj-y += core/
>  obj-$(CONFIG_MHI_BUS_PCI_GENERIC) += mhi_pci_generic.o
>  mhi_pci_generic-y += pci_generic.o
>  
> +# MHI client
> +mhi_uci-y := uci.o
> +obj-$(CONFIG_MHI_UCI) += mhi_uci.o
> diff --git a/drivers/bus/mhi/uci.c b/drivers/bus/mhi/uci.c
> new file mode 100644
> index 0000000..fb9c183
> --- /dev/null
> +++ b/drivers/bus/mhi/uci.c
> @@ -0,0 +1,665 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
> +
> +#include <linux/kernel.h>
> +#include <linux/mhi.h>
> +#include <linux/mod_devicetable.h>
> +#include <linux/module.h>
> +#include <linux/poll.h>
> +
> +#define MHI_DEVICE_NAME "mhi"
> +#define MHI_UCI_DRIVER_NAME "mhi_uci"
> +#define MHI_MAX_UCI_MINORS 128
> +
> +static DEFINE_IDR(uci_idr);
> +static DEFINE_MUTEX(uci_drv_mutex);
> +static struct class *uci_dev_class;
> +static int uci_dev_major;
> +
> +/**
> + * struct uci_chan - MHI channel for a UCI device
> + * @udev: associated UCI device object
> + * @ul_wq: wait queue for writer
> + * @write_lock: mutex write lock for ul channel
> + * @dl_wq: wait queue for reader
> + * @read_lock: mutex read lock for dl channel
> + * @dl_pending_lock: spin lock for dl_pending list
> + * @dl_pending: list of dl buffers userspace is waiting to read
> + * @cur_buf: current buffer userspace is reading
> + * @dl_size: size of the current dl buffer userspace is reading
> + * @ref_count: uci_chan reference count
> + */
> +struct uci_chan {
> +	struct uci_dev *udev;
> +	wait_queue_head_t ul_wq;
> +
> +	/* ul channel lock to synchronize multiple writes */
> +	struct mutex write_lock;
> +
> +	wait_queue_head_t dl_wq;
> +
> +	/* dl channel lock to synchronize multiple reads */
> +	struct mutex read_lock;
> +
> +	/*
> +	 * protects pending list in bh context, channel release, read and
> +	 * poll
> +	 */
> +	spinlock_t dl_pending_lock;
> +
> +	struct list_head dl_pending;
> +	struct uci_buf *cur_buf;
> +	size_t dl_size;
> +	struct kref ref_count;
> +};
> +
> +/**
> + * struct uci_buf - UCI buffer
> + * @data: data buffer
> + * @len: length of data buffer
> + * @node: list node of the UCI buffer
> + */
> +struct uci_buf {
> +	void *data;
> +	size_t len;
> +	struct list_head node;
> +};
> +
> +/**
> + * struct uci_dev - MHI UCI device
> + * @minor: UCI device node minor number
> + * @mhi_dev: associated mhi device object
> + * @uchan: UCI uplink and downlink channel object
> + * @mtu: max TRE buffer length
> + * @enabled: Flag to track the state of the UCI device
> + * @lock: mutex lock to manage uchan object
> + * @ref_count: uci_dev reference count
> + */
> +struct uci_dev {
> +	unsigned int minor;
> +	struct mhi_device *mhi_dev;
> +	struct uci_chan *uchan;
> +	size_t mtu;
> +	bool enabled;
> +
> +	/* synchronize open, release and driver remove */
> +	struct mutex lock;
> +	struct kref ref_count;
> +};
> +
> +static void mhi_uci_dev_chan_release(struct kref *ref)
> +{
> +	struct uci_buf *buf_itr, *tmp;
> +	struct uci_chan *uchan =
> +		container_of(ref, struct uci_chan, ref_count);
> +
> +	if (uchan->udev->enabled)
> +		mhi_unprepare_from_transfer(uchan->udev->mhi_dev);
> +
> +	spin_lock_bh(&uchan->dl_pending_lock);
> +	list_for_each_entry_safe(buf_itr, tmp, &uchan->dl_pending, node) {
> +		list_del(&buf_itr->node);
> +		kfree(buf_itr->data);
> +	}
> +	spin_unlock_bh(&uchan->dl_pending_lock);
> +
> +	wake_up(&uchan->ul_wq);
> +	wake_up(&uchan->dl_wq);
> +
> +	mutex_lock(&uchan->read_lock);
> +	if (uchan->cur_buf)
> +		kfree(uchan->cur_buf->data);
> +
> +	uchan->cur_buf = NULL;
> +	mutex_unlock(&uchan->read_lock);
> +
> +	mutex_destroy(&uchan->write_lock);
> +	mutex_destroy(&uchan->read_lock);
> +
> +	uchan->udev->uchan = NULL;
> +	kfree(uchan);
> +}
> +
> +static int mhi_queue_inbound(struct uci_dev *udev)
> +{
> +	struct mhi_device *mhi_dev = udev->mhi_dev;
> +	struct device *dev = &mhi_dev->dev;
> +	int nr_desc, i, ret = -EIO;
> +	size_t dl_buf_size;
> +	void *buf;
> +	struct uci_buf *ubuf;
> +
> +	/*
> +	 * skip queuing without error if dl channel is not supported. This
> +	 * allows open to succeed for udev, supporting ul only channel.
> +	 */
> +	if (!udev->mhi_dev->dl_chan)
> +		return 0;
> +
> +	nr_desc = mhi_get_free_desc_count(mhi_dev, DMA_FROM_DEVICE);
> +
> +	for (i = 0; i < nr_desc; i++) {
> +		buf = kmalloc(udev->mtu, GFP_KERNEL);
> +		if (!buf)
> +			return -ENOMEM;
> +
> +		dl_buf_size = udev->mtu - sizeof(*ubuf);
> +
> +		/* save uci_buf info at the end of buf */
> +		ubuf = buf + dl_buf_size;
> +		ubuf->data = buf;
> +
> +		dev_dbg(dev, "Allocated buf %d of %d size %zu\n", i, nr_desc,
> +			dl_buf_size);
> +
> +		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, dl_buf_size,
> +				    MHI_EOT);
> +		if (ret) {
> +			kfree(buf);
> +			dev_err(dev, "Failed to queue buffer %d\n", i);
> +			return ret;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +static int mhi_uci_dev_start_chan(struct uci_dev *udev)
> +{
> +	int ret = 0;
> +	struct uci_chan *uchan;
> +
> +	mutex_lock(&udev->lock);
> +	if (!udev->uchan || !kref_get_unless_zero(&udev->uchan->ref_count)) {
> +		uchan = kzalloc(sizeof(*uchan), GFP_KERNEL);
> +		if (!uchan) {
> +			ret = -ENOMEM;
> +			goto error_chan_start;
> +		}
> +
> +		udev->uchan = uchan;
> +		uchan->udev = udev;
> +		init_waitqueue_head(&uchan->ul_wq);
> +		init_waitqueue_head(&uchan->dl_wq);
> +		mutex_init(&uchan->write_lock);
> +		mutex_init(&uchan->read_lock);
> +		spin_lock_init(&uchan->dl_pending_lock);
> +		INIT_LIST_HEAD(&uchan->dl_pending);
> +
> +		ret = mhi_prepare_for_transfer(udev->mhi_dev);
> +		if (ret) {
> +			dev_err(&udev->mhi_dev->dev, "Error starting transfer channels\n");
> +			goto error_chan_cleanup;
> +		}
> +
> +		ret = mhi_queue_inbound(udev);
> +		if (ret)
> +			goto error_chan_cleanup;
> +
> +		kref_init(&uchan->ref_count);
> +	}
> +
> +	mutex_unlock(&udev->lock);
> +
> +	return 0;
> +
> +error_chan_cleanup:
> +	mhi_uci_dev_chan_release(&uchan->ref_count);
> +error_chan_start:
> +	mutex_unlock(&udev->lock);
> +	return ret;
> +}
> +
> +static void mhi_uci_dev_release(struct kref *ref)
> +{
> +	struct uci_dev *udev =
> +		container_of(ref, struct uci_dev, ref_count);
> +
> +	mutex_destroy(&udev->lock);
> +
> +	kfree(udev);
> +}
> +
> +static int mhi_uci_open(struct inode *inode, struct file *filp)
> +{
> +	unsigned int minor = iminor(inode);
> +	struct uci_dev *udev = NULL;
> +	int ret;
> +
> +	mutex_lock(&uci_drv_mutex);
> +	udev = idr_find(&uci_idr, minor);
> +	if (!udev) {
> +		pr_debug("uci dev: minor %d not found\n", minor);
> +		mutex_unlock(&uci_drv_mutex);
> +		return -ENODEV;
> +	}
> +
> +	kref_get(&udev->ref_count);
> +	mutex_unlock(&uci_drv_mutex);
> +
> +	ret = mhi_uci_dev_start_chan(udev);
> +	if (ret) {
> +		kref_put(&udev->ref_count, mhi_uci_dev_release);
> +		return ret;
> +	}
> +
> +	filp->private_data = udev;
> +
> +	return 0;
> +}
> +
> +static int mhi_uci_release(struct inode *inode, struct file *file)
> +{
> +	struct uci_dev *udev = file->private_data;
> +
> +	mutex_lock(&udev->lock);
> +	kref_put(&udev->uchan->ref_count, mhi_uci_dev_chan_release);
> +	mutex_unlock(&udev->lock);
> +
> +	kref_put(&udev->ref_count, mhi_uci_dev_release);
> +
> +	return 0;
> +}
> +
> +static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
> +{
> +	struct uci_dev *udev = file->private_data;
> +	struct mhi_device *mhi_dev = udev->mhi_dev;
> +	struct device *dev = &mhi_dev->dev;
> +	struct uci_chan *uchan = udev->uchan;
> +	__poll_t mask = 0;
> +
> +	poll_wait(file, &udev->uchan->ul_wq, wait);
> +	poll_wait(file, &udev->uchan->dl_wq, wait);
> +
> +	if (!udev->enabled) {
> +		mask = EPOLLERR;
> +		goto done;
> +	}
> +
> +	spin_lock_bh(&uchan->dl_pending_lock);
> +	if (!list_empty(&uchan->dl_pending) || uchan->cur_buf)
> +		mask |= EPOLLIN | EPOLLRDNORM;
> +	spin_unlock_bh(&uchan->dl_pending_lock);
> +
> +	if (mhi_get_free_desc_count(mhi_dev, DMA_TO_DEVICE) > 0)
> +		mask |= EPOLLOUT | EPOLLWRNORM;
> +
> +	dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
> +
> +done:
> +	return mask;
> +}
> +
> +static ssize_t mhi_uci_write(struct file *file,
> +			     const char __user *buf,
> +			     size_t count,
> +			     loff_t *offp)
> +{
> +	struct uci_dev *udev = file->private_data;
> +	struct mhi_device *mhi_dev = udev->mhi_dev;
> +	struct device *dev = &mhi_dev->dev;
> +	struct uci_chan *uchan = udev->uchan;
> +	size_t bytes_xfered = 0;
> +	int ret, nr_desc = 0;
> +
> +	/* if ul channel is not supported return error */
> +	if (!mhi_dev->ul_chan)
> +		return -EOPNOTSUPP;
> +
> +	if (!buf || !count)
> +		return -EINVAL;
> +
> +	dev_dbg(dev, "%s: to xfer: %zu bytes\n", __func__, count);
> +
> +	if (mutex_lock_interruptible(&uchan->write_lock))
> +		return -EINTR;
> +
> +	while (count) {
> +		size_t xfer_size;
> +		void *kbuf;
> +		enum mhi_flags flags;
> +
> +		/* wait for free descriptors */
> +		ret = wait_event_interruptible(uchan->ul_wq,
> +					       (!udev->enabled) ||
> +				(nr_desc = mhi_get_free_desc_count(mhi_dev,
> +					       DMA_TO_DEVICE)) > 0);
> +
> +		if (ret == -ERESTARTSYS) {
> +			dev_dbg(dev, "Interrupted by a signal in %s, exiting\n",
> +				__func__);
> +			goto err_mtx_unlock;
> +		}
> +
> +		if (!udev->enabled) {
> +			ret = -ENODEV;
> +			goto err_mtx_unlock;
> +		}
> +
> +		xfer_size = min_t(size_t, count, udev->mtu);
> +		kbuf = kmalloc(xfer_size, GFP_KERNEL);
> +		if (!kbuf) {
> +			ret = -ENOMEM;
> +			goto err_mtx_unlock;
> +		}
> +
> +		ret = copy_from_user(kbuf, buf, xfer_size);
> +		if (ret) {
> +			kfree(kbuf);
> +			ret = -EFAULT;
> +			goto err_mtx_unlock;
> +		}
> +
> +		/* if ring is full after this force EOT */
> +		if (nr_desc > 1 && (count - xfer_size))
> +			flags = MHI_CHAIN;
> +		else
> +			flags = MHI_EOT;
> +
> +		ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf, xfer_size,
> +				    flags);
> +		if (ret) {
> +			kfree(kbuf);
> +			goto err_mtx_unlock;
> +		}
> +
> +		bytes_xfered += xfer_size;
> +		count -= xfer_size;
> +		buf += xfer_size;
> +	}
> +
> +	mutex_unlock(&uchan->write_lock);
> +	dev_dbg(dev, "%s: bytes xferred: %zu\n", __func__, bytes_xfered);
> +
> +	return bytes_xfered;
> +
> +err_mtx_unlock:
> +	mutex_unlock(&uchan->write_lock);
> +
> +	return ret;
> +}
> +
> +static ssize_t mhi_uci_read(struct file *file,
> +			    char __user *buf,
> +			    size_t count,
> +			    loff_t *ppos)
> +{
> +	struct uci_dev *udev = file->private_data;
> +	struct mhi_device *mhi_dev = udev->mhi_dev;
> +	struct uci_chan *uchan = udev->uchan;
> +	struct device *dev = &mhi_dev->dev;
> +	struct uci_buf *ubuf;
> +	size_t rx_buf_size;
> +	char *ptr;
> +	size_t to_copy;
> +	int ret = 0;
> +
> +	/* if dl channel is not supported return error */
> +	if (!mhi_dev->dl_chan)
> +		return -EOPNOTSUPP;
> +
> +	if (!buf)
> +		return -EINVAL;
> +
> +	if (mutex_lock_interruptible(&uchan->read_lock))
> +		return -EINTR;
> +
> +	spin_lock_bh(&uchan->dl_pending_lock);
> +	/* No data available to read, wait */
> +	if (!uchan->cur_buf && list_empty(&uchan->dl_pending)) {
> +		dev_dbg(dev, "No data available to read, waiting\n");
> +
> +		spin_unlock_bh(&uchan->dl_pending_lock);
> +		ret = wait_event_interruptible(uchan->dl_wq,
> +					       (!udev->enabled ||
> +					      !list_empty(&uchan->dl_pending)));
> +
> +		if (ret == -ERESTARTSYS) {
> +			dev_dbg(dev, "Interrupted by a signal in %s, exiting\n",
> +				__func__);
> +			goto err_mtx_unlock;
> +		}
> +
> +		if (!udev->enabled) {
> +			ret = -ENODEV;
> +			goto err_mtx_unlock;
> +		}
> +		spin_lock_bh(&uchan->dl_pending_lock);
> +	}
> +
> +	/* new read, get the next descriptor from the list */
> +	if (!uchan->cur_buf) {
> +		ubuf = list_first_entry_or_null(&uchan->dl_pending,
> +						struct uci_buf, node);
> +		if (!ubuf) {
> +			ret = -EIO;
> +			goto err_spin_unlock;
> +		}
> +
> +		list_del(&ubuf->node);
> +		uchan->cur_buf = ubuf;
> +		uchan->dl_size = ubuf->len;
> +		dev_dbg(dev, "Got pkt of size: %zu\n", uchan->dl_size);
> +	}
> +	spin_unlock_bh(&uchan->dl_pending_lock);
> +
> +	ubuf = uchan->cur_buf;
> +
> +	/* Copy the buffer to user space */
> +	to_copy = min_t(size_t, count, uchan->dl_size);
> +	ptr = ubuf->data + (ubuf->len - uchan->dl_size);
> +
> +	ret = copy_to_user(buf, ptr, to_copy);
> +	if (ret) {
> +		ret = -EFAULT;
> +		goto err_mtx_unlock;
> +	}
> +
> +	dev_dbg(dev, "Copied %zu of %zu bytes\n", to_copy, uchan->dl_size);
> +	uchan->dl_size -= to_copy;
> +
> +	/* we finished with this buffer, queue it back to hardware */
> +	if (!uchan->dl_size) {
> +		uchan->cur_buf = NULL;
> +
> +		rx_buf_size = udev->mtu - sizeof(*ubuf);
> +		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, ubuf->data,
> +				    rx_buf_size, MHI_EOT);
> +		if (ret) {
> +			dev_err(dev, "Failed to recycle element: %d\n", ret);
> +			kfree(ubuf->data);
> +			goto err_mtx_unlock;
> +		}
> +	}
> +	mutex_unlock(&uchan->read_lock);
> +
> +	dev_dbg(dev, "%s: Returning %zu bytes\n", __func__, to_copy);
> +
> +	return to_copy;
> +
> +err_spin_unlock:
> +	spin_unlock_bh(&uchan->dl_pending_lock);
> +err_mtx_unlock:
> +	mutex_unlock(&uchan->read_lock);
> +	return ret;
> +}
> +
> +static const struct file_operations mhidev_fops = {
> +	.owner = THIS_MODULE,
> +	.open = mhi_uci_open,
> +	.release = mhi_uci_release,
> +	.read = mhi_uci_read,
> +	.write = mhi_uci_write,
> +	.poll = mhi_uci_poll,
> +};
> +
> +static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
> +			   struct mhi_result *mhi_result)
> +{
> +	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
> +	struct uci_chan *uchan = udev->uchan;
> +	struct device *dev = &mhi_dev->dev;
> +
> +	dev_dbg(dev, "%s: status: %d xfer_len: %zu\n", __func__,
> +		mhi_result->transaction_status, mhi_result->bytes_xferd);
> +
> +	kfree(mhi_result->buf_addr);
> +
> +	if (!mhi_result->transaction_status)
> +		wake_up(&uchan->ul_wq);
> +}
> +
> +static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
> +			   struct mhi_result *mhi_result)
> +{
> +	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
> +	struct uci_chan *uchan = udev->uchan;
> +	struct device *dev = &mhi_dev->dev;
> +	struct uci_buf *ubuf;
> +	size_t dl_buf_size = udev->mtu - sizeof(*ubuf);
> +
> +	dev_dbg(dev, "%s: status: %d receive_len: %zu\n", __func__,
> +		mhi_result->transaction_status, mhi_result->bytes_xferd);
> +
> +	if (mhi_result->transaction_status &&
> +	    mhi_result->transaction_status != -EOVERFLOW) {
> +		kfree(mhi_result->buf_addr);
> +		return;
> +	}
> +
> +	ubuf = mhi_result->buf_addr + dl_buf_size;
> +	ubuf->data = mhi_result->buf_addr;
> +	ubuf->len = mhi_result->bytes_xferd;
> +	spin_lock_bh(&uchan->dl_pending_lock);
> +	list_add_tail(&ubuf->node, &uchan->dl_pending);
> +	spin_unlock_bh(&uchan->dl_pending_lock);
> +
> +	wake_up(&uchan->dl_wq);
> +}
> +
> +static int mhi_uci_probe(struct mhi_device *mhi_dev,
> +			 const struct mhi_device_id *id)
> +{
> +	struct uci_dev *udev;
> +	struct device *dev;
> +	int index;
> +
> +	udev = kzalloc(sizeof(*udev), GFP_KERNEL);
> +	if (!udev)
> +		return -ENOMEM;
> +
> +	kref_init(&udev->ref_count);
> +	mutex_init(&udev->lock);
> +	udev->mhi_dev = mhi_dev;
> +
> +	mutex_lock(&uci_drv_mutex);
> +	index = idr_alloc(&uci_idr, udev, 0, MHI_MAX_UCI_MINORS, GFP_KERNEL);
> +	mutex_unlock(&uci_drv_mutex);
> +	if (index < 0) {
> +		kfree(udev);
> +		return index;
> +	}
> +
> +	udev->minor = index;
> +
> +	udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU);
> +	dev_set_drvdata(&mhi_dev->dev, udev);
> +	udev->enabled = true;
> +
> +	/* create device file node /dev/mhi_<mhi_dev_name> */
> +	dev = device_create(uci_dev_class, &mhi_dev->dev,
> +			    MKDEV(uci_dev_major, index), udev,
> +			    MHI_DEVICE_NAME "_%s", dev_name(&mhi_dev->dev));
> +	if (IS_ERR(dev)) {
> +		mutex_lock(&uci_drv_mutex);
> +		idr_remove(&uci_idr, udev->minor);
> +		mutex_unlock(&uci_drv_mutex);
> +		dev_set_drvdata(&mhi_dev->dev, NULL);
> +		kfree(udev);
> +		return PTR_ERR(dev);
> +	}
> +
> +	dev_dbg(&mhi_dev->dev, "probed uci dev: %s\n", id->chan);
> +
> +	return 0;
> +};
> +
> +static void mhi_uci_remove(struct mhi_device *mhi_dev)
> +{
> +	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
> +
> +	/* disable the node */
> +	mutex_lock(&udev->lock);
> +	udev->enabled = false;
> +
> +	/* delete the node to prevent new opens */
> +	device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
> +
> +	/* return error for any blocked read or write */
> +	if (udev->uchan) {
> +		wake_up(&udev->uchan->ul_wq);
> +		wake_up(&udev->uchan->dl_wq);
> +	}
> +	mutex_unlock(&udev->lock);
> +
> +	mutex_lock(&uci_drv_mutex);
> +	idr_remove(&uci_idr, udev->minor);
> +	kref_put(&udev->ref_count, mhi_uci_dev_release);
> +	mutex_unlock(&uci_drv_mutex);
> +}
> +
> +/* .driver_data stores max mtu */
> +static const struct mhi_device_id mhi_uci_match_table[] = {
> +	{ .chan = "QMI", .driver_data = 0x1000},
> +	{},
> +};
> +MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
> +
> +static struct mhi_driver mhi_uci_driver = {
> +	.id_table = mhi_uci_match_table,
> +	.remove = mhi_uci_remove,
> +	.probe = mhi_uci_probe,
> +	.ul_xfer_cb = mhi_ul_xfer_cb,
> +	.dl_xfer_cb = mhi_dl_xfer_cb,
> +	.driver = {
> +		.name = MHI_UCI_DRIVER_NAME,
> +	},
> +};
> +
> +static int __init mhi_uci_init(void)
> +{
> +	int ret;
> +
> +	ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
> +	if (ret < 0)
> +		return ret;
> +
> +	uci_dev_major = ret;
> +	uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
> +	if (IS_ERR(uci_dev_class)) {
> +		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
> +		return PTR_ERR(uci_dev_class);
> +	}
> +
> +	ret = mhi_driver_register(&mhi_uci_driver);
> +	if (ret) {
> +		class_destroy(uci_dev_class);
> +		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
> +	}
> +
> +	return ret;
> +}
> +
> +static void __exit mhi_uci_exit(void)
> +{
> +	mhi_driver_unregister(&mhi_uci_driver);
> +	class_destroy(uci_dev_class);
> +	unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
> +	idr_destroy(&uci_idr);
> +}
> +
> +module_init(mhi_uci_init);
> +module_exit(mhi_uci_exit);
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("MHI UCI Driver");
> -- 
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Loic Poulain]

On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
>
> This MHI client driver allows userspace clients to transfer
> raw data between MHI device and host using standard file operations.
> Driver instantiates UCI device object which is associated to device
> file node. UCI device object instantiates UCI channel object when device
> file node is opened. UCI channel object is used to manage MHI channels
> by calling MHI core APIs for read and write operations. MHI channels
> are started as part of device open(). MHI channels remain in start
> state until last release() is called on UCI device file node. Device
> file node is created with format

[...]

> +struct uci_chan {
> +       struct uci_dev *udev;
> +       wait_queue_head_t ul_wq;
> +
> +       /* ul channel lock to synchronize multiple writes */
> +       struct mutex write_lock;
> +
> +       wait_queue_head_t dl_wq;
> +
> +       /* dl channel lock to synchronize multiple reads */
> +       struct mutex read_lock;
> +
> +       /*
> +        * protects pending list in bh context, channel release, read and
> +        * poll
> +        */
> +       spinlock_t dl_pending_lock;
> +
> +       struct list_head dl_pending;
> +       struct uci_buf *cur_buf;
> +       size_t dl_size;
> +       struct kref ref_count;
> +};

[...]

> + * struct uci_dev - MHI UCI device
> + * @minor: UCI device node minor number
> + * @mhi_dev: associated mhi device object
> + * @uchan: UCI uplink and downlink channel object
> + * @mtu: max TRE buffer length
> + * @enabled: Flag to track the state of the UCI device
> + * @lock: mutex lock to manage uchan object
> + * @ref_count: uci_dev reference count
> + */
> +struct uci_dev {
> +       unsigned int minor;
> +       struct mhi_device *mhi_dev;
> +       struct uci_chan *uchan;

Why a pointer to uci_chan and not just plainly integrating the
structure here, AFAIU uci_chan describes the channels and is just a
subpart of uci_dev. That would reduce the number of dynamic
allocations you manage and the extra kref. do you even need a separate
structure for this?

[...]

> +static int mhi_uci_dev_start_chan(struct uci_dev *udev)
> +{
> +       int ret = 0;
> +       struct uci_chan *uchan;
> +
> +       mutex_lock(&udev->lock);
> +       if (!udev->uchan || !kref_get_unless_zero(&udev->uchan->ref_count)) {

This test is suspicious,  kref_get_unless_zero should be enough to test, right?

if (kref_get_unless_zero(&udev->ref))
    goto skip_init;

> +               uchan = kzalloc(sizeof(*uchan), GFP_KERNEL);
> +               if (!uchan) {
> +                       ret = -ENOMEM;
> +                       goto error_chan_start;
> +               }
> +
> +               udev->uchan = uchan;
> +               uchan->udev = udev;
> +               init_waitqueue_head(&uchan->ul_wq);
> +               init_waitqueue_head(&uchan->dl_wq);
> +               mutex_init(&uchan->write_lock);
> +               mutex_init(&uchan->read_lock);
> +               spin_lock_init(&uchan->dl_pending_lock);
> +               INIT_LIST_HEAD(&uchan->dl_pending);
> +
> +               ret = mhi_prepare_for_transfer(udev->mhi_dev);
> +               if (ret) {
> +                       dev_err(&udev->mhi_dev->dev, "Error starting transfer channels\n");
> +                       goto error_chan_cleanup;
> +               }
> +
> +               ret = mhi_queue_inbound(udev);
> +               if (ret)
> +                       goto error_chan_cleanup;
> +
> +               kref_init(&uchan->ref_count);
> +       }
> +
> +       mutex_unlock(&udev->lock);
> +
> +       return 0;
> +
> +error_chan_cleanup:
> +       mhi_uci_dev_chan_release(&uchan->ref_count);
> +error_chan_start:
> +       mutex_unlock(&udev->lock);
> +       return ret;
> +}

Regards,
Loic

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Hemant Kumar]

Hi Mani,

On 11/27/20 10:11 PM, Manivannan Sadhasivam wrote:
> Hi Hemant,
> 
> On Fri, Nov 27, 2020 at 07:26:06PM -0800, Hemant Kumar wrote:
>> This MHI client driver allows userspace clients to transfer
>> raw data between MHI device and host using standard file operations.
>> Driver instantiates UCI device object which is associated to device
>> file node. UCI device object instantiates UCI channel object when device
>> file node is opened. UCI channel object is used to manage MHI channels
>> by calling MHI core APIs for read and write operations. MHI channels
>> are started as part of device open(). MHI channels remain in start
>> state until last release() is called on UCI device file node. Device
>> file node is created with format
>>
>> /dev/mhi_<mhi_device_name>
>>
>> Currently it supports QMI channel.
>>
> 
> Thanks for the update. This patch looks good to me. But as I'm going to
> apply Loic's "bus: mhi: core: Indexed MHI controller name" patch, you
> need to update the documentation accordingly.

This is what i added in documentation on v13


Device file node is created with format:-

/dev/mhi_<mhi_device_name>

mhi_device_name includes mhi controller name and the name of the MHI
channel being used by MHI client in userspace to send or receive data
using MHI protocol.

​
Loic's patch is going to update the controller name as indexed 
controller name, which goes fine with or without his change going first.

For example:   With Loic's change name of device node would be 
/dev/mhi_mhi0_QMI

Without Loic's change it would be

/dev/mhi_0000:00:01.2_QMI

Please let me know if i am missing something.

Thanks,
Hemant

> 
>> Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
> 
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> Thanks,
> Mani
> 
[..]
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Hemant Kumar]

Hi Loic,

On 11/30/20 10:22 AM, Loic Poulain wrote:
> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>
>> This MHI client driver allows userspace clients to transfer
>> raw data between MHI device and host using standard file operations.
>> Driver instantiates UCI device object which is associated to device
>> file node. UCI device object instantiates UCI channel object when device
>> file node is opened. UCI channel object is used to manage MHI channels
>> by calling MHI core APIs for read and write operations. MHI channels
>> are started as part of device open(). MHI channels remain in start
>> state until last release() is called on UCI device file node. Device
>> file node is created with format
> 
> [...]
> 
>> +struct uci_chan {
>> +       struct uci_dev *udev;
>> +       wait_queue_head_t ul_wq;
>> +
>> +       /* ul channel lock to synchronize multiple writes */
>> +       struct mutex write_lock;
>> +
>> +       wait_queue_head_t dl_wq;
>> +
>> +       /* dl channel lock to synchronize multiple reads */
>> +       struct mutex read_lock;
>> +
>> +       /*
>> +        * protects pending list in bh context, channel release, read and
>> +        * poll
>> +        */
>> +       spinlock_t dl_pending_lock;
>> +
>> +       struct list_head dl_pending;
>> +       struct uci_buf *cur_buf;
>> +       size_t dl_size;
>> +       struct kref ref_count;
>> +};
> 
> [...]
> 
>> + * struct uci_dev - MHI UCI device
>> + * @minor: UCI device node minor number
>> + * @mhi_dev: associated mhi device object
>> + * @uchan: UCI uplink and downlink channel object
>> + * @mtu: max TRE buffer length
>> + * @enabled: Flag to track the state of the UCI device
>> + * @lock: mutex lock to manage uchan object
>> + * @ref_count: uci_dev reference count
>> + */
>> +struct uci_dev {
>> +       unsigned int minor;
>> +       struct mhi_device *mhi_dev;
>> +       struct uci_chan *uchan;
> 
> Why a pointer to uci_chan and not just plainly integrating the
> structure here, AFAIU uci_chan describes the channels and is just a
> subpart of uci_dev. That would reduce the number of dynamic
> allocations you manage and the extra kref. do you even need a separate
> structure for this?

This goes back to one of my patch versions i tried to address concern 
from Greg. Since we need to ref count the channel as well as the uci 
device i decoupled the two objects and used two reference counts for two 
different objects.

Copying from V7 patch history

V7:
- Decoupled uci device and uci channel objects. uci device is
   associated with device file node. uci channel is associated
   with MHI channels. uci device refers to uci channel to perform
   MHI channel operations for device file operations like read()
   and write(). uci device increments its reference count for
   every open(). uci device calls mhi_uci_dev_start_chan() to start
   the MHI channel. uci channel object is tracking number of times
   MHI channel is referred. This allows to keep the MHI channel in
   start state until last release() is called. After that uci channel
   reference count goes to 0 and uci channel clean up is performed
   which stops the MHI channel. After the last call to release() if
   driver is removed uci reference count becomes 0 and uci object is
   cleaned up.

> 
> [...]
> 
>> +static int mhi_uci_dev_start_chan(struct uci_dev *udev)
>> +{
>> +       int ret = 0;
>> +       struct uci_chan *uchan;
>> +
>> +       mutex_lock(&udev->lock);
>> +       if (!udev->uchan || !kref_get_unless_zero(&udev->uchan->ref_count)) {
> 
> This test is suspicious,  kref_get_unless_zero should be enough to test, right?
kref_get_unless_zero is de-referencing uchan->ref_count for the first 
open uchan is set to NULL, for that NULL check is added for uchan.
> 
> if (kref_get_unless_zero(&udev->ref))
>      goto skip_init;
> 
>> +               uchan = kzalloc(sizeof(*uchan), GFP_KERNEL);
>> +               if (!uchan) {
>> +                       ret = -ENOMEM;
>> +                       goto error_chan_start;
>> +               }
>> +
>> +               udev->uchan = uchan;
>> +               uchan->udev = udev;
>> +               init_waitqueue_head(&uchan->ul_wq);
>> +               init_waitqueue_head(&uchan->dl_wq);
>> +               mutex_init(&uchan->write_lock);
>> +               mutex_init(&uchan->read_lock);
>> +               spin_lock_init(&uchan->dl_pending_lock);
>> +               INIT_LIST_HEAD(&uchan->dl_pending);
>> +
>> +               ret = mhi_prepare_for_transfer(udev->mhi_dev);
>> +               if (ret) {
>> +                       dev_err(&udev->mhi_dev->dev, "Error starting transfer channels\n");
>> +                       goto error_chan_cleanup;
>> +               }
>> +
>> +               ret = mhi_queue_inbound(udev);
>> +               if (ret)
>> +                       goto error_chan_cleanup;
>> +
>> +               kref_init(&uchan->ref_count);
>> +       }
>> +
>> +       mutex_unlock(&udev->lock);
>> +
>> +       return 0;
>> +
>> +error_chan_cleanup:
>> +       mhi_uci_dev_chan_release(&uchan->ref_count);
>> +error_chan_start:
>> +       mutex_unlock(&udev->lock);
>> +       return ret;
>> +}
> 
> Regards,
> Loic
> 

Thanks,
Hemant
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Loic Poulain]

On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
>
> Hi Loic,
>
> On 11/30/20 10:22 AM, Loic Poulain wrote:
> > On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
> >>
> >> This MHI client driver allows userspace clients to transfer
> >> raw data between MHI device and host using standard file operations.
> >> Driver instantiates UCI device object which is associated to device
> >> file node. UCI device object instantiates UCI channel object when device
> >> file node is opened. UCI channel object is used to manage MHI channels
> >> by calling MHI core APIs for read and write operations. MHI channels
> >> are started as part of device open(). MHI channels remain in start
> >> state until last release() is called on UCI device file node. Device
> >> file node is created with format
> >
> > [...]
> >
> >> +struct uci_chan {
> >> +       struct uci_dev *udev;
> >> +       wait_queue_head_t ul_wq;
> >> +
> >> +       /* ul channel lock to synchronize multiple writes */
> >> +       struct mutex write_lock;
> >> +
> >> +       wait_queue_head_t dl_wq;
> >> +
> >> +       /* dl channel lock to synchronize multiple reads */
> >> +       struct mutex read_lock;
> >> +
> >> +       /*
> >> +        * protects pending list in bh context, channel release, read and
> >> +        * poll
> >> +        */
> >> +       spinlock_t dl_pending_lock;
> >> +
> >> +       struct list_head dl_pending;
> >> +       struct uci_buf *cur_buf;
> >> +       size_t dl_size;
> >> +       struct kref ref_count;
> >> +};
> >
> > [...]
> >
> >> + * struct uci_dev - MHI UCI device
> >> + * @minor: UCI device node minor number
> >> + * @mhi_dev: associated mhi device object
> >> + * @uchan: UCI uplink and downlink channel object
> >> + * @mtu: max TRE buffer length
> >> + * @enabled: Flag to track the state of the UCI device
> >> + * @lock: mutex lock to manage uchan object
> >> + * @ref_count: uci_dev reference count
> >> + */
> >> +struct uci_dev {
> >> +       unsigned int minor;
> >> +       struct mhi_device *mhi_dev;
> >> +       struct uci_chan *uchan;
> >
> > Why a pointer to uci_chan and not just plainly integrating the
> > structure here, AFAIU uci_chan describes the channels and is just a
> > subpart of uci_dev. That would reduce the number of dynamic
> > allocations you manage and the extra kref. do you even need a separate
> > structure for this?
>
> This goes back to one of my patch versions i tried to address concern
> from Greg. Since we need to ref count the channel as well as the uci
> device i decoupled the two objects and used two reference counts for two
> different objects.

What Greg complained about is the two kref in the same structure and
that you were using kref as an open() counter. But splitting your
struct in two in order to keep the two kref does not make the much
code better (and simpler). I'm still a bit puzzled about the driver
complexity, it's supposed to be just a passthrough interface to MHI
after all.

I would suggest several changes, that IMHO would simplify reviewing:
- Use only one structure representing the 'uci' context (uci_dev)
- Keep the read path simple (mhi_uci_read), do no use an intermediate
cur_buf pointer, only dequeue the buffer when it is fully consumed.
- As I commented before, take care of the dl_pending list access
concurrency, even in wait_event.
- You don't need to count the number of open() calls, AFAIK,
mhi_prepare_for_transfer() simply fails if channels are already
started...

For testing purpose, I've implemented those changes on my side (based
on your driver):
https://git.linaro.org/landing-teams/working/telit/linux.git/commit/?h=mhi_uci_test&id=45ff60703cc26913061a26260e39cf3ab3e57c2b

Feel free to pick (or not), I'm not going to 'block' that series if
others are fine with the current implementation.

Anyway, I've tested your series and it works on my side with
libqmi/qmicli (controlling SDX55 modem):
Tested-by: Loic Poulain <loic.poulain@linaro.org>

Regards,
Loic

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Jeffrey Hugo]

On 12/1/2020 10:36 AM, Loic Poulain wrote:
> On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>
>> Hi Loic,
>>
>> On 11/30/20 10:22 AM, Loic Poulain wrote:
>>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>>>
>>>> This MHI client driver allows userspace clients to transfer
>>>> raw data between MHI device and host using standard file operations.
>>>> Driver instantiates UCI device object which is associated to device
>>>> file node. UCI device object instantiates UCI channel object when device
>>>> file node is opened. UCI channel object is used to manage MHI channels
>>>> by calling MHI core APIs for read and write operations. MHI channels
>>>> are started as part of device open(). MHI channels remain in start
>>>> state until last release() is called on UCI device file node. Device
>>>> file node is created with format
>>>
>>> [...]
>>>
>>>> +struct uci_chan {
>>>> +       struct uci_dev *udev;
>>>> +       wait_queue_head_t ul_wq;
>>>> +
>>>> +       /* ul channel lock to synchronize multiple writes */
>>>> +       struct mutex write_lock;
>>>> +
>>>> +       wait_queue_head_t dl_wq;
>>>> +
>>>> +       /* dl channel lock to synchronize multiple reads */
>>>> +       struct mutex read_lock;
>>>> +
>>>> +       /*
>>>> +        * protects pending list in bh context, channel release, read and
>>>> +        * poll
>>>> +        */
>>>> +       spinlock_t dl_pending_lock;
>>>> +
>>>> +       struct list_head dl_pending;
>>>> +       struct uci_buf *cur_buf;
>>>> +       size_t dl_size;
>>>> +       struct kref ref_count;
>>>> +};
>>>
>>> [...]
>>>
>>>> + * struct uci_dev - MHI UCI device
>>>> + * @minor: UCI device node minor number
>>>> + * @mhi_dev: associated mhi device object
>>>> + * @uchan: UCI uplink and downlink channel object
>>>> + * @mtu: max TRE buffer length
>>>> + * @enabled: Flag to track the state of the UCI device
>>>> + * @lock: mutex lock to manage uchan object
>>>> + * @ref_count: uci_dev reference count
>>>> + */
>>>> +struct uci_dev {
>>>> +       unsigned int minor;
>>>> +       struct mhi_device *mhi_dev;
>>>> +       struct uci_chan *uchan;
>>>
>>> Why a pointer to uci_chan and not just plainly integrating the
>>> structure here, AFAIU uci_chan describes the channels and is just a
>>> subpart of uci_dev. That would reduce the number of dynamic
>>> allocations you manage and the extra kref. do you even need a separate
>>> structure for this?
>>
>> This goes back to one of my patch versions i tried to address concern
>> from Greg. Since we need to ref count the channel as well as the uci
>> device i decoupled the two objects and used two reference counts for two
>> different objects.
> 
> What Greg complained about is the two kref in the same structure and
> that you were using kref as an open() counter. But splitting your
> struct in two in order to keep the two kref does not make the much
> code better (and simpler). I'm still a bit puzzled about the driver
> complexity, it's supposed to be just a passthrough interface to MHI
> after all.
> 
> I would suggest several changes, that IMHO would simplify reviewing:
> - Use only one structure representing the 'uci' context (uci_dev)
> - Keep the read path simple (mhi_uci_read), do no use an intermediate
> cur_buf pointer, only dequeue the buffer when it is fully consumed.
> - As I commented before, take care of the dl_pending list access
> concurrency, even in wait_event.
> - You don't need to count the number of open() calls, AFAIK,
> mhi_prepare_for_transfer() simply fails if channels are already
> started...

Unless I missed something, you seem to have ignored the root issue that 
Hemant needs to solve, which is when to call 
mhi_unprepare_for_transfer().  You can't just call that when close() is 
called because there might be multiple users, and each one is going to 
trigger a close(), so you need to know how many close() instances to 
expect, and only call mhi_unprepare_for_transfer() for the last one.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Jeffrey Hugo]

On 12/1/2020 10:52 AM, Loic Poulain wrote:
> On Tue, 1 Dec 2020 at 18:37, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>
>> On 12/1/2020 10:36 AM, Loic Poulain wrote:
>>> On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>>>
>>>> Hi Loic,
>>>>
>>>> On 11/30/20 10:22 AM, Loic Poulain wrote:
>>>>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>>>>>
>>>>>> This MHI client driver allows userspace clients to transfer
>>>>>> raw data between MHI device and host using standard file operations.
>>>>>> Driver instantiates UCI device object which is associated to device
>>>>>> file node. UCI device object instantiates UCI channel object when device
>>>>>> file node is opened. UCI channel object is used to manage MHI channels
>>>>>> by calling MHI core APIs for read and write operations. MHI channels
>>>>>> are started as part of device open(). MHI channels remain in start
>>>>>> state until last release() is called on UCI device file node. Device
>>>>>> file node is created with format
>>>>>
>>>>> [...]
>>>>>
>>>>>> +struct uci_chan {
>>>>>> +       struct uci_dev *udev;
>>>>>> +       wait_queue_head_t ul_wq;
>>>>>> +
>>>>>> +       /* ul channel lock to synchronize multiple writes */
>>>>>> +       struct mutex write_lock;
>>>>>> +
>>>>>> +       wait_queue_head_t dl_wq;
>>>>>> +
>>>>>> +       /* dl channel lock to synchronize multiple reads */
>>>>>> +       struct mutex read_lock;
>>>>>> +
>>>>>> +       /*
>>>>>> +        * protects pending list in bh context, channel release, read and
>>>>>> +        * poll
>>>>>> +        */
>>>>>> +       spinlock_t dl_pending_lock;
>>>>>> +
>>>>>> +       struct list_head dl_pending;
>>>>>> +       struct uci_buf *cur_buf;
>>>>>> +       size_t dl_size;
>>>>>> +       struct kref ref_count;
>>>>>> +};
>>>>>
>>>>> [...]
>>>>>
>>>>>> + * struct uci_dev - MHI UCI device
>>>>>> + * @minor: UCI device node minor number
>>>>>> + * @mhi_dev: associated mhi device object
>>>>>> + * @uchan: UCI uplink and downlink channel object
>>>>>> + * @mtu: max TRE buffer length
>>>>>> + * @enabled: Flag to track the state of the UCI device
>>>>>> + * @lock: mutex lock to manage uchan object
>>>>>> + * @ref_count: uci_dev reference count
>>>>>> + */
>>>>>> +struct uci_dev {
>>>>>> +       unsigned int minor;
>>>>>> +       struct mhi_device *mhi_dev;
>>>>>> +       struct uci_chan *uchan;
>>>>>
>>>>> Why a pointer to uci_chan and not just plainly integrating the
>>>>> structure here, AFAIU uci_chan describes the channels and is just a
>>>>> subpart of uci_dev. That would reduce the number of dynamic
>>>>> allocations you manage and the extra kref. do you even need a separate
>>>>> structure for this?
>>>>
>>>> This goes back to one of my patch versions i tried to address concern
>>>> from Greg. Since we need to ref count the channel as well as the uci
>>>> device i decoupled the two objects and used two reference counts for two
>>>> different objects.
>>>
>>> What Greg complained about is the two kref in the same structure and
>>> that you were using kref as an open() counter. But splitting your
>>> struct in two in order to keep the two kref does not make the much
>>> code better (and simpler). I'm still a bit puzzled about the driver
>>> complexity, it's supposed to be just a passthrough interface to MHI
>>> after all.
>>>
>>> I would suggest several changes, that IMHO would simplify reviewing:
>>> - Use only one structure representing the 'uci' context (uci_dev)
>>> - Keep the read path simple (mhi_uci_read), do no use an intermediate
>>> cur_buf pointer, only dequeue the buffer when it is fully consumed.
>>> - As I commented before, take care of the dl_pending list access
>>> concurrency, even in wait_event.
>>> - You don't need to count the number of open() calls, AFAIK,
>>> mhi_prepare_for_transfer() simply fails if channels are already
>>> started...
>>
>> Unless I missed something, you seem to have ignored the root issue that
>> Hemant needs to solve, which is when to call
>> mhi_unprepare_for_transfer().  You can't just call that when close() is
>> called because there might be multiple users, and each one is going to
>> trigger a close(), so you need to know how many close() instances to
>> expect, and only call mhi_unprepare_for_transfer() for the last one.
> 
> That one part of his problem, yes, but if you unconditionally call
> mhi_prepare_for_transfer in open(), it should fail for subsequent
> users, and so only one user will successfully open the device.

I'm pretty sure that falls under "trying to prevent users from opening a 
device" which Greg gave a pretty strong NACK to.  So, that's not a solution.

So, the complete problem is -

N users need to be able to use the device (and by proxy, the channel) 
concurrently, but prepare needs to be called on the first user coming 
into the picture, and unprepare needs to be called on the last user 
exiting the picture.

Hemant has supported this usecase in every rev of this series I've 
looked at, but apparently every solution he has proposed to handle this 
has caused confusion.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Loic Poulain]

On Tue, 1 Dec 2020 at 18:37, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>
> On 12/1/2020 10:36 AM, Loic Poulain wrote:
> > On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
> >>
> >> Hi Loic,
> >>
> >> On 11/30/20 10:22 AM, Loic Poulain wrote:
> >>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
> >>>>
> >>>> This MHI client driver allows userspace clients to transfer
> >>>> raw data between MHI device and host using standard file operations.
> >>>> Driver instantiates UCI device object which is associated to device
> >>>> file node. UCI device object instantiates UCI channel object when device
> >>>> file node is opened. UCI channel object is used to manage MHI channels
> >>>> by calling MHI core APIs for read and write operations. MHI channels
> >>>> are started as part of device open(). MHI channels remain in start
> >>>> state until last release() is called on UCI device file node. Device
> >>>> file node is created with format
> >>>
> >>> [...]
> >>>
> >>>> +struct uci_chan {
> >>>> +       struct uci_dev *udev;
> >>>> +       wait_queue_head_t ul_wq;
> >>>> +
> >>>> +       /* ul channel lock to synchronize multiple writes */
> >>>> +       struct mutex write_lock;
> >>>> +
> >>>> +       wait_queue_head_t dl_wq;
> >>>> +
> >>>> +       /* dl channel lock to synchronize multiple reads */
> >>>> +       struct mutex read_lock;
> >>>> +
> >>>> +       /*
> >>>> +        * protects pending list in bh context, channel release, read and
> >>>> +        * poll
> >>>> +        */
> >>>> +       spinlock_t dl_pending_lock;
> >>>> +
> >>>> +       struct list_head dl_pending;
> >>>> +       struct uci_buf *cur_buf;
> >>>> +       size_t dl_size;
> >>>> +       struct kref ref_count;
> >>>> +};
> >>>
> >>> [...]
> >>>
> >>>> + * struct uci_dev - MHI UCI device
> >>>> + * @minor: UCI device node minor number
> >>>> + * @mhi_dev: associated mhi device object
> >>>> + * @uchan: UCI uplink and downlink channel object
> >>>> + * @mtu: max TRE buffer length
> >>>> + * @enabled: Flag to track the state of the UCI device
> >>>> + * @lock: mutex lock to manage uchan object
> >>>> + * @ref_count: uci_dev reference count
> >>>> + */
> >>>> +struct uci_dev {
> >>>> +       unsigned int minor;
> >>>> +       struct mhi_device *mhi_dev;
> >>>> +       struct uci_chan *uchan;
> >>>
> >>> Why a pointer to uci_chan and not just plainly integrating the
> >>> structure here, AFAIU uci_chan describes the channels and is just a
> >>> subpart of uci_dev. That would reduce the number of dynamic
> >>> allocations you manage and the extra kref. do you even need a separate
> >>> structure for this?
> >>
> >> This goes back to one of my patch versions i tried to address concern
> >> from Greg. Since we need to ref count the channel as well as the uci
> >> device i decoupled the two objects and used two reference counts for two
> >> different objects.
> >
> > What Greg complained about is the two kref in the same structure and
> > that you were using kref as an open() counter. But splitting your
> > struct in two in order to keep the two kref does not make the much
> > code better (and simpler). I'm still a bit puzzled about the driver
> > complexity, it's supposed to be just a passthrough interface to MHI
> > after all.
> >
> > I would suggest several changes, that IMHO would simplify reviewing:
> > - Use only one structure representing the 'uci' context (uci_dev)
> > - Keep the read path simple (mhi_uci_read), do no use an intermediate
> > cur_buf pointer, only dequeue the buffer when it is fully consumed.
> > - As I commented before, take care of the dl_pending list access
> > concurrency, even in wait_event.
> > - You don't need to count the number of open() calls, AFAIK,
> > mhi_prepare_for_transfer() simply fails if channels are already
> > started...
>
> Unless I missed something, you seem to have ignored the root issue that
> Hemant needs to solve, which is when to call
> mhi_unprepare_for_transfer().  You can't just call that when close() is
> called because there might be multiple users, and each one is going to
> trigger a close(), so you need to know how many close() instances to
> expect, and only call mhi_unprepare_for_transfer() for the last one.

That one part of his problem, yes, but if you unconditionally call
mhi_prepare_for_transfer in open(), it should fail for subsequent
users, and so only one user will successfully open the device.

Regards,
Loic

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Jeffrey Hugo]

On 12/1/2020 11:05 AM, Loic Poulain wrote:
> On Tue, 1 Dec 2020 at 18:51, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>
>> On 12/1/2020 10:52 AM, Loic Poulain wrote:
>>> On Tue, 1 Dec 2020 at 18:37, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>
>>>> On 12/1/2020 10:36 AM, Loic Poulain wrote:
>>>>> On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>>>>>
>>>>>> Hi Loic,
>>>>>>
>>>>>> On 11/30/20 10:22 AM, Loic Poulain wrote:
>>>>>>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
>>>>>>>>
>>>>>>>> This MHI client driver allows userspace clients to transfer
>>>>>>>> raw data between MHI device and host using standard file operations.
>>>>>>>> Driver instantiates UCI device object which is associated to device
>>>>>>>> file node. UCI device object instantiates UCI channel object when device
>>>>>>>> file node is opened. UCI channel object is used to manage MHI channels
>>>>>>>> by calling MHI core APIs for read and write operations. MHI channels
>>>>>>>> are started as part of device open(). MHI channels remain in start
>>>>>>>> state until last release() is called on UCI device file node. Device
>>>>>>>> file node is created with format
>>>>>>>
>>>>>>> [...]
>>>>>>>
>>>>>>>> +struct uci_chan {
>>>>>>>> +       struct uci_dev *udev;
>>>>>>>> +       wait_queue_head_t ul_wq;
>>>>>>>> +
>>>>>>>> +       /* ul channel lock to synchronize multiple writes */
>>>>>>>> +       struct mutex write_lock;
>>>>>>>> +
>>>>>>>> +       wait_queue_head_t dl_wq;
>>>>>>>> +
>>>>>>>> +       /* dl channel lock to synchronize multiple reads */
>>>>>>>> +       struct mutex read_lock;
>>>>>>>> +
>>>>>>>> +       /*
>>>>>>>> +        * protects pending list in bh context, channel release, read and
>>>>>>>> +        * poll
>>>>>>>> +        */
>>>>>>>> +       spinlock_t dl_pending_lock;
>>>>>>>> +
>>>>>>>> +       struct list_head dl_pending;
>>>>>>>> +       struct uci_buf *cur_buf;
>>>>>>>> +       size_t dl_size;
>>>>>>>> +       struct kref ref_count;
>>>>>>>> +};
>>>>>>>
>>>>>>> [...]
>>>>>>>
>>>>>>>> + * struct uci_dev - MHI UCI device
>>>>>>>> + * @minor: UCI device node minor number
>>>>>>>> + * @mhi_dev: associated mhi device object
>>>>>>>> + * @uchan: UCI uplink and downlink channel object
>>>>>>>> + * @mtu: max TRE buffer length
>>>>>>>> + * @enabled: Flag to track the state of the UCI device
>>>>>>>> + * @lock: mutex lock to manage uchan object
>>>>>>>> + * @ref_count: uci_dev reference count
>>>>>>>> + */
>>>>>>>> +struct uci_dev {
>>>>>>>> +       unsigned int minor;
>>>>>>>> +       struct mhi_device *mhi_dev;
>>>>>>>> +       struct uci_chan *uchan;
>>>>>>>
>>>>>>> Why a pointer to uci_chan and not just plainly integrating the
>>>>>>> structure here, AFAIU uci_chan describes the channels and is just a
>>>>>>> subpart of uci_dev. That would reduce the number of dynamic
>>>>>>> allocations you manage and the extra kref. do you even need a separate
>>>>>>> structure for this?
>>>>>>
>>>>>> This goes back to one of my patch versions i tried to address concern
>>>>>> from Greg. Since we need to ref count the channel as well as the uci
>>>>>> device i decoupled the two objects and used two reference counts for two
>>>>>> different objects.
>>>>>
>>>>> What Greg complained about is the two kref in the same structure and
>>>>> that you were using kref as an open() counter. But splitting your
>>>>> struct in two in order to keep the two kref does not make the much
>>>>> code better (and simpler). I'm still a bit puzzled about the driver
>>>>> complexity, it's supposed to be just a passthrough interface to MHI
>>>>> after all.
>>>>>
>>>>> I would suggest several changes, that IMHO would simplify reviewing:
>>>>> - Use only one structure representing the 'uci' context (uci_dev)
>>>>> - Keep the read path simple (mhi_uci_read), do no use an intermediate
>>>>> cur_buf pointer, only dequeue the buffer when it is fully consumed.
>>>>> - As I commented before, take care of the dl_pending list access
>>>>> concurrency, even in wait_event.
>>>>> - You don't need to count the number of open() calls, AFAIK,
>>>>> mhi_prepare_for_transfer() simply fails if channels are already
>>>>> started...
>>>>
>>>> Unless I missed something, you seem to have ignored the root issue that
>>>> Hemant needs to solve, which is when to call
>>>> mhi_unprepare_for_transfer().  You can't just call that when close() is
>>>> called because there might be multiple users, and each one is going to
>>>> trigger a close(), so you need to know how many close() instances to
>>>> expect, and only call mhi_unprepare_for_transfer() for the last one.
>>>
>>> That one part of his problem, yes, but if you unconditionally call
>>> mhi_prepare_for_transfer in open(), it should fail for subsequent
>>> users, and so only one user will successfully open the device.
>>
>> I'm pretty sure that falls under "trying to prevent users from opening a
>> device" which Greg gave a pretty strong NACK to.  So, that's not a solution.
> 
> That would deserve clarification since other drivers like
> virtio_console clearly prevent that (guest_connected).

Quoting Greg from the source - https://lkml.org/lkml/2020/9/17/873

"
I told you before, do not try to keep a device node from being opened
multiple times, as it will always fail (think about passing file handles
around between programs...)

If userspace wants to do this, it will do it.  If your driver can't
handle that, that's fine, userspace will learn not to do that.  But the
kernel can not prevent this from happening.
"

> 
>>
>> So, the complete problem is -
>>
>> N users need to be able to use the device (and by proxy, the channel)
>> concurrently, but prepare needs to be called on the first user coming
>> into the picture, and unprepare needs to be called on the last user
>> exiting the picture.
>>
>> Hemant has supported this usecase in every rev of this series I've
>> looked at, but apparently every solution he has proposed to handle this
>> has caused confusion.
> 
> Understood, but that can be done with a simple counter in that case.
> 
> Regards,
> Loic
> 


-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Loic Poulain]

On Tue, 1 Dec 2020 at 18:51, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>
> On 12/1/2020 10:52 AM, Loic Poulain wrote:
> > On Tue, 1 Dec 2020 at 18:37, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> >>
> >> On 12/1/2020 10:36 AM, Loic Poulain wrote:
> >>> On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> wrote:
> >>>>
> >>>> Hi Loic,
> >>>>
> >>>> On 11/30/20 10:22 AM, Loic Poulain wrote:
> >>>>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar <hemantk@codeaurora.org> wrote:
> >>>>>>
> >>>>>> This MHI client driver allows userspace clients to transfer
> >>>>>> raw data between MHI device and host using standard file operations.
> >>>>>> Driver instantiates UCI device object which is associated to device
> >>>>>> file node. UCI device object instantiates UCI channel object when device
> >>>>>> file node is opened. UCI channel object is used to manage MHI channels
> >>>>>> by calling MHI core APIs for read and write operations. MHI channels
> >>>>>> are started as part of device open(). MHI channels remain in start
> >>>>>> state until last release() is called on UCI device file node. Device
> >>>>>> file node is created with format
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>>> +struct uci_chan {
> >>>>>> +       struct uci_dev *udev;
> >>>>>> +       wait_queue_head_t ul_wq;
> >>>>>> +
> >>>>>> +       /* ul channel lock to synchronize multiple writes */
> >>>>>> +       struct mutex write_lock;
> >>>>>> +
> >>>>>> +       wait_queue_head_t dl_wq;
> >>>>>> +
> >>>>>> +       /* dl channel lock to synchronize multiple reads */
> >>>>>> +       struct mutex read_lock;
> >>>>>> +
> >>>>>> +       /*
> >>>>>> +        * protects pending list in bh context, channel release, read and
> >>>>>> +        * poll
> >>>>>> +        */
> >>>>>> +       spinlock_t dl_pending_lock;
> >>>>>> +
> >>>>>> +       struct list_head dl_pending;
> >>>>>> +       struct uci_buf *cur_buf;
> >>>>>> +       size_t dl_size;
> >>>>>> +       struct kref ref_count;
> >>>>>> +};
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>>> + * struct uci_dev - MHI UCI device
> >>>>>> + * @minor: UCI device node minor number
> >>>>>> + * @mhi_dev: associated mhi device object
> >>>>>> + * @uchan: UCI uplink and downlink channel object
> >>>>>> + * @mtu: max TRE buffer length
> >>>>>> + * @enabled: Flag to track the state of the UCI device
> >>>>>> + * @lock: mutex lock to manage uchan object
> >>>>>> + * @ref_count: uci_dev reference count
> >>>>>> + */
> >>>>>> +struct uci_dev {
> >>>>>> +       unsigned int minor;
> >>>>>> +       struct mhi_device *mhi_dev;
> >>>>>> +       struct uci_chan *uchan;
> >>>>>
> >>>>> Why a pointer to uci_chan and not just plainly integrating the
> >>>>> structure here, AFAIU uci_chan describes the channels and is just a
> >>>>> subpart of uci_dev. That would reduce the number of dynamic
> >>>>> allocations you manage and the extra kref. do you even need a separate
> >>>>> structure for this?
> >>>>
> >>>> This goes back to one of my patch versions i tried to address concern
> >>>> from Greg. Since we need to ref count the channel as well as the uci
> >>>> device i decoupled the two objects and used two reference counts for two
> >>>> different objects.
> >>>
> >>> What Greg complained about is the two kref in the same structure and
> >>> that you were using kref as an open() counter. But splitting your
> >>> struct in two in order to keep the two kref does not make the much
> >>> code better (and simpler). I'm still a bit puzzled about the driver
> >>> complexity, it's supposed to be just a passthrough interface to MHI
> >>> after all.
> >>>
> >>> I would suggest several changes, that IMHO would simplify reviewing:
> >>> - Use only one structure representing the 'uci' context (uci_dev)
> >>> - Keep the read path simple (mhi_uci_read), do no use an intermediate
> >>> cur_buf pointer, only dequeue the buffer when it is fully consumed.
> >>> - As I commented before, take care of the dl_pending list access
> >>> concurrency, even in wait_event.
> >>> - You don't need to count the number of open() calls, AFAIK,
> >>> mhi_prepare_for_transfer() simply fails if channels are already
> >>> started...
> >>
> >> Unless I missed something, you seem to have ignored the root issue that
> >> Hemant needs to solve, which is when to call
> >> mhi_unprepare_for_transfer().  You can't just call that when close() is
> >> called because there might be multiple users, and each one is going to
> >> trigger a close(), so you need to know how many close() instances to
> >> expect, and only call mhi_unprepare_for_transfer() for the last one.
> >
> > That one part of his problem, yes, but if you unconditionally call
> > mhi_prepare_for_transfer in open(), it should fail for subsequent
> > users, and so only one user will successfully open the device.
>
> I'm pretty sure that falls under "trying to prevent users from opening a
> device" which Greg gave a pretty strong NACK to.  So, that's not a solution.

That would deserve clarification since other drivers like
virtio_console clearly prevent that (guest_connected).

>
> So, the complete problem is -
>
> N users need to be able to use the device (and by proxy, the channel)
> concurrently, but prepare needs to be called on the first user coming
> into the picture, and unprepare needs to be called on the last user
> exiting the picture.
>
> Hemant has supported this usecase in every rev of this series I've
> looked at, but apparently every solution he has proposed to handle this
> has caused confusion.

Understood, but that can be done with a simple counter in that case.

Regards,
Loic

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jakub Kicinski]

On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
> This patch series adds support for UCI driver. UCI driver enables userspace
> clients to communicate to external MHI devices like modem and WLAN. UCI driver
> probe creates standard character device file nodes for userspace clients to
> perform open, read, write, poll and release file operations. These file
> operations call MHI core layer APIs to perform data transfer using MHI bus
> to communicate with MHI device. Patch is tested using arm64 based platform.

Wait, I thought this was for modems.

Why do WLAN devices need to communicate with user space?

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jeffrey Hugo]

On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
>> This patch series adds support for UCI driver. UCI driver enables userspace
>> clients to communicate to external MHI devices like modem and WLAN. UCI driver
>> probe creates standard character device file nodes for userspace clients to
>> perform open, read, write, poll and release file operations. These file
>> operations call MHI core layer APIs to perform data transfer using MHI bus
>> to communicate with MHI device. Patch is tested using arm64 based platform.
> 
> Wait, I thought this was for modems.
> 
> Why do WLAN devices need to communicate with user space?
> 

Why does it matter what type of device it is?  Are modems somehow unique 
in that they are the only type of device that userspace is allowed to 
interact with?

However, I'll bite.  Once such usecase would be QMI.  QMI is a generic 
messaging protocol, and is not strictly limited to the unique operations 
of a modem.

Another usecase would be Sahara - a custom file transfer protocol used 
for uploading firmware images, and downloading crashdumps.

Off the top of my head, this driver is useful for modems, wlan, and AI 
accelerators.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jakub Kicinski]

On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
> On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> > On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:  
> >> This patch series adds support for UCI driver. UCI driver enables userspace
> >> clients to communicate to external MHI devices like modem and WLAN. UCI driver
> >> probe creates standard character device file nodes for userspace clients to
> >> perform open, read, write, poll and release file operations. These file
> >> operations call MHI core layer APIs to perform data transfer using MHI bus
> >> to communicate with MHI device. Patch is tested using arm64 based platform.  
> > 
> > Wait, I thought this was for modems.
> > 
> > Why do WLAN devices need to communicate with user space?
> >   
> 
> Why does it matter what type of device it is?  Are modems somehow unique 
> in that they are the only type of device that userspace is allowed to 
> interact with?

Yes modems are traditionally highly weird and require some serial
device dance I don't even know about.

We have proper interfaces in Linux for configuring WiFi which work
across vendors. Having char device access to WiFi would be a step 
back.

> However, I'll bite.  Once such usecase would be QMI.  QMI is a generic 
> messaging protocol, and is not strictly limited to the unique operations 
> of a modem.
> 
> Another usecase would be Sahara - a custom file transfer protocol used 
> for uploading firmware images, and downloading crashdumps.

Thanks, I was asking for use cases, not which proprietary vendor
protocol you can implement over it.

None of the use cases you mention here should require a direct FW -
user space backdoor for WLAN.

> Off the top of my head, this driver is useful for modems, wlan, and AI 
> accelerators.

And other Qualcomm products are available as well :/

Kernel is supposed to create abstract interfaces for user space to
utilize. I will never understand why kernel is expected to be in
business of shipping this sort of vendor backdoors :/

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jeffrey Hugo]

On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
> On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
>> On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
>>> On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
>>>> This patch series adds support for UCI driver. UCI driver enables userspace
>>>> clients to communicate to external MHI devices like modem and WLAN. UCI driver
>>>> probe creates standard character device file nodes for userspace clients to
>>>> perform open, read, write, poll and release file operations. These file
>>>> operations call MHI core layer APIs to perform data transfer using MHI bus
>>>> to communicate with MHI device. Patch is tested using arm64 based platform.
>>>
>>> Wait, I thought this was for modems.
>>>
>>> Why do WLAN devices need to communicate with user space?
>>>    
>>
>> Why does it matter what type of device it is?  Are modems somehow unique
>> in that they are the only type of device that userspace is allowed to
>> interact with?
> 
> Yes modems are traditionally highly weird and require some serial
> device dance I don't even know about.
> 
> We have proper interfaces in Linux for configuring WiFi which work
> across vendors. Having char device access to WiFi would be a step
> back.

So a WLAN device is only ever allowed to do Wi-Fi?  It can't also have 
GPS functionality for example?

> 
>> However, I'll bite.  Once such usecase would be QMI.  QMI is a generic
>> messaging protocol, and is not strictly limited to the unique operations
>> of a modem.
>>
>> Another usecase would be Sahara - a custom file transfer protocol used
>> for uploading firmware images, and downloading crashdumps.
> 
> Thanks, I was asking for use cases, not which proprietary vendor
> protocol you can implement over it.
> 
> None of the use cases you mention here should require a direct FW -
> user space backdoor for WLAN.

Uploading runtime firmware, with variations based on the runtime mode. 
Flashing the onboard flash based on cryptographic keys.  Accessing 
configuration data.  Accessing device logs.  Configuring device logs. 
Synchronizing the device time reference to Linux local or remote time 
sources.  Enabling debugging/performance hardware.  Getting software 
diagnostic events.  Configuring redundancy hardware per workload. 
Uploading new cryptographic keys.  Invalidating cryptographic keys. 
Uploading factory test data and running factory tests.

Need more?

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 4/4] bus: mhi: Add userspace client interface driver

[Hemant Kumar]

Hi Loic,

On 12/1/20 10:04 AM, Jeffrey Hugo wrote:
> On 12/1/2020 11:05 AM, Loic Poulain wrote:
>> On Tue, 1 Dec 2020 at 18:51, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>
>>> On 12/1/2020 10:52 AM, Loic Poulain wrote:
>>>> On Tue, 1 Dec 2020 at 18:37, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>
>>>>> On 12/1/2020 10:36 AM, Loic Poulain wrote:
>>>>>> On Tue, 1 Dec 2020 at 02:16, Hemant Kumar <hemantk@codeaurora.org> 
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Loic,
>>>>>>>
>>>>>>> On 11/30/20 10:22 AM, Loic Poulain wrote:
>>>>>>>> On Sat, 28 Nov 2020 at 04:26, Hemant Kumar 
>>>>>>>> <hemantk@codeaurora.org> wrote:
>>>>>>>>>
>>>>>>>>> This MHI client driver allows userspace clients to transfer
>>>>>>>>> raw data between MHI device and host using standard file 
>>>>>>>>> operations.
>>>>>>>>> Driver instantiates UCI device object which is associated to 
>>>>>>>>> device
>>>>>>>>> file node. UCI device object instantiates UCI channel object 
>>>>>>>>> when device
>>>>>>>>> file node is opened. UCI channel object is used to manage MHI 
>>>>>>>>> channels
>>>>>>>>> by calling MHI core APIs for read and write operations. MHI 
>>>>>>>>> channels
>>>>>>>>> are started as part of device open(). MHI channels remain in start
>>>>>>>>> state until last release() is called on UCI device file node. 
>>>>>>>>> Device
>>>>>>>>> file node is created with format
>>>>>>>>
>>>>>>>> [...]
>>>>>>>>
>>>>>>>>> +struct uci_chan {
>>>>>>>>> +       struct uci_dev *udev;
>>>>>>>>> +       wait_queue_head_t ul_wq;
>>>>>>>>> +
>>>>>>>>> +       /* ul channel lock to synchronize multiple writes */
>>>>>>>>> +       struct mutex write_lock;
>>>>>>>>> +
>>>>>>>>> +       wait_queue_head_t dl_wq;
>>>>>>>>> +
>>>>>>>>> +       /* dl channel lock to synchronize multiple reads */
>>>>>>>>> +       struct mutex read_lock;
>>>>>>>>> +
>>>>>>>>> +       /*
>>>>>>>>> +        * protects pending list in bh context, channel 
>>>>>>>>> release, read and
>>>>>>>>> +        * poll
>>>>>>>>> +        */
>>>>>>>>> +       spinlock_t dl_pending_lock;
>>>>>>>>> +
>>>>>>>>> +       struct list_head dl_pending;
>>>>>>>>> +       struct uci_buf *cur_buf;
>>>>>>>>> +       size_t dl_size;
>>>>>>>>> +       struct kref ref_count;
>>>>>>>>> +};
>>>>>>>>
>>>>>>>> [...]
>>>>>>>>
>>>>>>>>> + * struct uci_dev - MHI UCI device
>>>>>>>>> + * @minor: UCI device node minor number
>>>>>>>>> + * @mhi_dev: associated mhi device object
>>>>>>>>> + * @uchan: UCI uplink and downlink channel object
>>>>>>>>> + * @mtu: max TRE buffer length
>>>>>>>>> + * @enabled: Flag to track the state of the UCI device
>>>>>>>>> + * @lock: mutex lock to manage uchan object
>>>>>>>>> + * @ref_count: uci_dev reference count
>>>>>>>>> + */
>>>>>>>>> +struct uci_dev {
>>>>>>>>> +       unsigned int minor;
>>>>>>>>> +       struct mhi_device *mhi_dev;
>>>>>>>>> +       struct uci_chan *uchan;
>>>>>>>>
>>>>>>>> Why a pointer to uci_chan and not just plainly integrating the
>>>>>>>> structure here, AFAIU uci_chan describes the channels and is just a
>>>>>>>> subpart of uci_dev. That would reduce the number of dynamic
>>>>>>>> allocations you manage and the extra kref. do you even need a 
>>>>>>>> separate
>>>>>>>> structure for this?
>>>>>>>
>>>>>>> This goes back to one of my patch versions i tried to address 
>>>>>>> concern
>>>>>>> from Greg. Since we need to ref count the channel as well as the uci
>>>>>>> device i decoupled the two objects and used two reference counts 
>>>>>>> for two
>>>>>>> different objects.
>>>>>>
>>>>>> What Greg complained about is the two kref in the same structure and
>>>>>> that you were using kref as an open() counter. But splitting your
>>>>>> struct in two in order to keep the two kref does not make the much
>>>>>> code better (and simpler). I'm still a bit puzzled about the driver
>>>>>> complexity, it's supposed to be just a passthrough interface to MHI
>>>>>> after all.
>>>>>>
>>>>>> I would suggest several changes, that IMHO would simplify reviewing:
>>>>>> - Use only one structure representing the 'uci' context (uci_dev)
>>>>>> - Keep the read path simple (mhi_uci_read), do no use an intermediate
>>>>>> cur_buf pointer, only dequeue the buffer when it is fully consumed.
>>>>>> - As I commented before, take care of the dl_pending list access
>>>>>> concurrency, even in wait_event.
>>>>>> - You don't need to count the number of open() calls, AFAIK,
>>>>>> mhi_prepare_for_transfer() simply fails if channels are already
>>>>>> started...
>>>>>
>>>>> Unless I missed something, you seem to have ignored the root issue 
>>>>> that
>>>>> Hemant needs to solve, which is when to call
>>>>> mhi_unprepare_for_transfer().  You can't just call that when 
>>>>> close() is
>>>>> called because there might be multiple users, and each one is going to
>>>>> trigger a close(), so you need to know how many close() instances to
>>>>> expect, and only call mhi_unprepare_for_transfer() for the last one.
>>>>
>>>> That one part of his problem, yes, but if you unconditionally call
>>>> mhi_prepare_for_transfer in open(), it should fail for subsequent
>>>> users, and so only one user will successfully open the device.
>>>
>>> I'm pretty sure that falls under "trying to prevent users from opening a
>>> device" which Greg gave a pretty strong NACK to.  So, that's not a 
>>> solution.
>>
>> That would deserve clarification since other drivers like
>> virtio_console clearly prevent that (guest_connected).
> 
> Quoting Greg from the source - https://lkml.org/lkml/2020/9/17/873
> 
> "
> I told you before, do not try to keep a device node from being opened
> multiple times, as it will always fail (think about passing file handles
> around between programs...)
> 
> If userspace wants to do this, it will do it.  If your driver can't
> handle that, that's fine, userspace will learn not to do that.  But the
> kernel can not prevent this from happening.
> "
> 
>>
>>>
>>> So, the complete problem is -
>>>
>>> N users need to be able to use the device (and by proxy, the channel)
>>> concurrently, but prepare needs to be called on the first user coming
>>> into the picture, and unprepare needs to be called on the last user
>>> exiting the picture.
>>>
>>> Hemant has supported this usecase in every rev of this series I've
>>> looked at, but apparently every solution he has proposed to handle this
>>> has caused confusion.
>>
>> Understood, but that can be done with a simple counter in that case.
If you look at the old original patch set, it was using simple counter 
and Greg suggested to move away from that and use kref object, and then all
incremental changes done using single kref object udev to two separate 
objects udev and uchan (introduced in v7).

Based on every reviewer's comment i thought driver looked cleaner and 
moved away from complicated implementations when it was introduced 
initially. If every one still agrees to that we can move forward with 
its current form which includes valuable suggestions from all the 
reviewers.

Thanks for testing my Change on your setup Loic. Mani wants me to update 
the Documentation patch by modifying the QMI usage example. I am going 
to upload next revision by updating the documentation.

LOOPBACK test application and its inclusion to mhi device id table would 
be a separate patch series. Will share the LOOPBACK test application 
public git hub link in Documentation.

Thanks Jeff for providing the background of the discussions from the past.

Thanks,
Hemant
>>
>> Regards,
>> Loic
>>
> 
> 

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jakub Kicinski]

On Tue, 1 Dec 2020 13:48:36 -0700 Jeffrey Hugo wrote:
> On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
> > On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:  
> >> On 12/1/2020 12:29 PM, Jakub Kicinski wrote:  
> >>> On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:  
> >>>> This patch series adds support for UCI driver. UCI driver enables userspace
> >>>> clients to communicate to external MHI devices like modem and WLAN. UCI driver
> >>>> probe creates standard character device file nodes for userspace clients to
> >>>> perform open, read, write, poll and release file operations. These file
> >>>> operations call MHI core layer APIs to perform data transfer using MHI bus
> >>>> to communicate with MHI device. Patch is tested using arm64 based platform.  
> >>>
> >>> Wait, I thought this was for modems.
> >>>
> >>> Why do WLAN devices need to communicate with user space?
> >>>      
> >>
> >> Why does it matter what type of device it is?  Are modems somehow unique
> >> in that they are the only type of device that userspace is allowed to
> >> interact with?  
> > 
> > Yes modems are traditionally highly weird and require some serial
> > device dance I don't even know about.
> > 
> > We have proper interfaces in Linux for configuring WiFi which work
> > across vendors. Having char device access to WiFi would be a step
> > back.  
> 
> So a WLAN device is only ever allowed to do Wi-Fi?  It can't also have 
> GPS functionality for example?

No, but it's also not true that the only way to implement GPS is by
opening a full on command/packet interface between fat proprietary
firmware and custom user space (which may or may not be proprietary 
as well).

> >> However, I'll bite.  Once such usecase would be QMI.  QMI is a generic
> >> messaging protocol, and is not strictly limited to the unique operations
> >> of a modem.
> >>
> >> Another usecase would be Sahara - a custom file transfer protocol used
> >> for uploading firmware images, and downloading crashdumps.  
> > 
> > Thanks, I was asking for use cases, not which proprietary vendor
> > protocol you can implement over it.
> > 
> > None of the use cases you mention here should require a direct FW -
> > user space backdoor for WLAN.  
> 
> Uploading runtime firmware, with variations based on the runtime mode. 
> Flashing the onboard flash based on cryptographic keys.  Accessing 
> configuration data.  Accessing device logs.  Configuring device logs. 
> Synchronizing the device time reference to Linux local or remote time 
> sources.  Enabling debugging/performance hardware.  Getting software 
> diagnostic events.  Configuring redundancy hardware per workload. 
> Uploading new cryptographic keys.  Invalidating cryptographic keys. 
> Uploading factory test data and running factory tests.
> 
> Need more?

This conversation is going nowhere. Are you trying to say that creating
a common Linux API for those features is impossible and each vendor
should be allowed to add their own proprietary way?

This has been proven incorrect again and again, and Wi-Fi is a good
example.

You can do whatever you want for GPS etc. but don't come nowhere near
networking with this attitude please.

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Manivannan Sadhasivam]

On Tue, Dec 01, 2020 at 12:03:02PM -0800, Jakub Kicinski wrote:
> On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
> > On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> > > On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:  
> > >> This patch series adds support for UCI driver. UCI driver enables userspace
> > >> clients to communicate to external MHI devices like modem and WLAN. UCI driver
> > >> probe creates standard character device file nodes for userspace clients to
> > >> perform open, read, write, poll and release file operations. These file
> > >> operations call MHI core layer APIs to perform data transfer using MHI bus
> > >> to communicate with MHI device. Patch is tested using arm64 based platform.  
> > > 
> > > Wait, I thought this was for modems.
> > > 
> > > Why do WLAN devices need to communicate with user space?
> > >   
> > 
> > Why does it matter what type of device it is?  Are modems somehow unique 
> > in that they are the only type of device that userspace is allowed to 
> > interact with?
> 
> Yes modems are traditionally highly weird and require some serial
> device dance I don't even know about.
> 
> We have proper interfaces in Linux for configuring WiFi which work
> across vendors. Having char device access to WiFi would be a step 
> back.
> 

This is not for configuring the WiFi. This driver is mostly used for modems and
the AI accelerator Jeff is working on. But there might be a usecase for WLAN
devices as well to collect crash dumps and download fw (typical vendor ways) but
having those features are add-ons IMO.

So I think we should not be blocked by those usecases.

Thanks,
Mani

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Bjorn Andersson]

On Tue 01 Dec 13:29 CST 2020, Jakub Kicinski wrote:

> On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
> > This patch series adds support for UCI driver. UCI driver enables userspace
> > clients to communicate to external MHI devices like modem and WLAN. UCI driver
> > probe creates standard character device file nodes for userspace clients to
> > perform open, read, write, poll and release file operations. These file
> > operations call MHI core layer APIs to perform data transfer using MHI bus
> > to communicate with MHI device. Patch is tested using arm64 based platform.
> 
> Wait, I thought this was for modems.
> 

No, this allows exposing particular channels from any type of MHI
devices.

For modems there is a legacy control path that uses UCI. But data
traffic, (non-legacy) modem control signals and e.g the bearer of GPS
data uses in-kernel drivers that are already in place.

> Why do WLAN devices need to communicate with user space?

They normally don't, all WLAN operations are dealt with within the
kernel. The use case that comes to mind for UCI when it comes to WiFi
products is to avoid implementing the Qualcomm debug (diag) protocol in
the kernel.


As such I think saying that it can be used to communicate with modem or
WLAN devices is misleading. Because while it could be done, it is only
used for dealing with optional side-band services on such products - not
the actual WiFi and modem functionality.

Regards,
Bjorn

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Jeffrey Hugo]

On 12/1/2020 7:55 PM, Jakub Kicinski wrote:
> On Tue, 1 Dec 2020 13:48:36 -0700 Jeffrey Hugo wrote:
>> On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
>>> On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
>>>> On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
>>>>> On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
>>>>>> This patch series adds support for UCI driver. UCI driver enables userspace
>>>>>> clients to communicate to external MHI devices like modem and WLAN. UCI driver
>>>>>> probe creates standard character device file nodes for userspace clients to
>>>>>> perform open, read, write, poll and release file operations. These file
>>>>>> operations call MHI core layer APIs to perform data transfer using MHI bus
>>>>>> to communicate with MHI device. Patch is tested using arm64 based platform.
>>>>>
>>>>> Wait, I thought this was for modems.
>>>>>
>>>>> Why do WLAN devices need to communicate with user space?
>>>>>       
>>>>
>>>> Why does it matter what type of device it is?  Are modems somehow unique
>>>> in that they are the only type of device that userspace is allowed to
>>>> interact with?
>>>
>>> Yes modems are traditionally highly weird and require some serial
>>> device dance I don't even know about.
>>>
>>> We have proper interfaces in Linux for configuring WiFi which work
>>> across vendors. Having char device access to WiFi would be a step
>>> back.
>>
>> So a WLAN device is only ever allowed to do Wi-Fi?  It can't also have
>> GPS functionality for example?
> 
> No, but it's also not true that the only way to implement GPS is by
> opening a full on command/packet interface between fat proprietary
> firmware and custom user space (which may or may not be proprietary
> as well).

Funny, that exactly what the GPS "API" in the kernel is, although a bit 
limited to the specifics on the standardized GPS "sentences" and not 
covering implementation specific configuration.

> 
>>>> However, I'll bite.  Once such usecase would be QMI.  QMI is a generic
>>>> messaging protocol, and is not strictly limited to the unique operations
>>>> of a modem.
>>>>
>>>> Another usecase would be Sahara - a custom file transfer protocol used
>>>> for uploading firmware images, and downloading crashdumps.
>>>
>>> Thanks, I was asking for use cases, not which proprietary vendor
>>> protocol you can implement over it.
>>>
>>> None of the use cases you mention here should require a direct FW -
>>> user space backdoor for WLAN.
>>
>> Uploading runtime firmware, with variations based on the runtime mode.
>> Flashing the onboard flash based on cryptographic keys.  Accessing
>> configuration data.  Accessing device logs.  Configuring device logs.
>> Synchronizing the device time reference to Linux local or remote time
>> sources.  Enabling debugging/performance hardware.  Getting software
>> diagnostic events.  Configuring redundancy hardware per workload.
>> Uploading new cryptographic keys.  Invalidating cryptographic keys.
>> Uploading factory test data and running factory tests.
>>
>> Need more?
> 
> This conversation is going nowhere. Are you trying to say that creating
> a common Linux API for those features is impossible and each vendor
> should be allowed to add their own proprietary way?
> 
> This has been proven incorrect again and again, and Wi-Fi is a good
> example.
> 
> You can do whatever you want for GPS etc. but don't come nowhere near
> networking with this attitude please.
> 

No I'm saying (and Bjorn/Mani by the looks of things), that there is 
commonality in the core features - IP traffic, Wi-Fi, etc but then there 
are vendor specific things which are either things you don't actually 
want in the kernel, don't want the kernel doing, or have little 
commonality between vendors such that attempting to unify them gains you 
little to nothing.

Over in the networking space, I can see where standardization is plenty 
useful.

I can't speak for other vendors, but a "modem" or a "wlan" device from 
Qualcomm is not something that just provides one service.  They tend to 
provide dozens of different functionalities, some of those are 
"standardized" like wi-fi where common wi-fi interfaces are used. 
Others are unique to Qualcomm.

The point is "wlan device" is a superset of "wi-fi".  You seem to be 
equating them to be the same in a "shoot first, ask questions later" manner.

This series provides a way for userspace to talk to remote MHI "widgets" 
for usecases not covered elsewhere.  Those "widgets" just happen to 
commonly provide modem/wlan services, but ones that don't are not excluded.

Regarding not coming near networking, I'd like to remind you it was you 
that decided to come over here to the non-networking area and try to 
make this about networking.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Leon Romanovsky]

On Tue, Dec 01, 2020 at 09:59:53PM -0700, Jeffrey Hugo wrote:
> On 12/1/2020 7:55 PM, Jakub Kicinski wrote:
> > On Tue, 1 Dec 2020 13:48:36 -0700 Jeffrey Hugo wrote:
> > > On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
> > > > On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
> > > > > On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> > > > > > On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
> > > > > > > This patch series adds support for UCI driver. UCI driver enables userspace
> > > > > > > clients to communicate to external MHI devices like modem and WLAN. UCI driver
> > > > > > > probe creates standard character device file nodes for userspace clients to
> > > > > > > perform open, read, write, poll and release file operations. These file
> > > > > > > operations call MHI core layer APIs to perform data transfer using MHI bus
> > > > > > > to communicate with MHI device. Patch is tested using arm64 based platform.
> > > > > >
> > > > > > Wait, I thought this was for modems.
> > > > > >
> > > > > > Why do WLAN devices need to communicate with user space?
> > > > >
> > > > > Why does it matter what type of device it is?  Are modems somehow unique
> > > > > in that they are the only type of device that userspace is allowed to
> > > > > interact with?
> > > >
> > > > Yes modems are traditionally highly weird and require some serial
> > > > device dance I don't even know about.
> > > >
> > > > We have proper interfaces in Linux for configuring WiFi which work
> > > > across vendors. Having char device access to WiFi would be a step
> > > > back.
> > >
> > > So a WLAN device is only ever allowed to do Wi-Fi?  It can't also have
> > > GPS functionality for example?
> >
> > No, but it's also not true that the only way to implement GPS is by
> > opening a full on command/packet interface between fat proprietary
> > firmware and custom user space (which may or may not be proprietary
> > as well).
>
> Funny, that exactly what the GPS "API" in the kernel is, although a bit
> limited to the specifics on the standardized GPS "sentences" and not
> covering implementation specific configuration.
>
> >
> > > > > However, I'll bite.  Once such usecase would be QMI.  QMI is a generic
> > > > > messaging protocol, and is not strictly limited to the unique operations
> > > > > of a modem.
> > > > >
> > > > > Another usecase would be Sahara - a custom file transfer protocol used
> > > > > for uploading firmware images, and downloading crashdumps.
> > > >
> > > > Thanks, I was asking for use cases, not which proprietary vendor
> > > > protocol you can implement over it.
> > > >
> > > > None of the use cases you mention here should require a direct FW -
> > > > user space backdoor for WLAN.
> > >
> > > Uploading runtime firmware, with variations based on the runtime mode.
> > > Flashing the onboard flash based on cryptographic keys.  Accessing
> > > configuration data.  Accessing device logs.  Configuring device logs.
> > > Synchronizing the device time reference to Linux local or remote time
> > > sources.  Enabling debugging/performance hardware.  Getting software
> > > diagnostic events.  Configuring redundancy hardware per workload.
> > > Uploading new cryptographic keys.  Invalidating cryptographic keys.
> > > Uploading factory test data and running factory tests.
> > >
> > > Need more?
> >
> > This conversation is going nowhere. Are you trying to say that creating
> > a common Linux API for those features is impossible and each vendor
> > should be allowed to add their own proprietary way?
> >
> > This has been proven incorrect again and again, and Wi-Fi is a good
> > example.
> >
> > You can do whatever you want for GPS etc. but don't come nowhere near
> > networking with this attitude please.
> >
>
> No I'm saying (and Bjorn/Mani by the looks of things), that there is
> commonality in the core features - IP traffic, Wi-Fi, etc but then there are
> vendor specific things which are either things you don't actually want in
> the kernel, don't want the kernel doing, or have little commonality between
> vendors such that attempting to unify them gains you little to nothing.
>
> Over in the networking space, I can see where standardization is plenty
> useful.
>
> I can't speak for other vendors, but a "modem" or a "wlan" device from
> Qualcomm is not something that just provides one service.  They tend to
> provide dozens of different functionalities, some of those are
> "standardized" like wi-fi where common wi-fi interfaces are used. Others are
> unique to Qualcomm.
>
> The point is "wlan device" is a superset of "wi-fi".  You seem to be
> equating them to be the same in a "shoot first, ask questions later" manner.
>
> This series provides a way for userspace to talk to remote MHI "widgets" for
> usecases not covered elsewhere.  Those "widgets" just happen to commonly
> provide modem/wlan services, but ones that don't are not excluded.
>
> Regarding not coming near networking, I'd like to remind you it was you that
> decided to come over here to the non-networking area and try to make this
> about networking.

Like it or not, but Jakub is absolutely right with his claim that
providing user-visible interfaces without any standardization is proven
as wrong.

Thanks

>
> --
> Jeffrey Hugo
> Qualcomm Technologies, Inc. is a member of the
> Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Manivannan Sadhasivam]

On Sun, Dec 06, 2020 at 10:33:02AM +0200, Leon Romanovsky wrote:
> On Tue, Dec 01, 2020 at 09:59:53PM -0700, Jeffrey Hugo wrote:
> > On 12/1/2020 7:55 PM, Jakub Kicinski wrote:
> > > On Tue, 1 Dec 2020 13:48:36 -0700 Jeffrey Hugo wrote:
> > > > On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
> > > > > On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
> > > > > > On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> > > > > > > On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
> > > > > > > > This patch series adds support for UCI driver. UCI driver enables userspace
> > > > > > > > clients to communicate to external MHI devices like modem and WLAN. UCI driver
> > > > > > > > probe creates standard character device file nodes for userspace clients to
> > > > > > > > perform open, read, write, poll and release file operations. These file
> > > > > > > > operations call MHI core layer APIs to perform data transfer using MHI bus
> > > > > > > > to communicate with MHI device. Patch is tested using arm64 based platform.
> > > > > > >
> > > > > > > Wait, I thought this was for modems.
> > > > > > >

[...]

> Like it or not, but Jakub is absolutely right with his claim that
> providing user-visible interfaces without any standardization is proven
> as wrong.
> 

Everybody agrees with standardizing things but the problem is, the
standardization will only happen when more than one person implements the
same functionality.

The primary discussion is around the usage of chardev nodes for WLAN but
we made it clear that WLAN doesn't need this chardev node for working at
all. I agree that the commit message is a bit misleading and I hope that
Hemant will fix it in next revision.

Thanks,
Mani

> Thanks
> 
> >
> > --
> > Jeffrey Hugo
> > Qualcomm Technologies, Inc. is a member of the
> > Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [PATCH v13 0/4] userspace MHI client interface driver

[Leon Romanovsky]

On Tue, Dec 08, 2020 at 10:29:27PM +0530, Manivannan Sadhasivam wrote:
> On Sun, Dec 06, 2020 at 10:33:02AM +0200, Leon Romanovsky wrote:
> > On Tue, Dec 01, 2020 at 09:59:53PM -0700, Jeffrey Hugo wrote:
> > > On 12/1/2020 7:55 PM, Jakub Kicinski wrote:
> > > > On Tue, 1 Dec 2020 13:48:36 -0700 Jeffrey Hugo wrote:
> > > > > On 12/1/2020 1:03 PM, Jakub Kicinski wrote:
> > > > > > On Tue, 1 Dec 2020 12:40:50 -0700 Jeffrey Hugo wrote:
> > > > > > > On 12/1/2020 12:29 PM, Jakub Kicinski wrote:
> > > > > > > > On Fri, 27 Nov 2020 19:26:02 -0800 Hemant Kumar wrote:
> > > > > > > > > This patch series adds support for UCI driver. UCI driver enables userspace
> > > > > > > > > clients to communicate to external MHI devices like modem and WLAN. UCI driver
> > > > > > > > > probe creates standard character device file nodes for userspace clients to
> > > > > > > > > perform open, read, write, poll and release file operations. These file
> > > > > > > > > operations call MHI core layer APIs to perform data transfer using MHI bus
> > > > > > > > > to communicate with MHI device. Patch is tested using arm64 based platform.
> > > > > > > >
> > > > > > > > Wait, I thought this was for modems.
> > > > > > > >
>
> [...]
>
> > Like it or not, but Jakub is absolutely right with his claim that
> > providing user-visible interfaces without any standardization is proven
> > as wrong.
> >
>
> Everybody agrees with standardizing things but the problem is, the
> standardization will only happen when more than one person implements the
> same functionality.

From my experience in RDMA and netdev, I can't agree with both of your
statements. There are a lot of people who see standardization as a bad
thing. Also we are pushing even one person to make user visible interfaces
right from the beginning without relation to how wide it will be adopted
later.

Thanks