linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC 00/11] Intro to Hardware timestamping engine
@ 2021-06-25 23:55 Dipen Patel
  2021-06-25 23:55 ` [RFC 01/11] Documentation: Add HTE subsystem guide Dipen Patel
                   ` (11 more replies)
  0 siblings, 12 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

This patch series introduces new subsystem called hardware timestamping
engine (HTE). It offers functionality such as timestamping through hardware
means in realtime. The HTE subsystem centralizes HTE provider and consumers
where providers can register themselves with subsystem and the consumers can
request interested entity which could be lines, GPIO, signals or buses. The
HTE subsystem provides timestamp in nano seconds, having said that the provider
need to convert the timestamp if its not in that unit. There was upstream
discussion about the same at
https://lore.kernel.org/lkml/4c46726d-fa35-1a95-4295-bca37c8b6fe3@nvidia.com/

To summarize upstream discussion:
- It was heavily favoured by Linus and Kent to extend GPIOLIB and supporting
GPIO drivers to add HTE functionality and I agreed to experiment with it.
This patch series implements and extends GPIOLIB and GPIO tegra driver.
- Discussed possibility to add HTE provider as irqchip instead which
was argued against as HTE devices are not necessarily event emitting
devices.
- Discussed other possibility if HTE device can be added as posix clock
type like PTP clocks. That was also argues against since HTE devices
are not necessarily tightly coupled with hardware clock.

Typical HTE provider does following:
- Register itself with HTE subsystem
- Provide *request, *release, *enable, *disable timestamp callbacks and
optional get_clk_src_info callback to HTE subsystem.
- Provide optional xlate callback to the subsystem which can translate
consumer provided logical ids into actual ids of the entity, where entity here
is the provider dependent and could be GPIO, in chip lines or signals, buses
etc...This converted id will be used between HTE subsystem and the provider for
below bullet point.
- Push timestamps to the subsystem. This happens when HTE provider has
timestamp data available and willing to push it to HTE subsystem. The HTE
subsystem stores it into software buffer for the consumers.
- Unregister itself

Typical HTE consumer does following:
- Request interested entity it wishes to timestamp in realtime to the
subsystem. During this call HTE subsystem allocates software buffer to
store timestamps data.
- The subsystem does necessary communications with the provider to
complete the request, which includes translating logical id of the entity to
provider dependent physical/actual id and enabling hardware timestamping on
requested id.
- It can optionally specify callback during registration, this cb will
be called when provider pushes timestamps. Once notified through cb, the
consumer can call retrieve API to read the data from the software buffer.
If cb is not provided, the consumers can elect to call blocking version of
retrieve API.
- Manage pre allocated software buffer if needed. It includes changing buffer
length and watermark/threshold. The subsystem automatically sets watermark or
threshold at 1, consumers can later change it to any other value it wishes. The
main purpose for having threshold functionality is to notify consumer either
through callback if provided or unblock waiting consumer when threshold is
reached.
- Retrieve timestamp using various means provided by subsystem.
- Release entity and its resources.

HTE and GPIOLIB:
- For the HTE provider which can timestamp GPIO lines.
- For the GPIO consumers, either in kernel or userspace, The GPIOLIB and its
CDEV framework are extended as frontend to the HTE by introducing new APIs.
- Tegra194 AON GPIO controller has HTE support also known as GTE
(Generic Timestamping Engine). The tegra gpio driver is modified to accommodate
HTE functionality.

Dipen Patel (11):
  Documentation: Add HTE subsystem guide
  drivers: Add HTE subsystem
  hte: Add tegra194 HTE kernel provider
  dt-bindings: Add HTE bindings
  hte: Add Tegra194 IRQ HTE test driver
  gpiolib: Add HTE support
  gpio: tegra186: Add HTE in gpio-tegra186 driver
  gpiolib: cdev: Add hardware timestamp clock type
  tools: gpio: Add new hardware clock type
  hte: Add tegra GPIO HTE test driver
  MAINTAINERS: Added HTE Subsystem

 .../bindings/gpio/nvidia,tegra186-gpio.txt    |    7 +
 .../devicetree/bindings/hte/hte-consumer.yaml |   47 +
 .../devicetree/bindings/hte/hte.yaml          |   34 +
 .../bindings/hte/nvidia,tegra194-hte.yaml     |   83 +
 Documentation/hte/hte.rst                     |  198 +++
 Documentation/hte/index.rst                   |   21 +
 Documentation/hte/tegra194-hte.rst            |   65 +
 Documentation/index.rst                       |    1 +
 MAINTAINERS                                   |    8 +
 drivers/Kconfig                               |    2 +
 drivers/Makefile                              |    1 +
 drivers/gpio/gpio-tegra186.c                  |   78 +
 drivers/gpio/gpiolib-cdev.c                   |   65 +-
 drivers/gpio/gpiolib.c                        |   92 ++
 drivers/gpio/gpiolib.h                        |   11 +
 drivers/hte/Kconfig                           |   49 +
 drivers/hte/Makefile                          |    4 +
 drivers/hte/hte-tegra194-gpio-test.c          |  255 +++
 drivers/hte/hte-tegra194-irq-test.c           |  400 +++++
 drivers/hte/hte-tegra194.c                    |  554 +++++++
 drivers/hte/hte.c                             | 1368 +++++++++++++++++
 include/linux/gpio/consumer.h                 |   21 +-
 include/linux/gpio/driver.h                   |   13 +
 include/linux/hte.h                           |  278 ++++
 include/uapi/linux/gpio.h                     |    1 +
 tools/gpio/gpio-event-mon.c                   |    6 +-
 26 files changed, 3657 insertions(+), 5 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
 create mode 100644 Documentation/hte/hte.rst
 create mode 100644 Documentation/hte/index.rst
 create mode 100644 Documentation/hte/tegra194-hte.rst
 create mode 100644 drivers/hte/Kconfig
 create mode 100644 drivers/hte/Makefile
 create mode 100644 drivers/hte/hte-tegra194-gpio-test.c
 create mode 100644 drivers/hte/hte-tegra194-irq-test.c
 create mode 100644 drivers/hte/hte-tegra194.c
 create mode 100644 drivers/hte/hte.c
 create mode 100644 include/linux/hte.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 75+ messages in thread

* [RFC 01/11] Documentation: Add HTE subsystem guide
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-07-04 18:55   ` Jonathan Cameron
  2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
                   ` (10 subsequent siblings)
  11 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Adding hte document which can help understand various APIs implemented
in HTE framework for the HTE producers and the consumers.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 Documentation/hte/hte.rst | 198 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 198 insertions(+)
 create mode 100644 Documentation/hte/hte.rst

diff --git a/Documentation/hte/hte.rst b/Documentation/hte/hte.rst
new file mode 100644
index 000000000000..11744dbc6d16
--- /dev/null
+++ b/Documentation/hte/hte.rst
@@ -0,0 +1,198 @@
+============================================
+The Linux Hardware Timestamping Engine (HTE)
+============================================
+
+:Author: Dipen Patel
+
+Introduction
+------------
+
+The certain devices have the built in hardware timestamping engine which can
+monitor sets of system signals, lines, buses etc... in realtime for the state
+change; upon detecting the change it can automatically store the timestamp at
+the moment of occurrence. Such functionality may help achieve better accuracy
+in obtaining timestamp than using software counterparts i.e. ktime and friends.
+
+This document describes the API that can be used by hardware timestamping
+engine provider and consumer drivers that want to use the hardware timestamping
+engine (HTE) framework.
+
+The HTE framework APIs for the providers
+----------------------------------------
+Each driver must #include <linux/hte.h>. The ``linux/hte.h`` declares the
+following functions for the provider:
+
+.. c:function:: int hte_register_chip( struct hte_chip *chip )
+		int hte_unregister_chip( struct hte_chip *chip )
+
+	The provider uses these APIs to un/register itself with HTE framework.
+
+.. c:function:: int hte_push_ts_ns_atomic( const struct hte_chip *chip, u32 xlated_id, struct hte_ts_data *data, size_t n )
+
+	The provider pushes timestamp data in nano seconds unit using this API.
+
+The detail about parameters and API usage are described in each functions
+definitions in ``drivers/hte/hte.c`` file.
+
+The HTE framework APIs for the consumers
+----------------------------------------
+The consumers use following APIs to control the line for the timestamp:
+
+.. c:function:: int hte_release_ts( struct hte_ts_desc *desc )
+		int devm_hte_release_ts( struct device *dev, struct hte_ts_desc *desc )
+
+	The consumer uses API to release specified desc from timestamping.
+	The API frees resources associated with the desc and disables the
+	timestamping on it. The later is managed version of the same API.
+
+.. c:function:: struct hte_ts_desc *of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
+		struct hte_ts_desc *devm_of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
+
+	The consumers can use above request APIs to request real timestamp
+	capability on specified entity. The later is resource managed version
+	of the of_hte_request_ts API. Both the APIs expect consumer to follow
+	device tree bindings for the HTE consumer. The details about binding
+	is in ``Documentation/devicetree/bindings/hte/hte-consumer.yaml``.
+
+.. c:function:: struct hte_ts_desc *hte_req_ts_by_dt_node( struct device_node *of_node, unsigned int id, void (*cb)(enum hte_notify n) )
+
+	The consumer can request timestamping directly specifying provider
+	device tree node.
+
+.. c:function:: int hte_enable_ts( struct hte_ts_desc *desc )
+.. c:function:: int hte_disable_ts( struct hte_ts_desc *desc )
+
+	The consumer can enable/disable timestamping on given desc.
+
+.. c:function:: int hte_retrieve_ts_ns( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
+		int hte_retrieve_ts_ns_wait( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
+
+	The consumer uses above two API versions to get/retrieve timestamp data
+	for the given desc. The later is blocking version.
+
+.. c:function:: hte_get_clk_src_info(const struct hte_line_desc *desc, struct hte_clk_info *ci)
+
+	The consumer retrieves clock source information that provider uses to
+	timestamp entity in the structure hte_clk_info. This information
+	specifies clock rate in HZ and clock.
+
+The details on struct hte_clk_info
+-----------------------------------
+This structure presents detail of the hardware clock that provider uses for
+realtime timestamping purposes. The consumer can use hte_get_clk_src_info API
+to get the information in hte_clk_info structure. It has hz and type parameters
+where hz represents clock rate in HZ and type is clock type of clockid_t and
+of CLOCK_* family (for example, CLOCK_MONOTONIC).
+
+The consumers calling of_hte_request_ts or hte_req_ts_by_dt_node APIs with
+cb parameter set, usually will call hte_retrieve_ts (non blocking
+version) after being notified by the callbacks from HTE subsystem. The
+consumers calling those requests APIs with cb parameter NULL, usually will call
+hte_retrieve_ts_wait API.
+
+The HTE subsystem provides software buffer per requested id/entity to store
+timestamp data (struct hte_ts_data type). The consumers can manage the buffer.
+It also provides buffer watermark which can notify (if cb parameter is provided
+during request API call) consumer or unblock consumers calling
+hte_retrieve_ts_wait API. The following APIs are used to manipulate the
+software buffer:
+
+.. c:function:: int hte_set_buf_len( const struct hte_ts_desc *desc,unsigned int len )
+		int hte_get_buf_len( const struct hte_ts_desc *desc )
+
+	The consumer uses above APIs to set/get software buffer depth.
+
+.. c:function:: int hte_set_buf_watermark( const struct hte_ts_desc *desc, unsigned int val )
+		int hte_get_buf_watermark( const struct hte_ts_desc *desc )
+
+	The consumer uses above APIs to set/get software threshold, threshold
+	can be used to notity or unblock waiting consumer when data becomes
+	available equal or above to threshold value.
+
+.. c:function:: size_t hte_available_ts( const struct hte_ts_desc *desc )
+
+	The consumer uses above API to get available timestamp data stored
+	in the software buffer for the desc.
+
+The detail about parameters and API usage are described in each functions
+definitions in ``drivers/hte/hte.c`` file.
+
+The HTE timestamp element detail
+--------------------------------
+The struct hte_ts_data, declared at ``include/linux/hte.h``, is used to pass
+timestamp details between the consumers and the providers. It expresses
+timestamp data in nano second in u64 data type. For now all the HTE APIs
+using struct hte_ts_data requires tsc to be in nano seconds. The timestamp
+element structure stores below information along with timestamp data::
+
+ struct hte_ts_data {
+	/*
+	 * Timestamp value
+	 */
+	u64 tsc;
+	/*
+	 * The sequence counter, keep track of the number of timestamps.
+	 * It can be used to check if data is dropped in between.
+	 */
+	u64 seq;
+	/* Direction of the event, i.e. falling or rising */
+	int dir;
+ };
+
+The typical hte_ts_data data life cycle::
+In this example the provider provides timestamp in nano seconds and for the
+GPIO line::
+
+ - Monitors GPIO line change.
+ - Detects the state change on GPIO line.
+ - Converts timestamps in nano seconds and stores it in tsc.
+ - Stores GPIO direction in dir variable if the provider has that hardware
+ capability.
+ - Pushes this hte_timestamp_el object to HTE subsystem.
+ - HTE subsystem increments seq counter and stores it in software buffer
+ dedicated to requested GPIO line.
+ - Waiting consumer gets notified.
+ - The consumer calls the retrieve timestamp API.
+
+HTE subsystem debugfs attributes
+--------------------------------
+HTE subsystem creates debugfs attributes at ``/sys/kernel/debug/hte/``.
+It also creates line/signal related debugfs attributes at
+``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
+
+`ts_requested`
+		The total number of entities requested from the given provider,
+		where entity is the provider specific and could represent
+		lines, GPIO, chip signals, buses etc...
+                The attribute will be availble at
+		``/sys/kernel/debug/hte/<provider>/``.
+
+		Read only value
+
+`total_ts`
+		The total number of entities supported by the provider.
+                The attribute will be availble at
+		``/sys/kernel/debug/hte/<provider>/``.
+
+		Read only value
+
+`ts_buffer_depth`
+		The software buffer lenth to store timestamp data.
+                The attribute will be availble at
+		``/sys/kernel/debug/hte/<provider>/<label or id>/``.
+
+		Read only value
+
+`ts_buffer_watermark`
+		The software buffer watermark or threshold.
+                The attribute will be availble at
+		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
+
+		Read only value
+
+`dropped_timestamps`
+		The dropped timestamps for a given line.
+                The attribute will be availble at
+		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
+
+		Read only value
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 02/11] drivers: Add HTE subsystem
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
  2021-06-25 23:55 ` [RFC 01/11] Documentation: Add HTE subsystem guide Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 17:40   ` Randy Dunlap
                     ` (2 more replies)
  2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
                   ` (9 subsequent siblings)
  11 siblings, 3 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Some devices can timestamp system lines/signals/Buses in real-time
using the hardware counter or other hardware means which can give
finer granularity and help avoid jitter introduced by software means
of timestamping. To utilize such functionality there has to be
framework where such devices can register themselves as producers or
providers so that the consumers or clients devices can request specific
line from the providers. This patch introduces such subsystem as
hardware timestamping engine (HTE).

It provides below APIs for the provider:
- hte_register_chip() -- To register the HTE chip.
- hte_unregister_chip() -- To unregister the HTE chip.
- hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.

It provides below APIs for the consumer:
- of_hte_request_ts() -- To request timestamp functionality.
- devm_of_hte_request_ts() -- Managed version of the above.
- hte_req_ts_by_dt_node() -- To request timestamp functionality by
using HTE provider dt node.
- devm_hte_release_ts() -- The managed version to release timestamp
functionality and associated resources.
- hte_retrieve_ts_ns() -- To retrieve timestamps.
- hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
- hte_enable_ts() -- To disable timestamp functionality.
- hte_disable_ts() -- To enable timestamp functionality.
- hte_available_ts() -- To query available timestamp data.
- hte_release_ts() -- To release timestamp functionality and its
associated resources.
- hte_get_clk_src_info() -- To query clock source information from
the provider

It provides centralized software buffer management per requested id to
store the timestamp data for the consumers as below:
- hte_set_buf_len() -- To set the buffer length.
- hte_get_buf_len() -- To get the buffer length.
- hte_set_buf_watermark() -- To set the software threshold/watermark.
- hte_get_buf_watermark() -- To get the software threshold/watermark.

The detail about parameters and API usage are described in each
functions definitions in drivers/hte/hte.c file.

The patch adds compilation support in Makefile and menu options in
Kconfig.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/Kconfig      |    2 +
 drivers/Makefile     |    1 +
 drivers/hte/Kconfig  |   22 +
 drivers/hte/Makefile |    1 +
 drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/hte.h  |  278 +++++++++
 6 files changed, 1672 insertions(+)
 create mode 100644 drivers/hte/Kconfig
 create mode 100644 drivers/hte/Makefile
 create mode 100644 drivers/hte/hte.c
 create mode 100644 include/linux/hte.h

diff --git a/drivers/Kconfig b/drivers/Kconfig
index 47980c6b1945..9b078964974b 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
 source "drivers/counter/Kconfig"
 
 source "drivers/most/Kconfig"
+
+source "drivers/hte/Kconfig"
 endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index 5a6d613e868d..0a996a698e4c 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
 obj-$(CONFIG_INTERCONNECT)	+= interconnect/
 obj-$(CONFIG_COUNTER)		+= counter/
 obj-$(CONFIG_MOST)		+= most/
+obj-$(CONFIG_HTE)		+= hte/
diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
new file mode 100644
index 000000000000..394e112f7dfb
--- /dev/null
+++ b/drivers/hte/Kconfig
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menuconfig HTE
+        bool "Hardware Timestamping Engine (HTE) Support"
+        help
+          Hardware Timestamping Engine (HTE) Support.
+
+          Some devices provide hardware timestamping engine which can timestamp
+	  certain device lines/signals in realtime. This way to provide
+	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
+	  comes with benefit for the applications like autonomous machines
+	  needing accurate timestamping event with less jitter.
+
+	  This framework provides a generic interface to such HTE devices
+          within the Linux kernel. It provides an API to register and
+	  unregister a HTE provider chip, configurable sw buffer to
+	  store the timestamps, push the timestamp from the HTE providers and
+	  retrieve timestamps for the consumers. It also provides means for the
+	  consumers to request signals it wishes to hardware timestamp and
+	  release them if not required.
+
+          If unsure, say no.
+
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
new file mode 100644
index 000000000000..9899dbe516f7
--- /dev/null
+++ b/drivers/hte/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_HTE)		+= hte.o
diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
new file mode 100644
index 000000000000..c53260d1e250
--- /dev/null
+++ b/drivers/hte/hte.c
@@ -0,0 +1,1368 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/err.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/kfifo.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/uaccess.h>
+#include <linux/hte.h>
+#include <linux/delay.h>
+#include <linux/debugfs.h>
+
+/* Global list of the HTE devices */
+static DEFINE_SPINLOCK(hte_lock);
+static LIST_HEAD(hte_devices);
+
+enum {
+	HTE_TS_REGISTERED,
+	HTE_TS_DISABLE,
+};
+
+/* Default FIFO depth */
+#define HTE_EV_FIFO_EL		32
+
+#define HTE_TS_NAME_LEN		10
+
+struct hte_ts_buf;
+
+/**
+ * struct hte_ts_buf_acc_func - Software buffer management functions.
+ * @store: Store timestamp from atomic context as providers most likely
+ * be pushing timestamps from their interrupt handlers.
+ * @read: Read timestamps from the buffer.
+ * @el_available: Available timestamps to retrieve. The client can use this to
+ * query available elements so that it can pre-allocate internal buffer to send
+ * to during hte_retrieve_ts_ns API.
+ * @set_length: Set length/depth of the buffer.
+ * @get_length: Get length/depth of the buffer.
+ * @set_watermark: Set software threshold of the buffer.
+ * @get_watermark: Get software threshold of the buffer.
+ * @release: Release/free buffer.
+ * @reset: Reset the buffer.
+ */
+struct hte_ts_buf_acc_func {
+	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
+	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
+		    size_t *copied);
+	size_t (*el_available)(struct hte_ts_buf *buf);
+	int (*set_length)(struct hte_ts_buf *buf,
+			  size_t length, size_t bpd);
+	size_t (*get_length)(struct hte_ts_buf *buf);
+	int (*set_watermark)(struct hte_ts_buf *buf,
+			     size_t val);
+	size_t (*get_watermark)(struct hte_ts_buf *buf);
+	void (*release)(struct hte_ts_buf *buf);
+	void (*reset)(struct hte_ts_buf *buf);
+};
+
+/**
+ * struct hte_ts_buf - Software buffer per requested id or entity to store
+ * timestamps.
+ *
+ * @datum_len: Buffer depth or number of elements.
+ * @bytes_per_datum: Element size in bytes.
+ * @watermark: Software threshold at which client will be notified.
+ * @valid: Validity of the buffer.
+ * @pollq: Waitqueue for the blocking clients.
+ * @access: Various buffer management functions.
+ */
+struct hte_ts_buf {
+	size_t datum_len;
+	size_t bytes_per_datum;
+	size_t watermark;
+	bool valid;
+	wait_queue_head_t pollq;
+	const struct hte_ts_buf_acc_func *access;
+};
+
+/**
+ * struct hte_ts_info - Information related to requested timestamp.
+ *
+ * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
+ * See xlate callback API.
+ * @flags: Flags holding state informations.
+ * @seq: Timestamp sequence counter.
+ * @dropped_ts: Dropped timestamps.
+ * @cb: Callback to notify clients.
+ * @mlock: Lock during timestamp request/release APIs.
+ * @ts_dbg_root: Root for the debug fs.
+ * @gdev: HTE abstract device that this timestamp belongs to.
+ * @buf: Per requested timestamp software buffer.
+ * @desc: Timestamp descriptor understood between clients and HTE subsystem.
+ */
+struct hte_ts_info {
+	u32 xlated_id;
+	unsigned long flags;
+	u64 seq;
+	atomic_t dropped_ts;
+	void (*cb)(enum hte_notify n);
+	struct mutex mlock;
+	struct dentry *ts_dbg_root;
+	struct hte_device *gdev;
+	struct hte_ts_buf *buf;
+	struct hte_ts_desc *desc;
+};
+
+/**
+ * struct hte_device - HTE abstract device
+ * @nlines: Number of entities this device supports.
+ * @ts_req: Total number of entities requested.
+ * @ei: Timestamp information.
+ * @sdev: Device used at various debug prints.
+ * @dbg_root: Root directory for debug fs.
+ * @list: List node for internal use.
+ * @chip: HTE chip providing this HTE device.
+ * @owner: helps prevent removal of modules when in use.
+ */
+struct hte_device {
+	u32 nlines;
+	atomic_t ts_req;
+	struct hte_ts_info *ei;
+	struct device *sdev;
+	struct dentry *dbg_root;
+	struct list_head list;
+	struct hte_chip *chip;
+	struct module *owner;
+};
+
+/* Buffer management functions */
+
+/**
+ * struct hte_kfifo - Software buffer wrapper.
+ * @buffer: Abstract buffer device.
+ * @gkf: Actual software buffer type, this case its FIFO.
+ */
+struct hte_kfifo {
+	struct hte_ts_buf buffer;
+	struct kfifo gkf;
+};
+
+#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
+
+static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
+					size_t n)
+{
+	struct hte_kfifo *kf = buf_to_kfifo(r);
+
+	if (unlikely(!r->valid))
+		return 0;
+
+	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
+}
+
+static inline int hte_ts_buf_read(struct hte_ts_buf *r,
+				  unsigned char *buf, size_t n,
+				  size_t *copied)
+{
+	struct hte_kfifo *kf = buf_to_kfifo(r);
+
+	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
+		return -EINVAL;
+
+	*copied = kfifo_out(&kf->gkf, buf, n);
+
+	return 0;
+}
+
+static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
+{
+	struct hte_kfifo *kf = buf_to_kfifo(r);
+
+	if (!r->valid)
+		return 0;
+
+	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
+}
+
+static int hte_ts_buf_set_length(struct hte_ts_buf *r,
+				 size_t length, size_t bpd)
+{
+	int ret = 0;
+	struct hte_kfifo *buf;
+
+	if ((length == 0) || (bpd == 0) || !r)
+		return -EINVAL;
+
+	buf = buf_to_kfifo(r);
+
+	if (r->datum_len != length) {
+		if (r->valid)
+			kfifo_free(&buf->gkf);
+		r->valid = false;
+		r->datum_len = length;
+		r->bytes_per_datum = bpd;
+		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
+		if (!ret)
+			r->valid = true;
+	}
+
+	return ret;
+}
+
+static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
+{
+	if ((!r->valid) || !r->datum_len)
+		return 0;
+
+	return r->datum_len;
+}
+
+static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
+{
+	if ((!r->valid) || (val > r->datum_len))
+		return -EINVAL;
+
+	r->watermark = val;
+
+	return 0;
+}
+
+static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
+{
+	if (!r->valid)
+		return 0;
+
+	return r->watermark;
+}
+
+static inline void hte_ts_buf_release(struct hte_ts_buf *r)
+{
+	struct hte_kfifo *kf = buf_to_kfifo(r);
+
+	r->valid = false;
+	kfifo_free(&kf->gkf);
+	kfree(kf);
+}
+
+static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
+{
+	struct hte_kfifo *kf = buf_to_kfifo(r);
+
+	if (!r->valid)
+		return;
+
+	kfifo_reset(&kf->gkf);
+}
+
+static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
+	.store = &hte_ts_store_to_buf,
+	.read = &hte_ts_buf_read,
+	.el_available = &hte_ts_buf_el_available,
+	.set_length = &hte_ts_buf_set_length,
+	.get_length = &hte_ts_buf_get_length,
+	.set_watermark = &hte_ts_buf_set_watermark,
+	.get_watermark = &hte_ts_buf_get_watermark,
+	.release = &hte_ts_buf_release,
+	.reset = &hte_ts_buf_reset,
+};
+
+static struct hte_ts_buf *hte_ts_buf_allocate(void)
+{
+	struct hte_kfifo *kf;
+
+	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
+	if (!kf)
+		return ERR_PTR(-ENOMEM);
+
+	init_waitqueue_head(&kf->buffer.pollq);
+	kf->buffer.watermark = 1;
+	kf->buffer.datum_len = 0;
+	kf->buffer.valid = false;
+	kf->buffer.access = &kfifo_access_funcs;
+
+	return &kf->buffer;
+}
+/* End of buffer management */
+
+/* Debugfs management */
+
+#ifdef CONFIG_DEBUG_FS
+
+static struct dentry *hte_root;
+
+static void __init hte_subsys_dbgfs_init(void)
+{
+	/* creates /sys/kernel/debug/hte/ */
+	hte_root = debugfs_create_dir("hte", NULL);
+}
+subsys_initcall(hte_subsys_dbgfs_init);
+
+static void hte_chip_dbgfs_init(struct hte_device *gdev)
+{
+	const struct hte_chip *chip = gdev->chip;
+	const char *name = chip->name ? chip->name : dev_name(chip->dev);
+
+	gdev->dbg_root = debugfs_create_dir(name, hte_root);
+	if (!gdev->dbg_root)
+		return;
+
+	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
+				&gdev->ts_req);
+	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
+			   &gdev->nlines);
+}
+
+static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
+{
+	if (!ei->gdev->dbg_root || !name)
+		return;
+
+	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
+	if (!ei->ts_dbg_root)
+		return;
+
+	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
+			      &ei->buf->datum_len);
+	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
+			      &ei->buf->watermark);
+	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
+				&ei->dropped_ts);
+}
+
+static inline void hte_dbgfs_deinit(struct dentry *root)
+{
+	if (!root)
+		return;
+
+	debugfs_remove_recursive(root);
+}
+
+#else
+
+static void hte_chip_dbgfs_init(struct hte_device *gdev)
+{
+}
+
+static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
+{
+}
+
+static inline void hte_dbgfs_deinit(struct dentry *root)
+{
+}
+
+#endif
+/* end of debugfs management*/
+
+/* Driver APIs */
+
+/**
+ * hte_release_ts() - Consumer calls this API to release the entity, where
+ * entity could be anything providers support, like lines, signals, buses,
+ * etc...
+ *
+ * The correct sequence to call this API is as below:
+ * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
+ * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
+ * still care about the data.
+ * 3) Call this API.
+ * Above sequence makes sure that entity gets released race free.
+ *
+ * @desc: timestamp descriptor, this is the same as returned by the request API.
+ *
+ * Context: hte_dbgfs_deinit() function call may use sleeping locks,
+ *	    not suitable from atomic context in that case.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_release_ts(struct hte_ts_desc *desc)
+{
+	u32 id;
+	int ret = 0;
+	struct hte_device *gdev;
+	struct hte_ts_info *ei;
+	struct hte_ts_buf *buf;
+
+	if (!desc)
+		return -EINVAL;
+
+	ei = (struct hte_ts_info *)desc->data_subsys;
+
+	if (!ei || !ei->gdev || !ei->buf)
+		return -EINVAL;
+
+	gdev = ei->gdev;
+	buf = ei->buf;
+	id = desc->con_id;
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_info(gdev->sdev, "id:%d is not registered", id);
+		return -EUSERS;
+	}
+
+	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
+	if (ret) {
+		dev_err(gdev->sdev, "id: %d free failed\n", id);
+		goto out;
+	}
+
+	atomic_dec(&gdev->ts_req);
+	atomic_set(&ei->dropped_ts, 0);
+
+	kfree(desc->name);
+	kfree(desc);
+	ei->desc = NULL;
+	ei->seq = 0;
+	buf->access->release(buf);
+
+	hte_dbgfs_deinit(ei->ts_dbg_root);
+	module_put(gdev->owner);
+
+	clear_bit(HTE_TS_REGISTERED, &ei->flags);
+
+out:
+	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hte_release_ts);
+
+static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
+{
+	u32 ts_id;
+	struct hte_device *gdev;
+	struct hte_ts_info *ei;
+	int ret;
+
+	if (!desc)
+		return -EINVAL;
+
+	ei = (struct hte_ts_info *)desc->data_subsys;
+
+	if (!ei || !ei->gdev)
+		return -EINVAL;
+
+	gdev = ei->gdev;
+	ts_id = desc->con_id;
+
+	mutex_lock(&ei->mlock);
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
+		ret = -EUSERS;
+		goto out;
+	}
+
+	if (en) {
+		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
+			ret = 0;
+			goto out;
+		}
+		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
+		if (ret) {
+			dev_warn(gdev->sdev, "id: %d enable failed\n",
+				 ts_id);
+			goto out;
+		}
+
+		clear_bit(HTE_TS_DISABLE, &ei->flags);
+		ret = 0;
+	} else {
+		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
+			ret = 0;
+			goto out;
+		}
+		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
+		if (ret) {
+			dev_warn(gdev->sdev, "id: %d disable failed\n",
+				 ts_id);
+			goto out;
+		}
+
+		set_bit(HTE_TS_DISABLE, &ei->flags);
+		ret = 0;
+	}
+
+out:
+	mutex_unlock(&ei->mlock);
+	return ret;
+}
+
+/**
+ * hte_disable_ts() - Disable timestamp on given descriptor.
+ *
+ * @desc: ts descriptor, this is the same as returned by the request API.
+ *
+ * Context: Holds mutex lock, not suitable from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_disable_ts(struct hte_ts_desc *desc)
+{
+	return hte_ts_dis_en_common(desc, false);
+}
+EXPORT_SYMBOL_GPL(hte_disable_ts);
+
+/**
+ * hte_enable_ts() - Enable timestamp on given descriptor.
+ *
+ * @desc: ts descriptor, this is the same as returned by the request API.
+ *
+ * Context: Holds mutex lock, not suitable from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_enable_ts(struct hte_ts_desc *desc)
+{
+	return hte_ts_dis_en_common(desc, true);
+}
+EXPORT_SYMBOL_GPL(hte_enable_ts);
+
+static int hte_simple_xlate(struct hte_chip *gc,
+			    const struct of_phandle_args *args,
+			    struct hte_ts_desc *desc,
+			    u32 *id)
+{
+	if (!id || !desc || !gc)
+		return -EINVAL;
+
+	/*
+	 * For the providers which do not have any internal mappings between
+	 * logically exposed ids and actual ids, will set both
+	 * the same.
+	 *
+	 * In case there is a internal mapping needed, providers will need to
+	 * provide its own xlate function where con_id will be sent as
+	 * args[0] and it will return xlated id. Later xlated id will be
+	 * used for any future exchanges between provider and subsystems.
+	 */
+
+	if (args) {
+		if (gc->of_hte_n_cells < 1)
+			return -EINVAL;
+
+		if (args->args_count != gc->of_hte_n_cells)
+			return -EINVAL;
+
+		*id = args->args[0];
+		desc->con_id = *id;
+	} else {
+		*id = desc->con_id;
+	}
+
+	if (desc->con_id > gc->nlines)
+		return -EINVAL;
+
+	desc->data_subsys = NULL;
+
+	return 0;
+}
+
+static struct hte_device *of_node_to_htedevice(struct device_node *np)
+{
+	struct hte_device *gdev;
+
+	spin_lock(&hte_lock);
+
+	list_for_each_entry(gdev, &hte_devices, list)
+		if (gdev->chip && gdev->chip->dev &&
+		    gdev->chip->dev->of_node == np) {
+			spin_unlock(&hte_lock);
+			return gdev;
+		}
+
+	spin_unlock(&hte_lock);
+
+	return ERR_PTR(-ENODEV);
+}
+
+static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
+			 u32 xlated_id, void (*cb)(enum hte_notify n))
+{
+	struct hte_ts_info *ei;
+	struct hte_ts_buf *buf;
+	int ret;
+	u32 con_id = desc->con_id;
+
+	if (!try_module_get(gdev->owner))
+		return -ENODEV;
+
+	ei = &gdev->ei[xlated_id];
+	ei->xlated_id = xlated_id;
+
+	/*
+	 * There a chance that multiple consumers requesting same entity,
+	 * lock here.
+	 */
+	mutex_lock(&ei->mlock);
+
+	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
+		dev_dbg(gdev->chip->dev, "id:%u is already registered",
+			xlated_id);
+		ret = -EUSERS;
+		goto unlock;
+	}
+
+	buf = hte_ts_buf_allocate();
+	if (IS_ERR(buf)) {
+		dev_err(gdev->chip->dev, "Buffer allocation failed");
+		ret = PTR_ERR(buf);
+		goto unlock;
+	}
+
+	/* Set default here, let consumer decide how much to set later */
+	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
+				      sizeof(struct hte_ts_data));
+
+	if (ret) {
+		dev_err(gdev->chip->dev, "Fifo set length failed");
+		goto buf_rel;
+	}
+
+	buf->access->reset(buf);
+	buf->valid = true;
+
+	ei->buf = buf;
+	ei->cb = cb;
+
+	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev, "ts request failed\n");
+		goto buf_rel;
+	}
+
+	desc->data_subsys = ei;
+	ei->desc = desc;
+
+	atomic_inc(&gdev->ts_req);
+	set_bit(HTE_TS_REGISTERED, &ei->flags);
+	mutex_unlock(&ei->mlock);
+
+	if (!desc->name) {
+		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
+		if (desc->name)
+			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
+				  con_id);
+	}
+
+	hte_ts_dbgfs_init(desc->name, ei);
+
+	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
+		__func__, con_id, xlated_id);
+
+	return 0;
+
+buf_rel:
+	buf->access->release(buf);
+unlock:
+	module_put(gdev->owner);
+	mutex_unlock(&ei->mlock);
+
+	return ret;
+}
+
+static struct hte_device *of_hte_dev_get(struct device *dev,
+					 struct device_node *np,
+					 const char *label,
+					 struct of_phandle_args *args)
+{
+	struct hte_device *gdev = NULL;
+	int index = 0;
+	int err;
+
+	if (label) {
+		index = of_property_match_string(np, "hte-names", label);
+		if (index < 0)
+			return ERR_PTR(index);
+	}
+
+	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
+					 args);
+	if (err) {
+		pr_err("%s(): can't parse \"htes\" property\n", __func__);
+		return ERR_PTR(err);
+	}
+
+	gdev = of_node_to_htedevice(args->np);
+	if (IS_ERR(gdev)) {
+		pr_err("%s(): HTE chip not found\n", __func__);
+		of_node_put(args->np);
+		return gdev;
+	}
+
+	return gdev;
+}
+
+static struct hte_ts_desc *__hte_req_ts(struct device *dev,
+					struct device_node *np,
+					const char *label,
+					void (*cb)(enum hte_notify n))
+{
+	struct hte_device *gdev = NULL;
+	struct hte_ts_desc *desc;
+	struct of_phandle_args args;
+	int ret;
+	u32 xlated_id;
+
+	gdev = of_hte_dev_get(dev, np, label, &args);
+	if (IS_ERR(gdev))
+		return ERR_CAST(gdev);
+
+	if (!gdev->chip) {
+		pr_debug("requested id does not have provider\n");
+		return ERR_PTR(-ENODEV);
+	}
+
+	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+	if (!desc)
+		return ERR_PTR(-ENOMEM);
+
+	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
+	if (ret < 0)
+		goto put;
+
+	desc->name = NULL;
+	if (label)
+		desc->name = kstrdup(label, GFP_KERNEL);
+
+	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
+	if (ret < 0)
+		goto put;
+
+	return desc;
+
+put:
+	of_node_put(args.np);
+	kfree(desc);
+
+	return ERR_PTR(ret);
+}
+
+/**
+ * of_hte_request_ts() - Consumer calls this API to request the HTE facility
+ * on the specified entity, where entity is provider specific for example,
+ * GPIO lines, signals, buses etc...
+ *
+ * @dev: Consumer device.
+ * @label: Optional label.
+ * @cb: Optional notify callback to consumer when data is pushed by the
+ * provider.
+ *
+ * Context: Holds mutex lock, not suitable from atomic context.
+ * Returns: Timestamp descriptor on success or error ptr on failure.
+ */
+struct hte_ts_desc *of_hte_request_ts(struct device *dev,
+				      const char *label,
+				      void (*cb)(enum hte_notify n))
+{
+
+	if (dev && dev->of_node)
+		return __hte_req_ts(dev, dev->of_node, label, cb);
+	else
+		return ERR_PTR(-EOPNOTSUPP);
+}
+EXPORT_SYMBOL_GPL(of_hte_request_ts);
+
+static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)
+{
+	struct hte_ts_desc **p = res;
+
+	if (WARN_ON(!p || !*p))
+		return 0;
+
+	return *p == data;
+}
+
+static void __devm_hte_release_ts(struct device *dev, void *res)
+{
+	hte_release_ts(*(struct hte_ts_desc **)res);
+}
+
+/**
+ * devm_hte_release_ts() - Resource managed hte_release_ts().
+ * @dev: HTE consumer/client device.
+ * @desc: HTE ts descriptor.
+ *
+ * Release timestamp functionality and its resources previously allocated using
+ * of_hte_request_ts(). Calling this function is usually not needed because
+ * devm-allocated resources are automatically released on driver detach.
+ *
+ * Context: Same as hte_release_ts() function.
+ * Returns: 0 on success otherwise negative error code.
+ */
+int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
+{
+	return devres_release(dev, __devm_hte_release_ts,
+			      devm_hte_ts_match_desc, desc);
+}
+EXPORT_SYMBOL_GPL(devm_hte_release_ts);
+
+/**
+ * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().
+ */
+struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
+					   const char *label,
+					   void (*cb)(enum hte_notify n))
+{
+
+	struct hte_ts_desc **ptr, *desc;
+
+	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);
+	if (!ptr)
+		return ERR_PTR(-ENOMEM);
+
+	desc = of_hte_request_ts(dev, label, cb);
+	if (!IS_ERR(desc)) {
+		*ptr = desc;
+		devres_add(dev, ptr);
+	} else {
+		devres_free(ptr);
+	}
+	return desc;
+}
+EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
+
+static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
+					  size_t val)
+{
+	struct hte_ts_info *ei;
+
+	if (!desc || !desc->data_subsys || !val) {
+		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
+		return NULL;
+	}
+
+	ei = desc->data_subsys;
+	if (!ei || !ei->buf) {
+		pr_debug("%s:%d\n", __func__, __LINE__);
+		return NULL;
+	}
+
+	return ei;
+}
+
+static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
+{
+	size_t el_avail;
+
+	el_avail = buffer->access->el_available(buffer);
+
+	return (el_avail >= to_read) ? false : true;
+}
+
+static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
+			       struct hte_ts_data *el, size_t n, bool block)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+	int ret;
+	size_t to_read, copied;
+
+	ei = hte_para_check(desc, n);
+	if (!ei)
+		return -EINVAL;
+
+	buffer = ei->buf;
+
+	to_read = min_t(size_t, n, buffer->watermark);
+
+	do {
+		if (hte_ts_buf_wait(buffer, to_read)) {
+			if (!block) {
+				/* Possibly early here to retrieve, try again */
+				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
+					__func__, ret);
+				return -EAGAIN;
+			}
+			ret = wait_event_interruptible(buffer->pollq,
+					!hte_ts_buf_wait(buffer, to_read));
+			if (ret)
+				return ret;
+		}
+		ret = buffer->access->read(buffer, (void *)el,
+					   n * buffer->bytes_per_datum,
+					   &copied);
+		if (ret < 0)
+			return ret;
+
+		if (copied > 0)
+			return 0;
+		else if (copied == 0 && !block)
+			return -EAGAIN;
+	} while (copied == 0);
+
+	return 0;
+}
+
+/**
+ * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
+ * nano seconds i.e. el->tsc will be in ns.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ * @el: buffer to store the timestamp details.
+ * @n: Number of struct hte_timestamp_el elements.
+ *
+ * Context: Can be called from the atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
+		       struct hte_ts_data *el, size_t n)
+{
+	return _hte_retrieve_ts_ns(desc, el, n, false);
+}
+EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
+
+/**
+ * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
+ * @desc: ts descriptor, same as returned from request API.
+ * @el: buffer to store the timestamp data.
+ * @n: Number of struct hte_ts_data data.
+ *
+ * Context: Can not be called from the atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
+			    struct hte_ts_data *el, size_t n)
+{
+	return _hte_retrieve_ts_ns(desc, el, n, true);
+}
+EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
+
+/**
+ * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
+ * depth.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ * @len: New length/depth.
+ *
+ * The correct sequence to set buffer length is as below:
+ * 1) Disable timestamp by calling hte_disable_ts API.
+ * 2) Optionally retrieve all the timestamps by calling non blocking
+ *    hte_retrieve_ts_ns() API. This step only needed if you still care about
+ *    the data.
+ * 3) Call this API.
+ * 4) Enable timestamp by calling hte_enable_ts API.
+ *
+ * This API destroys previously allocated buffer and creates new one, because
+ * of that, it is mandatory to follow above sequence to make sure there is no
+ * race between various other APIs in the subsystem.
+ *
+ * By default during the request API call, HTE subsystem allocates software
+ * buffer with predefined length, this API gives flexibility to adjust the
+ * length according to consumer's need.
+ *
+ * Context: Can not be called from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+	int ret;
+
+	ei = hte_para_check(desc, len);
+	if (!ei)
+		return -EINVAL;
+
+	buffer = ei->buf;
+	ret = buffer->access->set_length(buffer, len,
+					 sizeof(struct hte_ts_data));
+	if (ret)
+		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hte_set_buf_len);
+
+/**
+ * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
+ * depth or length.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ *
+ * Context: Any context.
+ * Returns: Positive length on success or 0 on failure.
+ */
+size_t hte_get_buf_len(const struct hte_ts_desc *desc)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+
+	ei = hte_para_check(desc, 1);
+	if (!ei)
+		return 0;
+
+	buffer = ei->buf;
+
+	return buffer->access->get_length(buffer);
+}
+EXPORT_SYMBOL_GPL(hte_get_buf_len);
+
+/**
+ * hte_available_ts() - Returns total available timestamps.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ *
+ * The API helps consumers to pre-allocate its internal buffer required
+ * during hte_retrieve_ts_ns call.
+ *
+ * Context: Any context.
+ * Returns: Positive value if elements are available else 0. The value is
+ * number of total available struct hte_timestamp_el elements available not
+ * the size in bytes.
+ */
+size_t hte_available_ts(const struct hte_ts_desc *desc)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+
+	ei = hte_para_check(desc, 1);
+	if (!ei)
+		return 0;
+
+	buffer = ei->buf;
+
+	return buffer->access->el_available(buffer);
+}
+EXPORT_SYMBOL_GPL(hte_available_ts);
+
+/**
+ * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
+ * buffer watermark. The correct sequence to call this API is as below:
+ * 1) Disable timestamp by calling hte_disable_ts API.
+ * 2) Call this API.
+ * 3) Enable timestamp by calling hte_enable_ts API.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ * @val: New watermark.
+ *
+ * By default during the request API call, HTE subsystem sets watermark as 1,
+ * this API gives flexibility to adjust the watermark according to consumer's
+ * need. The consumers will get notification through callback registered during
+ * request API either when timestamp is dropped or watermark is reached or will
+ * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
+ * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
+ *
+ * Context: Any context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+	int ret;
+
+	ei = hte_para_check(desc, val);
+	if (!ei)
+		return -EINVAL;
+
+	buffer = ei->buf;
+	ret = buffer->access->set_watermark(buffer, val);
+	if (ret)
+		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
+
+/**
+ * hte_get_buf_watermark() - Consumer calls this API to get software
+ * buffer watermark.
+ * @desc: ts descriptor, same as returned from request API.
+ *
+ * Context: Any context.
+ * Returns: Positive current watermark on success or 0 on failure.
+ */
+size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
+{
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+
+	ei = hte_para_check(desc, 1);
+	if (!ei)
+		return 0;
+
+	buffer = ei->buf;
+
+	return buffer->access->get_watermark(buffer);
+}
+EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
+
+/**
+ * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
+ * node directly, where meaning of the entity is provider specific, for example
+ * lines, signals, GPIOs, buses etc...
+ *
+ * @of_node: HTE provider device node.
+ * @id: entity id to monitor, this id belongs to HTE provider of_node.
+ * @cb: Optional callback to notify.
+ *
+ * Context: Holds mutex lock, can not be called from atomic context.
+ * Returns: ts descriptor on success or error pointers.
+ */
+struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
+					  unsigned int id,
+					  void (*cb)(enum hte_notify n))
+{
+	struct hte_device *gdev;
+	struct hte_ts_desc *desc;
+	int ret;
+	u32 xlated_id;
+
+	gdev = of_node_to_htedevice(of_node);
+	if (IS_ERR(gdev))
+		return ERR_PTR(-ENOTSUPP);
+
+	if (!gdev->chip || !gdev->chip->ops)
+		return ERR_PTR(-ENOTSUPP);
+
+	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+	if (!desc) {
+		ret = -ENOMEM;
+		goto out_put_device;
+	}
+
+	desc->con_id = id;
+	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev,
+			"failed to xlate id: %d\n", id);
+		goto out_free_desc;
+	}
+
+	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
+	if (ret < 0) {
+		dev_err(gdev->chip->dev,
+			"failed to request id: %d\n", id);
+		goto out_free_desc;
+	}
+
+	return desc;
+
+out_free_desc:
+	kfree(desc);
+
+out_put_device:
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
+
+/**
+ * hte_get_clk_src_info() - Consumer calls this API to query clock source
+ * information of the desc.
+ *
+ * @desc: ts descriptor, same as returned from request API.
+ *
+ * Context: Any context.
+ * Returns: 0 on success else negative error code on failure.
+ */
+int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+			 struct hte_clk_info *ci)
+{
+	struct hte_chip *chip;
+	struct hte_ts_info *ei;
+
+	if (!desc || !desc->data_subsys || !ci) {
+		pr_debug("%s:%d\n", __func__, __LINE__);
+		return -EINVAL;
+	}
+
+	ei = desc->data_subsys;
+	if (!ei || !ei->gdev || !ei->gdev->chip)
+		return -EINVAL;
+
+	chip = ei->gdev->chip;
+	if (!chip->ops->get_clk_src_info)
+		return -ENOTSUPP;
+
+	return chip->ops->get_clk_src_info(chip, ci);
+}
+EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
+
+static inline void hte_add_to_device_list(struct hte_device *gdev)
+{
+	struct hte_device *prev;
+
+	if (list_empty(&hte_devices)) {
+		list_add_tail(&gdev->list, &hte_devices);
+		return;
+	}
+
+	prev = list_last_entry(&hte_devices, struct hte_device, list);
+	list_add_tail(&gdev->list, &hte_devices);
+}
+
+/**
+ * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
+ * seconds i.e data->tsc will be in ns, it is assumed that provider will be
+ * using this API from its ISR or atomic context.
+ *
+ * @chip: The HTE chip, used during the registration.
+ * @xlated_id: entity id understood by both subsystem and provider, usually this
+ * is obtained from xlate callback during request API.
+ * @data: timestamp data.
+ * @n: Size of the data.
+ *
+ * Context: Atomic.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
+			  struct hte_ts_data *data, size_t n)
+{
+	unsigned int ret;
+	bool notify;
+	size_t el_avail;
+	struct hte_ts_buf *buffer;
+	struct hte_ts_info *ei;
+
+	if (!chip || !data || !chip->gdev)
+		return -EINVAL;
+
+	if (xlated_id > chip->nlines)
+		return -EINVAL;
+
+	ei = &chip->gdev->ei[xlated_id];
+
+	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
+	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
+		dev_dbg(chip->dev, "Unknown timestamp push\n");
+		return -EINVAL;
+	}
+
+	/* timestamp sequence counter, start from 0 */
+	data->seq = ei->seq++;
+
+	buffer = ei->buf;
+	el_avail = buffer->access->el_available(buffer);
+	ret = buffer->access->store(buffer, data, n);
+	if (ret != n) {
+		atomic_inc(&ei->dropped_ts);
+		if (ei->cb)
+			ei->cb(HTE_TS_DROPPED);
+		return -ENOMEM;
+	}
+
+	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;
+
+	/*
+	 * If there is a callback, its consumer's job to retrieve the timestamp.
+	 * For the rest, wake up the process.
+	 */
+	if (notify && ei->cb) {
+		ei->cb(HTE_TS_AVAIL);
+		return 0;
+	} else if (notify) {
+		wake_up_interruptible(&buffer->pollq);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
+
+/**
+ * hte_register_chip() - Used by provider to register a HTE chip.
+ * @chip: the HTE chip to add to subsystem.
+ *
+ * Context: Can not be called from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_register_chip(struct hte_chip *chip)
+{
+	struct hte_device *gdev;
+	int ret;
+	u32 i;
+
+	if (!chip || !chip->dev || !chip->dev->of_node)
+		return -EINVAL;
+
+	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
+		dev_err(chip->dev, "Driver needs to provide ops\n");
+		return -EINVAL;
+	}
+
+	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
+	if (!gdev)
+		return -ENOMEM;
+
+	gdev->chip = chip;
+	chip->gdev = gdev;
+	gdev->nlines = chip->nlines;
+	gdev->sdev = chip->dev;
+
+	/*
+	 * Allocate all the supported entities here at once, this will have
+	 * following advantages:
+	 * When provider pushes timestamp, it can then just send the
+	 * xlated_id, subsystem will use it as an index which
+	 * gives us the constant time access; this is important as mostly
+	 * providers will be pushing the timestamps from their ISR.
+	 */
+	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
+			   GFP_KERNEL);
+	if (!gdev->ei) {
+		ret = -ENOMEM;
+		goto err_free_gdev;
+	}
+
+	for (i = 0; i < chip->nlines; i++) {
+		gdev->ei[i].flags = 0;
+		gdev->ei[i].gdev = gdev;
+		gdev->ei[i].seq = 0;
+		mutex_init(&gdev->ei[i].mlock);
+	}
+
+	if (chip->dev->driver)
+		gdev->owner = chip->dev->driver->owner;
+	else
+		gdev->owner = THIS_MODULE;
+
+	if (!chip->xlate) {
+		chip->xlate = hte_simple_xlate;
+		/* Just a id number to monitor */
+		chip->of_hte_n_cells = 1;
+	}
+
+	of_node_get(chip->dev->of_node);
+
+	INIT_LIST_HEAD(&gdev->list);
+
+	spin_lock(&hte_lock);
+	hte_add_to_device_list(gdev);
+	spin_unlock(&hte_lock);
+
+	hte_chip_dbgfs_init(gdev);
+
+	dev_dbg(chip->dev, "Added hte chip\n");
+	return 0;
+
+err_free_gdev:
+	kfree(gdev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hte_register_chip);
+
+/**
+ * hte_unregister_chip() - Used by the provider to remove a HTE chip.
+ * @chip: the HTE chip to remove.
+ *
+ * Context: Can not be called from atomic context.
+ * Returns: 0 on success or a negative error code on failure.
+ */
+int hte_unregister_chip(struct hte_chip *chip)
+{
+	struct hte_device *gdev = chip->gdev;
+
+	spin_lock(&hte_lock);
+	list_del(&gdev->list);
+	spin_unlock(&hte_lock);
+
+	gdev->chip = NULL;
+
+	of_node_put(chip->dev->of_node);
+	hte_dbgfs_deinit(gdev->dbg_root);
+	kfree(gdev->ei);
+	kfree(gdev);
+
+	dev_dbg(chip->dev, "Removed hte chip\n");
+	return 0;
+}
+EXPORT_SYMBOL_GPL(hte_unregister_chip);
+
+/* Driver APIs ends */
diff --git a/include/linux/hte.h b/include/linux/hte.h
new file mode 100644
index 000000000000..e1737579d4c4
--- /dev/null
+++ b/include/linux/hte.h
@@ -0,0 +1,278 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#ifndef __LINUX_HTE_H
+#define __LINUX_HTE_H
+
+struct hte_chip;
+struct hte_device;
+struct of_phandle_args;
+
+/**
+ * Used by providers to indicate the direction of the timestamp.
+ */
+#define HTE_EVENT_RISING_EDGE          0x1
+#define HTE_EVENT_FALLING_EDGE         0x2
+
+/**
+ * struct hte_ts_data - HTE timestamp data.
+ * The provider uses and fills timestamp related details during push_timestamp
+ * API call. The consumer uses during retrieve_timestamp API call.
+ *
+ * @tsc: Timestamp value.
+ * @seq: Sequence counter of the timestamps.
+ * @dir: Direction of the event at the time of timestamp.
+ */
+struct hte_ts_data {
+	u64 tsc;
+	u64 seq;
+	int dir;
+};
+
+/**
+ * struct hte_clk_info - Clock source info that HTE provider uses.
+ * The provider uses hardware clock as a source to timestamp real time. This
+ * structure presents the clock information to consumers. 
+ *
+ * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
+ * @type: Clock type. CLOCK_* types.
+ */
+struct hte_clk_info {
+	u64 hz;
+	clockid_t type;
+};
+
+/**
+ * HTE subsystem notifications for the consumers.
+ *
+ * @HTE_TS_AVAIL: Timestamps available notification.
+ * @HTE_TS_DROPPED: Timestamps dropped notification.
+ */
+enum hte_notify {
+	HTE_TS_AVAIL = 1,
+	HTE_TS_DROPPED,
+	HTE_NUM_NOTIFIER,
+};
+
+/**
+ * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
+ * communication token between consumers to subsystem and subsystem to
+ * providers.
+ *
+ * @con_id: This is the same id sent in request APIs.
+ * @name: Descriptive name of the entity that is being monitored for the
+ * realtime timestamping.
+ * @data_subsys: Subsystem's private data relate to requested con_id.
+ */
+struct hte_ts_desc {
+	u32 con_id;
+	char *name;
+	void *data_subsys;
+};
+
+/**
+ * struct hte_ops - HTE operations set by providers.
+ *
+ * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @enable: Hook to enable the specified timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @disable: Hook to disable specified timestamp. Returns 0 on success,
+ * non-zero for failures.
+ * @get_clk_src_info: Optional hook to get the clock information provider uses
+ * to timestamp. Returns 0 for success and negative error code for failure. On
+ * success HTE subsystem fills up provided struct hte_clk_info.
+ *
+ * xlated_id parameter is used to communicate between HTE subsystem and the
+ * providers. It is the same id returned during xlate API call and translated
+ * by the provider. This may be helpful as both subsystem and provider locate
+ * the requested entity in constant time, where entity could be anything from
+ * lines, signals, events, buses etc.. that providers support.
+ */
+struct hte_ops {
+	int (*request)(struct hte_chip *chip, u32 xlated_id);
+	int (*release)(struct hte_chip *chip, u32 xlated_id);
+	int (*enable)(struct hte_chip *chip, u32 xlated_id);
+	int (*disable)(struct hte_chip *chip, u32 xlated_id);
+	int (*get_clk_src_info)(struct hte_chip *chip,
+				struct hte_clk_info *ci);
+};
+
+/**
+ * struct hte_chip - Abstract HTE chip structure.
+ * @name: functional name of the HTE IP block.
+ * @dev: device providing the HTE.
+ * @ops: callbacks for this HTE.
+ * @nlines: number of lines/signals supported by this chip.
+ * @xlate: Callback which translates consumer supplied logical ids to
+ * physical ids, return from 0 for the success and negative for the
+ * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
+ * @of_hte_n_cells: Number of cells used to form the HTE specifier.
+ * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
+ * @data: chip specific private data.
+ */
+struct hte_chip {
+	const char *name;
+	struct device *dev;
+	const struct hte_ops *ops;
+	u32 nlines;
+	int (*xlate)(struct hte_chip *gc,
+		     const struct of_phandle_args *args,
+		     struct hte_ts_desc *desc, u32 *xlated_id);
+	u8 of_hte_n_cells;
+
+	/* only used internally by the HTE framework */
+	struct hte_device *gdev;
+	void *data;
+};
+
+#if IS_ENABLED(CONFIG_HTE)
+/* HTE APIs for the providers */
+int hte_register_chip(struct hte_chip *chip);
+int hte_unregister_chip(struct hte_chip *chip);
+int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
+			  struct hte_ts_data *data, size_t n);
+
+/* HTE APIs for the consumers */
+
+int hte_release_ts(struct hte_ts_desc *desc);
+struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
+				      void (*cb)(enum hte_notify n));
+
+struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
+					   const char *label,
+					   void (*cb)(enum hte_notify n));
+struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
+					  unsigned int id,
+					  void (*cb)(enum hte_notify n));
+int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
+int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
+		       size_t n);
+int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
+			    struct hte_ts_data *el, size_t n);
+int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
+size_t hte_get_buf_len(const struct hte_ts_desc *desc);
+int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
+size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
+size_t hte_available_ts(const struct hte_ts_desc *desc);
+int hte_enable_ts(struct hte_ts_desc *desc);
+int hte_disable_ts(struct hte_ts_desc *desc);
+int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+			 struct hte_clk_info *ci);
+
+#else /* !CONFIG_HTE */
+static inline int hte_register_chip(struct hte_chip *chip)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_unregister_chip(struct hte_chip *chip)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_push_ts_ns_atomic(const struct hte_chip *chip,
+					u32 xlated_id,
+					const struct hte_ts_data *data,
+					size_t n)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_release_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline
+struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
+				      void (*cb)(enum hte_notify ac))
+{
+	return ERR_PTR(-ENOTSUPP);
+}
+
+static inline
+struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
+					   const char *label,
+					   void (*cb)(enum hte_notify ac))
+{
+	return ERR_PTR(-ENOTSUPP);
+}
+
+static inline int devm_hte_release_ts(struct device *dev,
+				      struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
+				     struct hte_ts_data *el, size_t n)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
+					  struct hte_ts_data *el, size_t n)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_set_buf_len(const struct hte_ts_desc *desc,
+				  size_t len)
+{
+	return -ENOTSUPP;
+}
+
+static inline size_t hte_get_buf_len(const struct hte_ts_desc *desc)
+{
+	return 0;
+}
+
+static inline int hte_set_buf_watermark(const struct hte_ts_desc *desc,
+					size_t val)
+{
+	return -ENOTSUPP;
+}
+
+static inline size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
+{
+	return 0;
+}
+
+static inline size_t hte_available_ts(const struct hte_ts_desc *desc)
+{
+	return 0;
+}
+
+static inline int hte_enable_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_disable_ts(struct hte_ts_desc *desc)
+{
+	return -ENOTSUPP;
+}
+
+static inline int hte_get_clk_src_info(const struct hte_ts_desc *desc,
+				       struct hte_clk_info *ci)
+{
+	return -ENOTSUPP;
+}
+
+static inline
+struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
+					  unsigned int id,
+					  void (*cb)(enum hte_notify n))
+{
+	return ERR_PTR(-ENOTSUPP);
+}
+#endif /* !CONFIG_HTE */
+
+#endif
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
  2021-06-25 23:55 ` [RFC 01/11] Documentation: Add HTE subsystem guide Dipen Patel
  2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-07-01 14:21   ` Kent Gibson
                     ` (2 more replies)
  2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
                   ` (8 subsequent siblings)
  11 siblings, 3 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Tegra194 device has multiple HTE instances also known as GTE
(Generic hardware Timestamping Engine) which can timestamp subset of
SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
and exposes timestamping ability on those lines to the consumers
through HTE subsystem.

Also, with this patch, added:
- documentation about this provider and its capabilities at
Documentation/hte.
- Compilation support in Makefile and Kconfig

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 Documentation/hte/index.rst        |  21 ++
 Documentation/hte/tegra194-hte.rst |  65 ++++
 Documentation/index.rst            |   1 +
 drivers/hte/Kconfig                |  12 +
 drivers/hte/Makefile               |   1 +
 drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
 6 files changed, 654 insertions(+)
 create mode 100644 Documentation/hte/index.rst
 create mode 100644 Documentation/hte/tegra194-hte.rst
 create mode 100644 drivers/hte/hte-tegra194.c

diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
new file mode 100644
index 000000000000..f311ebec6b47
--- /dev/null
+++ b/Documentation/hte/index.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============================================
+The Linux Hardware Timestamping Engine (HTE)
+============================================
+
+The HTE Subsystem
+=================
+
+.. toctree::
+   :maxdepth: 1
+
+   hte
+
+HTE Tegra Provider
+==================
+
+.. toctree::
+   :maxdepth: 1
+
+   tegra194-hte
\ No newline at end of file
diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
new file mode 100644
index 000000000000..c23eaafcf080
--- /dev/null
+++ b/Documentation/hte/tegra194-hte.rst
@@ -0,0 +1,65 @@
+HTE Kernel provider driver
+==========================
+
+Description
+-----------
+The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
+known as generic timestamping engine (GTE). This provider driver implements
+two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
+timestamp from the system counter TSC which has 31.25MHz clock rate, and the
+driver converts clock tick rate to nano seconds before storing it as timestamp
+value.
+
+GPIO GTE
+--------
+
+This GTE instance help timestamps GPIO in real time, for that to happen GPIO
+needs to be configured as input and IRQ needs to ba enabled as well. The only
+always on (AON) gpio controller instance supports timestamping GPIOs in
+realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
+controller as it requires very specific bits to be set in GPIO config register.
+It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
+GPIO functionality is accessed from the GPIOLIB. It can support both the in
+kernel and userspace consumers. In the later case, requests go through GPIOLIB
+CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
+subsystem and GPIO GTE for in kernel consumers.
+
+.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
+
+	To enable HTE on given GPIO line.
+
+.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
+
+	To retrieve hardwre timestamp in nano seconds.
+
+.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
+
+	To query if HTE is enabled on the given GPIO.
+
+There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
+driver which demonstrates above APIs for the Jetson AGX platform. For userspace
+consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
+IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
+in nano second.
+
+IRQ GTE
+--------
+
+This GTE instance helps timestamp LIC IRQ lines in real time. There are 352 IRQ
+lines which this instance can help timestamp realtime. The hte devicetree
+binding described at ``Documentation/devicetree/bindings/hte/`` gives out
+example how consumer can request IRQ line, since it is one to one mapping,
+consumers can simply specify IRQ number that they are interested in. There is
+no userspace consumer support for this GTE instance. The sample test code
+hte-tegra194-irq-test.c, located in ``drivers/hte/`` directory,
+demonstrates how to use IRQ GTE instance. The below is sample device tree
+snippet code for the test driver::
+
+ tegra_hte_irq_test {
+        compatible = "nvidia,tegra194-hte-irq-test";
+        htes = <&tegra_hte_lic 0x19>;
+        hte-names = "hte-lic";
+ };
+
+The source code of the driver both IRQ and GPIO GTE is locate at
+``drivers/hte/hte-tegra194.c``.
\ No newline at end of file
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 1b13c2445e87..b41118577fe6 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -138,6 +138,7 @@ needed).
    misc-devices/index
    scheduler/index
    mhi/index
+   hte/index
 
 Architecture-agnostic documentation
 -----------------------------------
diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index 394e112f7dfb..f7b01fcc7190 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -20,3 +20,15 @@ menuconfig HTE
 
           If unsure, say no.
 
+if HTE
+
+config HTE_TEGRA194
+	tristate "NVIDIA Tegra194 HTE Support"
+	depends on ARCH_TEGRA_194_SOC
+	help
+	  Enable this option for integrated hardware timestamping engine also
+	  known as generic timestamping engine (GTE) support on NVIDIA Tegra194
+	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
+	  lines for timestamping in realtime.
+
+endif
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index 9899dbe516f7..52f978cfc913 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1 +1,2 @@
 obj-$(CONFIG_HTE)		+= hte.o
+obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
\ No newline at end of file
diff --git a/drivers/hte/hte-tegra194.c b/drivers/hte/hte-tegra194.c
new file mode 100644
index 000000000000..8ad10efd3641
--- /dev/null
+++ b/drivers/hte/hte-tegra194.c
@@ -0,0 +1,554 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/stat.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/hte.h>
+#include <linux/uaccess.h>
+
+#define HTE_SUSPEND	0
+
+/* HTE source clock TSC is 31.25MHz */
+#define HTE_TS_CLK_RATE_HZ	31250000ULL
+#define HTE_CLK_RATE_NS		32
+#define HTE_TS_NS_SHIFT	__builtin_ctz(HTE_CLK_RATE_NS)
+
+#define NV_AON_SLICE_INVALID	-1
+
+/* AON HTE line map For slice 1 */
+#define NV_AON_HTE_SLICE1_IRQ_GPIO_28	12
+#define NV_AON_HTE_SLICE1_IRQ_GPIO_29	13
+
+/* AON HTE line map For slice 2 */
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_0	0
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_1	1
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_2	2
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_3	3
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_4	4
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_5	5
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_6	6
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_7	7
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_8	8
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_9	9
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_10	10
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_11	11
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_12	12
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_13	13
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_14	14
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_15	15
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_16	16
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_17	17
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_18	18
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_19	19
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_20	20
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_21	21
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_22	22
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_23	23
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_24	24
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_25	25
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_26	26
+#define NV_AON_HTE_SLICE2_IRQ_GPIO_27	27
+
+/* AON GPIO port AA pins */
+#define NV_AON_GPIO_PORT_AA_0		0
+#define NV_AON_GPIO_PORT_AA_1		1
+#define NV_AON_GPIO_PORT_AA_2		2
+#define NV_AON_GPIO_PORT_AA_3		3
+#define NV_AON_GPIO_PORT_AA_4		4
+#define NV_AON_GPIO_PORT_AA_5		5
+#define NV_AON_GPIO_PORT_AA_6		6
+#define NV_AON_GPIO_PORT_AA_7		7
+
+/* AON GPIO port BB pins */
+#define NV_AON_GPIO_PORT_BB_0		8
+#define NV_AON_GPIO_PORT_BB_1		9
+#define NV_AON_GPIO_PORT_BB_2		10
+#define NV_AON_GPIO_PORT_BB_3		11
+
+/* AON GPIO port CC pins */
+#define NV_AON_GPIO_PORT_CC_0		16
+#define NV_AON_GPIO_PORT_CC_1		17
+#define NV_AON_GPIO_PORT_CC_2		18
+#define NV_AON_GPIO_PORT_CC_3		19
+#define NV_AON_GPIO_PORT_CC_4		20
+#define NV_AON_GPIO_PORT_CC_5		21
+#define NV_AON_GPIO_PORT_CC_6		22
+#define NV_AON_GPIO_PORT_CC_7		23
+
+/* AON GPIO port DD pins */
+#define NV_AON_GPIO_PORT_DD_0		24
+#define NV_AON_GPIO_PORT_DD_1		25
+#define NV_AON_GPIO_PORT_DD_2		26
+
+/* AON GPIO port EE pins */
+#define NV_AON_GPIO_PORT_EE_0		32
+#define NV_AON_GPIO_PORT_EE_1		33
+#define NV_AON_GPIO_PORT_EE_2		34
+#define NV_AON_GPIO_PORT_EE_3		35
+#define NV_AON_GPIO_PORT_EE_4		36
+#define NV_AON_GPIO_PORT_EE_5		37
+#define NV_AON_GPIO_PORT_EE_6		38
+
+
+#define HTE_TECTRL		0x0
+#define HTE_TETSCH		0x4
+#define HTE_TETSCL		0x8
+#define HTE_TESRC		0xC
+#define HTE_TECCV		0x10
+#define HTE_TEPCV		0x14
+#define HTE_TECMD		0x1C
+#define HTE_TESTATUS		0x20
+#define HTE_SLICE0_TETEN	0x40
+#define HTE_SLICE1_TETEN	0x60
+
+#define HTE_SLICE_SIZE		(HTE_SLICE1_TETEN - HTE_SLICE0_TETEN)
+
+#define HTE_TECTRL_ENABLE_ENABLE	0x1
+
+#define HTE_TECTRL_OCCU_SHIFT		0x8
+#define HTE_TECTRL_INTR_SHIFT		0x1
+#define HTE_TECTRL_INTR_ENABLE		0x1
+
+#define HTE_TESRC_SLICE_SHIFT		16
+#define HTE_TESRC_SLICE_DEFAULT_MASK	0xFF
+
+#define HTE_TECMD_CMD_POP		0x1
+
+#define HTE_TESTATUS_OCCUPANCY_SHIFT	8
+#define HTE_TESTATUS_OCCUPANCY_MASK	0xFF
+
+struct hte_slices {
+	u32 r_val;
+	unsigned long flags;
+	/* to prevent lines mapped to same slice updating its register */
+	spinlock_t s_lock;
+};
+
+struct tegra_hte_line_mapped {
+	int slice;
+	u32 bit_index;
+};
+
+struct tegra_hte_line_table {
+	int map_sz;
+	const struct tegra_hte_line_mapped *map;
+};
+
+struct tegra_hte_soc {
+	int hte_irq;
+	u32 itr_thrshld;
+	u32 conf_rval;
+	struct hte_slices *sl;
+	const struct tegra_hte_line_table *line_map;
+	struct hte_chip *chip;
+	void __iomem *regs;
+};
+
+static const struct tegra_hte_line_mapped tegra194_aon_gpio_map[] = {
+	/* gpio, slice, bit_index */
+	[NV_AON_GPIO_PORT_AA_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_11},
+	[NV_AON_GPIO_PORT_AA_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_10},
+	[NV_AON_GPIO_PORT_AA_2]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_9},
+	[NV_AON_GPIO_PORT_AA_3]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_8},
+	[NV_AON_GPIO_PORT_AA_4]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_7},
+	[NV_AON_GPIO_PORT_AA_5]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_6},
+	[NV_AON_GPIO_PORT_AA_6]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_5},
+	[NV_AON_GPIO_PORT_AA_7]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_4},
+	[NV_AON_GPIO_PORT_BB_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_3},
+	[NV_AON_GPIO_PORT_BB_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_2},
+	[NV_AON_GPIO_PORT_BB_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_1},
+	[NV_AON_GPIO_PORT_BB_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_0},
+	[12] = {NV_AON_SLICE_INVALID, 0},
+	[13] = {NV_AON_SLICE_INVALID, 0},
+	[14] = {NV_AON_SLICE_INVALID, 0},
+	[15] = {NV_AON_SLICE_INVALID, 0},
+	[NV_AON_GPIO_PORT_CC_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_22},
+	[NV_AON_GPIO_PORT_CC_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_21},
+	[NV_AON_GPIO_PORT_CC_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_20},
+	[NV_AON_GPIO_PORT_CC_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_19},
+	[NV_AON_GPIO_PORT_CC_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_18},
+	[NV_AON_GPIO_PORT_CC_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_17},
+	[NV_AON_GPIO_PORT_CC_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_16},
+	[NV_AON_GPIO_PORT_CC_7] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_15},
+	[NV_AON_GPIO_PORT_DD_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_14},
+	[NV_AON_GPIO_PORT_DD_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_13},
+	[NV_AON_GPIO_PORT_DD_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_12},
+	[27] = {NV_AON_SLICE_INVALID, 0},
+	[28] = {NV_AON_SLICE_INVALID, 0},
+	[29] = {NV_AON_SLICE_INVALID, 0},
+	[30] = {NV_AON_SLICE_INVALID, 0},
+	[31] = {NV_AON_SLICE_INVALID, 0},
+	[NV_AON_GPIO_PORT_EE_0] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_29},
+	[NV_AON_GPIO_PORT_EE_1] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_28},
+	[NV_AON_GPIO_PORT_EE_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_27},
+	[NV_AON_GPIO_PORT_EE_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_26},
+	[NV_AON_GPIO_PORT_EE_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_25},
+	[NV_AON_GPIO_PORT_EE_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_24},
+	[NV_AON_GPIO_PORT_EE_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_23},
+};
+
+static const struct tegra_hte_line_table aon_hte_map = {
+	.map_sz = ARRAY_SIZE(tegra194_aon_gpio_map),
+	.map = tegra194_aon_gpio_map,
+};
+
+static inline u32 tegra_hte_readl(struct tegra_hte_soc *hte, u32 reg)
+{
+	return readl(hte->regs + reg);
+}
+
+static inline void tegra_hte_writel(struct tegra_hte_soc *hte, u32 reg,
+				    u32 val)
+{
+	writel(val, hte->regs + reg);
+}
+
+static inline int tegra_hte_map_to_line_id(u32 eid, struct tegra_hte_soc *gs,
+					  u32 *mapped)
+{
+	const struct tegra_hte_line_mapped *m;
+
+	if (gs->line_map) {
+		m = gs->line_map->map;
+		if (eid > gs->line_map->map_sz)
+			return -EINVAL;
+		if (m[eid].slice == NV_AON_SLICE_INVALID)
+			return -EINVAL;
+
+		*mapped = (m[eid].slice << 5) + m[eid].bit_index;
+	} else {
+		*mapped = eid;
+	}
+
+	return 0;
+}
+
+static int tegra_hte_line_xlate(struct hte_chip *gc,
+				 const struct of_phandle_args *args,
+				 struct hte_ts_desc *desc, u32 *xlated_id)
+{
+	int ret = 0;
+
+	if (!gc || !desc || !xlated_id)
+		return -EINVAL;
+
+	if (args) {
+		if (gc->of_hte_n_cells < 1)
+			return -EINVAL;
+
+		if (args->args_count != gc->of_hte_n_cells)
+			return -EINVAL;
+
+		desc->con_id = args->args[0];
+	}
+
+	ret = tegra_hte_map_to_line_id(desc->con_id, gc->data,
+				       xlated_id);
+	if (ret < 0) {
+		dev_dbg(gc->dev, "con_id:%u mapping failed\n",
+			desc->con_id);
+		return ret;
+	}
+
+	if (*xlated_id > gc->nlines)
+		return -EINVAL;
+
+	dev_dbg(gc->dev, "requested id:%u, xlated id:%u\n",
+		desc->con_id, *xlated_id);
+
+	return 0;
+}
+
+static int tegra_hte_en_dis_common(struct hte_chip *chip, u32 line_id, bool en)
+{
+	u32 slice, sl_bit_shift, line_bit, val, reg;
+	struct tegra_hte_soc *gs;
+
+	sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	if (!chip)
+		return -EINVAL;
+
+	gs = (struct tegra_hte_soc *)chip->data;
+
+	if (line_id > chip->nlines) {
+		dev_err(chip->dev,
+			"line id: %u is not supported by this controller\n",
+			line_id);
+		return -EINVAL;
+	}
+
+	slice = line_id >> sl_bit_shift;
+	line_bit = line_id & (HTE_SLICE_SIZE - 1);
+	reg = (slice << sl_bit_shift) + HTE_SLICE0_TETEN;
+
+	spin_lock(&gs->sl[slice].s_lock);
+
+	if (test_bit(HTE_SUSPEND, &gs->sl[slice].flags)) {
+		spin_unlock(&gs->sl[slice].s_lock);
+		dev_dbg(chip->dev, "device suspended");
+		return -EBUSY;
+	}
+
+	val = tegra_hte_readl(gs, reg);
+	if (en)
+		val = val | (1 << line_bit);
+	else
+		val = val & (~(1 << line_bit));
+	tegra_hte_writel(gs, reg, val);
+
+	spin_unlock(&gs->sl[slice].s_lock);
+
+	dev_dbg(chip->dev, "line: %u, slice %u, line_bit %u, reg:0x%x\n",
+		line_id, slice, line_bit, reg);
+
+	return 0;
+}
+
+static int tegra_hte_request(struct hte_chip *chip, u32 line_id)
+{
+	return tegra_hte_en_dis_common(chip, line_id, true);
+}
+
+static int tegra_hte_release(struct hte_chip *chip, u32 line_id)
+{
+	return tegra_hte_en_dis_common(chip, line_id, false);
+}
+
+static int tegra_hte_clk_src_info(struct hte_chip *chip,
+				  struct hte_clk_info *ci)
+{
+	(void)chip;
+
+	ci->hz = HTE_TS_CLK_RATE_HZ;
+	ci->type = CLOCK_MONOTONIC;
+
+	return 0;
+}
+
+static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
+{
+	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
+	u64 tsc;
+	int dir;
+	struct hte_ts_data el;
+
+	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
+		HTE_TESTATUS_OCCUPANCY_SHIFT) &
+		HTE_TESTATUS_OCCUPANCY_MASK) {
+		tsh = tegra_hte_readl(gs, HTE_TETSCH);
+		tsl = tegra_hte_readl(gs, HTE_TETSCL);
+		tsc = (((u64)tsh << 32) | tsl);
+
+		src = tegra_hte_readl(gs, HTE_TESRC);
+		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
+			    HTE_TESRC_SLICE_DEFAULT_MASK;
+
+		pv = tegra_hte_readl(gs, HTE_TEPCV);
+		cv = tegra_hte_readl(gs, HTE_TECCV);
+		acv = pv ^ cv;
+		while (acv) {
+			bit_index = __builtin_ctz(acv);
+			if ((pv >> bit_index) & BIT(0))
+				dir = HTE_EVENT_RISING_EDGE;
+			else
+				dir = HTE_EVENT_FALLING_EDGE;
+
+			line_id = bit_index + (slice << 5);
+			el.dir = dir;
+			el.tsc = tsc << HTE_TS_NS_SHIFT;
+			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
+					      sizeof(el));
+			acv &= ~BIT(bit_index);
+		}
+		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
+	}
+}
+
+static irqreturn_t tegra_hte_isr(int irq, void *dev_id)
+{
+	struct tegra_hte_soc *gs = dev_id;
+
+	tegra_hte_read_fifo(gs);
+
+	return IRQ_HANDLED;
+}
+
+static const struct of_device_id tegra_hte_of_match[] = {
+	{ .compatible = "nvidia,tegra194-gte-lic"},
+	{ .compatible = "nvidia,tegra194-gte-aon", .data = &aon_hte_map},
+	{ }
+};
+MODULE_DEVICE_TABLE(of, tegra_hte_of_match);
+
+static const struct hte_ops g_ops = {
+	.request = tegra_hte_request,
+	.release = tegra_hte_release,
+	.enable = tegra_hte_request,
+	.disable = tegra_hte_release,
+	.get_clk_src_info = tegra_hte_clk_src_info,
+};
+
+static int tegra_hte_probe(struct platform_device *pdev)
+{
+	int ret;
+	u32 i, slices, val = 0;
+	struct device *dev;
+	struct tegra_hte_soc *hte_dev;
+	struct hte_chip *gc;
+
+	dev = &pdev->dev;
+
+	hte_dev = devm_kzalloc(dev, sizeof(*hte_dev), GFP_KERNEL);
+	if (!hte_dev)
+		return -ENOMEM;
+
+	gc = devm_kzalloc(dev, sizeof(*gc), GFP_KERNEL);
+	if (!gc)
+		return -ENOMEM;
+
+	dev_set_drvdata(&pdev->dev, hte_dev);
+	hte_dev->line_map = of_device_get_match_data(&pdev->dev);
+
+	hte_dev->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(hte_dev->regs))
+		return PTR_ERR(hte_dev->regs);
+
+	ret = of_property_read_u32(dev->of_node, "int-threshold",
+				   &hte_dev->itr_thrshld);
+	if (ret != 0)
+		hte_dev->itr_thrshld = 1;
+
+	ret = of_property_read_u32(dev->of_node, "slices", &slices);
+	if (ret != 0) {
+		dev_err(dev, "Could not read slices\n");
+		return -EINVAL;
+	}
+
+	hte_dev->sl = devm_kzalloc(dev, sizeof(struct hte_slices) * slices,
+				   GFP_KERNEL);
+	if (!hte_dev->sl)
+		return -ENOMEM;
+
+	ret = platform_get_irq(pdev, 0);
+	if (ret < 0) {
+		dev_err(dev, "get irq failed.\n");
+		return ret;
+	}
+	hte_dev->hte_irq = ret;
+	ret = devm_request_irq(dev, hte_dev->hte_irq, tegra_hte_isr, 0,
+			       dev_name(dev), hte_dev);
+	if (ret < 0) {
+		dev_err(dev, "request irq failed.\n");
+		return ret;
+	}
+
+	gc->nlines = slices << 5;
+	gc->ops = &g_ops;
+	gc->dev = dev;
+	hte_dev->chip = gc;
+	gc->data = (void *)hte_dev;
+	gc->xlate = tegra_hte_line_xlate;
+	gc->of_hte_n_cells = 1;
+
+	ret = hte_register_chip(hte_dev->chip);
+
+	if (ret)
+		dev_err(gc->dev, "hte chip register failed");
+
+	for (i = 0; i < slices; i++) {
+		hte_dev->sl[i].flags = 0;
+		spin_lock_init(&hte_dev->sl[i].s_lock);
+	}
+
+	val = HTE_TECTRL_ENABLE_ENABLE |
+	      (HTE_TECTRL_INTR_ENABLE << HTE_TECTRL_INTR_SHIFT) |
+	      (hte_dev->itr_thrshld << HTE_TECTRL_OCCU_SHIFT);
+	tegra_hte_writel(hte_dev, HTE_TECTRL, val);
+
+	dev_dbg(gc->dev, "lines: %d, slices:%d", gc->nlines, slices);
+	return 0;
+}
+
+static int tegra_hte_remove(struct platform_device *pdev)
+{
+	struct tegra_hte_soc *gs = dev_get_drvdata(&pdev->dev);
+
+	tegra_hte_writel(gs, HTE_TECTRL, 0);
+
+	return hte_unregister_chip(gs->chip);
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int tegra_hte_resume_early(struct device *dev)
+{
+	u32 i;
+	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
+	u32 slices = gs->chip->nlines >> 5;
+	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	tegra_hte_writel(gs, HTE_TECTRL, gs->conf_rval);
+
+	for (i = 0; i < slices; i++) {
+		spin_lock(&gs->sl[i].s_lock);
+		tegra_hte_writel(gs,
+				 ((i << sl_bit_shift) + HTE_SLICE0_TETEN),
+				 gs->sl[i].r_val);
+		clear_bit(HTE_SUSPEND, &gs->sl[i].flags);
+		spin_unlock(&gs->sl[i].s_lock);
+	}
+
+	return 0;
+}
+
+static int tegra_hte_suspend_late(struct device *dev)
+{
+	u32 i;
+	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
+	u32 slices = gs->chip->nlines >> 5;
+	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
+
+	gs->conf_rval = tegra_hte_readl(gs, HTE_TECTRL);
+	for (i = 0; i < slices; i++) {
+		spin_lock(&gs->sl[i].s_lock);
+		gs->sl[i].r_val = tegra_hte_readl(gs,
+				((i << sl_bit_shift) + HTE_SLICE0_TETEN));
+		set_bit(HTE_SUSPEND, &gs->sl[i].flags);
+		spin_unlock(&gs->sl[i].s_lock);
+	}
+
+	return 0;
+}
+#endif
+
+static const struct dev_pm_ops tegra_hte_pm = {
+	SET_LATE_SYSTEM_SLEEP_PM_OPS(tegra_hte_suspend_late,
+				     tegra_hte_resume_early)
+};
+
+static struct platform_driver tegra_hte_driver = {
+	.probe = tegra_hte_probe,
+	.remove = tegra_hte_remove,
+	.driver = {
+		.name = "tegra_hte",
+		.pm = &tegra_hte_pm,
+		.of_match_table = tegra_hte_of_match,
+	},
+};
+
+module_platform_driver(tegra_hte_driver);
+
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_DESCRIPTION("NVIDIA Tegra HTE (Hardware Timestamping Engine) driver");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 04/11] dt-bindings: Add HTE bindings
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (2 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 10:56   ` Linus Walleij
                     ` (2 more replies)
  2021-06-25 23:55 ` [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver Dipen Patel
                   ` (7 subsequent siblings)
  11 siblings, 3 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Introduces HTE devicetree binding details for the HTE subsystem. It
includes examples for the consumers, binding details for the providers
and specific binding details for the Tegra194 based HTE providers.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 .../devicetree/bindings/hte/hte-consumer.yaml | 47 +++++++++++
 .../devicetree/bindings/hte/hte.yaml          | 34 ++++++++
 .../bindings/hte/nvidia,tegra194-hte.yaml     | 83 +++++++++++++++++++
 3 files changed, 164 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
 create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml

diff --git a/Documentation/devicetree/bindings/hte/hte-consumer.yaml b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
new file mode 100644
index 000000000000..79ae1f7d5185
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
@@ -0,0 +1,47 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/hte-consumer.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: HTE Consumer Device Tree Bindings
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+description: |
+  HTE properties should be named "htes". The exact meaning of each htes
+  property must be documented in the device tree binding for each device.
+  An optional property "hte-names" may contain a list of strings to label
+  each of the HTE devices listed in the "htes" property.
+
+  The "hte-names" property if specified is used to map the name of the HTE
+  device requested by the devm_of_hte_request_ts() or of_hte_request_ts
+  call to an index into the list given by the "htes" property.
+
+properties:
+  htes:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    description:
+      The list of HTE provider phandle. The provider must document the number
+      of cell that must be passed in this property along with phandle.
+
+  hte-names:
+    $ref: /schemas/types.yaml#/definitions/string-array
+    description:
+      An optional string property.
+
+required:
+  - "htes"
+
+dependencies:
+  hte-names: [ htes ]
+
+additionalProperties: true
+
+examples:
+  - |
+    hte_irq_consumer {
+              htes = <&tegra_hte_lic 0x19>;
+              hte-names = "hte-irq";
+    };
diff --git a/Documentation/devicetree/bindings/hte/hte.yaml b/Documentation/devicetree/bindings/hte/hte.yaml
new file mode 100644
index 000000000000..e285c38f1a05
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/hte.yaml
@@ -0,0 +1,34 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/hte.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: HTE providers
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+properties:
+  $nodename:
+    pattern: "^hte(@.*|-[0-9a-f])*$"
+
+  "#hte-cells":
+    description:
+      Number of cells in a HTE specifier.
+
+required:
+  - "#hte-cells"
+
+additionalProperties: true
+
+examples:
+  - |
+    tegra_hte_aon: hte@c1e0000 {
+              compatible = "nvidia,tegra194-gte-aon";
+              reg = <0xc1e0000 0x10000>;
+              interrupts = <0 13 0x4>;
+              int-threshold = <1>;
+              slices = <3>;
+              #hte-cells = <1>;
+    };
\ No newline at end of file
diff --git a/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
new file mode 100644
index 000000000000..bb76cc1971f0
--- /dev/null
+++ b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
@@ -0,0 +1,83 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Tegra194 on chip generic hardware timestamping engine (HTE)
+
+maintainers:
+  - Dipen Patel <dipenp@nvidia.com>
+
+description: |
+  Tegra194 SoC has multiple generic hardware timestamping engines which can
+  monitor subset of GPIO and on chip IRQ lines for the state change, upon
+  detection it will record timestamp (taken from system counter) in its
+  internal hardware FIFO. It has bitmap array arranged in 32bit slices where
+  each bit represent signal/line to enable or disable for the hardware
+  timestamping.
+
+properties:
+  compatible:
+    enum:
+      - nvidia,tegra194-gte-aon
+      - nvidia,tegra194-gte-lic
+
+  reg:
+    maxItems: 1
+
+  interrupts:
+    maxItems: 1
+
+  int-threshold:
+    description:
+      HTE device generates its interrupt based on this u32 FIFO threshold
+      value. The recommended value is 1.
+    minimum: 1
+    maximum: 256
+
+  slices:
+   description:
+    HTE lines are arranged in 32 bit slice where each bit represents different
+    line/signal that it can enable/configure for the timestamp. It is u32
+    property and depends on the HTE instance in the chip.
+   oneOf:
+    - items:
+        - const: 3
+    - items:
+        - const: 11
+
+  '#hte-cells':
+    const: 1
+
+required:
+  - compatible
+  - reg
+  - interrupts
+  - slices
+  - "#hte-cells"
+
+additionalProperties: false
+
+examples:
+  - |
+    tegra_hte_aon: hte@c1e0000 {
+              compatible = "nvidia,tegra194-gte-aon";
+              reg = <0xc1e0000 0x10000>;
+              interrupts = <0 13 0x4>;
+              int-threshold = <1>;
+              slices = <3>;
+              #hte-cells = <1>;
+    };
+
+  - |
+    tegra_hte_lic: hte@3aa0000 {
+              compatible = "nvidia,tegra194-gte-lic";
+              reg = <0x3aa0000 0x10000>;
+              interrupts = <0 11 0x4>;
+              int-threshold = <1>;
+              slices = <11>;
+              #hte-cells = <1>;
+    };
+
+...
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (3 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 17:42   ` Randy Dunlap
  2021-06-25 23:55 ` [RFC 06/11] gpiolib: Add HTE support Dipen Patel
                   ` (6 subsequent siblings)
  11 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Tegra194 has IRQ HTE provider which can timestamp IRQ lines in realtime
, this test driver implements consumer side which tests such provider
through HTE subsystem. During its probe, it registers sysfs interface
to easily navigate from userspace as below.

All the files are at /sys/kernel/tegra_hte_irq_test/.

- en_dis - Write only, Value 1 enables HTE line, 0 disables it
- buf_len - Write/Read, sets/gets per line buffer length
- watermark - Write/Read, sets/gets software threshold/watermark

Its devicetree detail can be accessed from
Documentation/hte/tegra194-hte.rst.

This driver can be compiled as loadable module and is tested on Jetson
AGX platform using 0x19 IRQ line which belongs to one of the
i2c controller 3160000.i2c.

i2cdetect -y 1 from the userspace on this platform should be enough to
generate this IRQ at such point HTE should be able to generate
timestamps for this test consumer driver.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/hte/Kconfig                 |   7 +
 drivers/hte/Makefile                |   3 +-
 drivers/hte/hte-tegra194-irq-test.c | 400 ++++++++++++++++++++++++++++
 3 files changed, 409 insertions(+), 1 deletion(-)
 create mode 100644 drivers/hte/hte-tegra194-irq-test.c

diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index f7b01fcc7190..c4d335c41254 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -31,4 +31,11 @@ config HTE_TEGRA194
 	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
 	  lines for timestamping in realtime.
 
+config HTE_TEGRA194_IRQ_TEST
+        tristate "NVIDIA Tegra194 HTE LIC IRQ Test"
+        depends on HTE_TEGRA194
+        help
+          The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
+	  usage for the LIC IRQ hardware timestamp.
+
 endif
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index 52f978cfc913..b1cde6bc939b 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1,2 +1,3 @@
 obj-$(CONFIG_HTE)		+= hte.o
-obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
\ No newline at end of file
+obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
+obj-$(CONFIG_HTE_TEGRA194_IRQ_TEST) += hte-tegra194-irq-test.o
\ No newline at end of file
diff --git a/drivers/hte/hte-tegra194-irq-test.c b/drivers/hte/hte-tegra194-irq-test.c
new file mode 100644
index 000000000000..c51eaeb3d1ea
--- /dev/null
+++ b/drivers/hte/hte-tegra194-irq-test.c
@@ -0,0 +1,400 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/version.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
+#include <linux/hte.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+
+/*
+ * Tegra194 On chip HTE (hardware timestamping engine) also known as GTE
+ * (generic timestamping engine) can monitor LIC (Legacy interrupt controller)
+ * IRQ lines for the event and timestamp accordingly in realtime. Follow
+ * technical reference manual for the IRQ numbers and descriptions.
+ *
+ * This sample HTE IRQ test driver demonstrating HTE API usage by enabling
+ * lic irq line in HTE to monitor and timestamp.
+ */
+
+/*
+ * Used to increase line buffer length to this power in case of the dropped
+ * timestamps.
+ */
+static unsigned int len_pow = 2;
+module_param(len_pow, uint, 0660);
+
+static struct tegra_hte_test {
+	size_t buf_len;
+	bool update_buf_len;
+	struct hte_ts_desc *desc;
+	struct work_struct ev_work;
+	struct kobject *kobj;
+	struct device *pdev;
+} hte;
+
+static void hte_callback(enum hte_notify n)
+{
+	if (n == HTE_TS_AVAIL) {
+		hte.update_buf_len = false;
+	} else if (n == HTE_TS_DROPPED) {
+		dev_info(hte.pdev, "Timestamp dropped\n");
+		hte.update_buf_len = true;
+	} else {
+		dev_dbg(hte.pdev, "Wrong notify value (%d)\n", n);
+		return;
+	}
+	schedule_work(&hte.ev_work);
+}
+
+static void tegra_hte_irq_get_ts(void)
+{
+	int ret, i;
+	size_t avail;
+	struct hte_ts_data *el;
+
+	avail = hte_available_ts(hte.desc);
+
+	/*
+	 * Workqueue only got scheduled from the hte_callback, it is highly
+	 * unlikely that there is no timestamp to retrieve.
+	 */
+	if (unlikely(!avail)) {
+		dev_dbg(hte.pdev, "timestamp not available\n");
+		goto error;
+	}
+
+	el = kzalloc(avail * sizeof(*el), GFP_KERNEL);
+	if (!el) {
+		dev_dbg(hte.pdev, "Can not allocate %lu bytes memory\n",
+			avail * sizeof(*el));
+		/*
+		 * We have two options here:
+		 * 1. Release the line as system is running low memory.
+		 * 2. Run an loop to retrieve an element till its drained.
+		 *
+		 * We will use 1st option.
+		 */
+		goto error;
+	}
+
+	ret = hte_retrieve_ts_ns_wait(hte.desc, el, avail);
+	if (ret < 0) {
+		dev_dbg(hte.pdev,
+			"Something went wrong retrieving timestamp data\n");
+		kfree(el);
+		goto error;
+	}
+
+	for (i = 0; i < avail; i++) {
+		dev_info(hte.pdev, "IRQ HW timestamp(%llu): %llu, edge: %s\n",
+			 el[i].seq, el[i].tsc,
+			 (el[i].dir == 1) ? "rising" : "falling");
+	}
+
+	kfree(el);
+
+	return;
+
+error:
+	hte_release_ts(hte.desc);
+}
+
+static void hte_ts_work(struct work_struct *data)
+{
+	size_t temp;
+	int ret;
+	(void) data;
+
+	if (hte.update_buf_len) {
+		ret = hte_disable_ts(hte.desc);
+		if (ret) {
+			dev_err(hte.pdev, "Not able to disable line\n");
+			goto error;
+		}
+
+		temp = hte.buf_len * len_pow;
+		ret = hte_set_buf_len(hte.desc, temp);
+		if (ret) {
+			dev_err(hte.pdev, "Not able to set new buf len (%lu)\n",
+				temp);
+			goto error;
+		}
+
+		hte.buf_len = hte_get_buf_len(hte.desc);
+		if (unlikely(hte.buf_len != temp)) {
+			dev_err(hte.pdev, "New length is (%lu) != (%lu)\n",
+				hte.buf_len, temp);
+			goto error;
+		}
+
+		dev_dbg(hte.pdev, "New buffer length (%lu)\n", hte.buf_len);
+		hte.update_buf_len = false;
+		ret = hte_enable_ts(hte.desc);
+		if (ret) {
+			dev_err(hte.pdev, "failed to enable line\n");
+			goto error;
+		}
+
+		return;
+	}
+
+	tegra_hte_irq_get_ts();
+
+	return;
+
+error:
+	hte_release_ts(hte.desc);
+}
+
+/*
+ * Sysfs attribute to request/release HTE IRQ line.
+ */
+static ssize_t store_en_dis(struct kobject *kobj,
+			    struct kobj_attribute *attr,
+			    const char *buf, size_t count)
+{
+	int ret = count;
+	unsigned long val = 0;
+	struct hte_clk_info ci;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (val == 1) {
+		if (hte.desc) {
+			ret = -EEXIST;
+			goto error;
+		}
+
+		hte.desc = devm_of_hte_request_ts(hte.pdev, "hte-lic",
+						  hte_callback);
+		if (IS_ERR(hte.desc)) {
+			ret = PTR_ERR(hte.desc);
+			hte.desc = NULL;
+			goto error;
+		}
+
+		hte_get_clk_src_info(hte.desc, &ci);
+		dev_info(hte.pdev, "clk rate:%llu, clk type: %d\n",
+			 ci.hz, ci.type);
+
+		hte.buf_len = hte_get_buf_len(hte.desc);
+		if (hte.buf_len < 0) {
+			ret = hte.buf_len;
+			hte_release_ts(hte.desc);
+			hte.desc = NULL;
+			goto error;
+		}
+	} else if (val == 0) {
+		if (!hte.desc) {
+			ret = -EINVAL;
+			goto error;
+		}
+		/*
+		 * Ideally, you never need to call this API, simply removing
+		 * this module should be enough, it is being called here just
+		 * for demonstration.
+		 */
+		ret = devm_hte_release_ts(hte.pdev, hte.desc);
+		if (ret)
+			goto error;
+
+		hte.desc = NULL;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+struct kobj_attribute en_dis_attr =
+		__ATTR(en_dis, 0220, NULL, store_en_dis);
+
+/*
+ * Sysfs attribute to set/get watermark.
+ */
+static ssize_t store_watermark(struct kobject *kobj,
+			       struct kobj_attribute *attr,
+			       const char *buf, size_t count)
+{
+	int ret = count;
+	size_t val = 0;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (hte.desc) {
+		ret = hte_set_buf_watermark(hte.desc, val);
+		if (ret < 0)
+			goto error;
+	} else {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+static ssize_t show_watermark(struct kobject *kobj,
+			      struct kobj_attribute *attr,
+			      char *buf)
+{
+	size_t ret;
+
+	if (hte.desc) {
+		ret = hte_get_buf_watermark(hte.desc);
+		if (!ret)
+			goto error;
+	} else {
+		goto error;
+	}
+
+	return scnprintf(buf, PAGE_SIZE, "%lu\n", ret);
+
+error:
+	return -EINVAL;
+}
+
+struct kobj_attribute watermark_attr =
+		__ATTR(watermark, 0660, show_watermark, store_watermark);
+
+/*
+ * Sysfs attribute to set/get buffer.
+ */
+static ssize_t store_buf_len(struct kobject *kobj,
+			     struct kobj_attribute *attr,
+			     const char *buf, size_t count)
+{
+	int ret = count;
+	size_t val = 0;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (hte.desc) {
+		ret = hte_set_buf_len(hte.desc, val);
+		if (ret < 0)
+			goto error;
+	} else {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+static ssize_t show_buf_len(struct kobject *kobj,
+			    struct kobj_attribute *attr,
+			    char *buf)
+{
+	size_t ret;
+
+	if (hte.desc) {
+		ret = hte_get_buf_len(hte.desc);
+		if (!ret)
+			goto error;
+	} else {
+		goto error;
+	}
+
+	return scnprintf(buf, PAGE_SIZE, "%lu\n", ret);
+
+error:
+	return -EINVAL;
+}
+
+struct kobj_attribute buf_len_attr =
+		__ATTR(buf_len, 0660, show_buf_len, store_buf_len);
+
+static struct attribute *attrs[] = {
+	&en_dis_attr.attr,
+	&watermark_attr.attr,
+	&buf_len_attr.attr,
+	NULL,
+};
+
+static struct attribute_group tegra_hte_test_attr_group = {
+	.attrs = attrs,
+};
+
+static int tegra_hte_test_sysfs_create(void)
+{
+	int ret;
+
+	/* Creates /sys/kernel/tegra_hte_irq_test */
+	hte.kobj = kobject_create_and_add("tegra_hte_irq_test", kernel_kobj);
+	if (!hte.kobj)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(hte.kobj, &tegra_hte_test_attr_group);
+	if (ret)
+		kobject_put(hte.kobj);
+	return ret;
+}
+
+static const struct of_device_id tegra_hte_irq_test_of_match[] = {
+	{ .compatible = "nvidia,tegra194-hte-irq-test"},
+	{ }
+};
+MODULE_DEVICE_TABLE(of, tegra_hte_irq_test_of_match);
+
+static int tegra_hte_test_probe(struct platform_device *pdev)
+{
+	int ret;
+
+	dev_set_drvdata(&pdev->dev, &hte);
+	hte.pdev = &pdev->dev;
+
+	ret = tegra_hte_test_sysfs_create();
+	if (ret != 0) {
+		dev_err(hte.pdev, "sysfs creation failed\n");
+		return -ENXIO;
+	}
+
+	INIT_WORK(&hte.ev_work, hte_ts_work);
+
+	return 0;
+}
+
+static int tegra_hte_test_remove(struct platform_device *pdev)
+{
+	cancel_work_sync(&hte.ev_work);
+	kobject_put(hte.kobj);
+
+	return 0;
+}
+
+static struct platform_driver tegra_hte_irq_test_driver = {
+	.probe = tegra_hte_test_probe,
+	.remove = tegra_hte_test_remove,
+	.driver = {
+		.name = "tegra_hte_irq_test",
+		.of_match_table = tegra_hte_irq_test_of_match,
+	},
+};
+module_platform_driver(tegra_hte_irq_test_driver);
+
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 06/11] gpiolib: Add HTE support
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (4 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 11:41   ` Linus Walleij
  2021-07-01 14:24   ` Kent Gibson
  2021-06-25 23:55 ` [RFC 07/11] gpio: tegra186: Add HTE in gpio-tegra186 driver Dipen Patel
                   ` (5 subsequent siblings)
  11 siblings, 2 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Some GPIO chip can provide hardware timestamp support on its GPIO lines
, in order to support that additional functions needs to be added which
can talk to both GPIO chip and HTE (hardware timestamping engine)
subsystem. This patch introduces functions which gpio consumer can use
to request hardware assisted timestamping. Below is the list of the APIs
that are added in gpiolib subsystem.

- gpiod_hw_timestamp_control - to enable/disable HTE on specified GPIO
line. This API will return HTE specific descriptor for the specified
GPIO line during the enable call, it will be stored as pointer in the
gpio_desc structure as hw_ts_data.
- gpiod_is_hw_timestamp_enabled - to query if HTE is enabled on
specified GPIO line.
- gpiod_get_hw_timestamp - to retrieve hardware timestamps.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/gpio/gpiolib.c        | 92 +++++++++++++++++++++++++++++++++++
 drivers/gpio/gpiolib.h        | 11 +++++
 include/linux/gpio/consumer.h | 21 +++++++-
 include/linux/gpio/driver.h   | 13 +++++
 4 files changed, 135 insertions(+), 2 deletions(-)

diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index 220a9d8dd4e3..335eaddfde98 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -2361,6 +2361,98 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
 }
 EXPORT_SYMBOL_GPL(gpiod_direction_output);
 
+/**
+ * gpiod_hw_timestamp_control - set the hardware assisted timestamp control.
+ * @desc:	GPIO to set
+ * @enable:	Set true to enable the hardware timestamp, false otherwise.
+ *
+ * Certain GPIO chip can rely on hardware assisted timestamp engines which can
+ * record timestamp at the occurance of the configured events on selected GPIO
+ * lines. This is helper API to control such engine.
+ *
+ * Return 0 in case of success, else an error code.
+ */
+int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable)
+{
+	struct gpio_chip	*gc;
+	int			ret = 0;
+
+	VALIDATE_DESC(desc);
+	gc = desc->gdev->chip;
+
+	if (!gc->timestamp_control) {
+		gpiod_warn(desc,
+			   "%s: Hardware assisted ts not supported\n",
+			   __func__);
+		return -ENOTSUPP;
+	}
+
+	ret = gc->timestamp_control(gc, gpio_chip_hwgpio(desc),
+				    &desc->hdesc, enable);
+
+	if (ret) {
+		gpiod_warn(desc,
+			   "%s: ts control operation failed\n", __func__);
+		return ret;
+	}
+
+	if (!enable)
+		desc->hdesc = NULL;
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(gpiod_hw_timestamp_control);
+
+/**
+ * gpiod_is_hw_timestamp_enabled - check if hardware assisted timestamp is
+ * enabled.
+ * @desc:	GPIO to check
+ *
+ * Return true in case of success, false otherwise.
+ */
+bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
+{
+	if (!desc)
+		return false;
+
+	return (desc->hdesc) ? true : false;
+}
+EXPORT_SYMBOL_GPL(gpiod_is_hw_timestamp_enabled);
+
+/**
+ * gpiod_get_hw_timestamp - Get hardware timestamp in nano seconds.
+ * @desc:	GPIO to get the timestamp.
+ * @block:	Set true to block until data is available.
+ *
+ * Return non-zero on success, else 0.
+ */
+u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
+{
+	struct gpio_chip	*gc;
+	int			ret = 0;
+	u64 ts;
+
+	VALIDATE_DESC(desc);
+	gc = desc->gdev->chip;
+
+	if (!gc->get_hw_timestamp) {
+		gpiod_warn(desc,
+			   "%s: Hardware assisted ts not supported\n",
+			   __func__);
+		return -ENOTSUPP;
+	}
+
+	ret = gc->get_hw_timestamp(gc, block, desc->hdesc, &ts);
+	if (ret) {
+		gpiod_warn(desc,
+			   "%s: get timestamp operation failed\n", __func__);
+		return 0;
+	}
+
+	return ts;
+}
+EXPORT_SYMBOL_GPL(gpiod_get_hw_timestamp);
+
 /**
  * gpiod_set_config - sets @config for a GPIO
  * @desc: descriptor of the GPIO for which to set the configuration
diff --git a/drivers/gpio/gpiolib.h b/drivers/gpio/gpiolib.h
index 30bc3f80f83e..5393e1d90f61 100644
--- a/drivers/gpio/gpiolib.h
+++ b/drivers/gpio/gpiolib.h
@@ -15,6 +15,7 @@
 #include <linux/device.h>
 #include <linux/module.h>
 #include <linux/cdev.h>
+#include <linux/hte.h>
 
 #define GPIOCHIP_NAME	"gpiochip"
 
@@ -117,6 +118,7 @@ struct gpio_desc {
 #define FLAG_EDGE_RISING     16	/* GPIO CDEV detects rising edge events */
 #define FLAG_EDGE_FALLING    17	/* GPIO CDEV detects falling edge events */
 #define FLAG_EVENT_CLOCK_REALTIME	18 /* GPIO CDEV reports REALTIME timestamps in events */
+#define FLAG_EVENT_CLOCK_HARDWARE	19 /* GPIO CDEV reports hardware timestamps in events */
 
 	/* Connection label */
 	const char		*label;
@@ -129,6 +131,15 @@ struct gpio_desc {
 	/* debounce period in microseconds */
 	unsigned int		debounce_period_us;
 #endif
+	/*
+	 * Hardware timestamp engine related internal data structure.
+	 * This gets set when the consumer calls gpiod_hw_timestamp_control to enable
+	 * hardware timestamping on the specified GPIO line. The API calls into HTE
+	 * subsystem, in turns HTE subsystem return the HTE descriptor for the GPIO
+	 * line. The hdesc will be later used with gpiod_is_hw_timestamp_enabled
+	 * and gpiod_get_hw_timestamp API calls.
+	 */
+	struct hte_ts_desc *hdesc;
 };
 
 #define gpiod_not_found(desc)		(IS_ERR(desc) && PTR_ERR(desc) == -ENOENT)
diff --git a/include/linux/gpio/consumer.h b/include/linux/gpio/consumer.h
index c73b25bc9213..476ee04de7d0 100644
--- a/include/linux/gpio/consumer.h
+++ b/include/linux/gpio/consumer.h
@@ -112,6 +112,9 @@ int gpiod_get_direction(struct gpio_desc *desc);
 int gpiod_direction_input(struct gpio_desc *desc);
 int gpiod_direction_output(struct gpio_desc *desc, int value);
 int gpiod_direction_output_raw(struct gpio_desc *desc, int value);
+int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable);
+bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc);
+u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block);
 
 /* Value get/set from non-sleeping context */
 int gpiod_get_value(const struct gpio_desc *desc);
@@ -353,8 +356,22 @@ static inline int gpiod_direction_output_raw(struct gpio_desc *desc, int value)
 	WARN_ON(desc);
 	return -ENOSYS;
 }
-
-
+static inline int gpiod_hw_timestamp_control(struct gpio_desc *desc,
+					     bool enable)
+{
+	WARN_ON(desc);
+	return -ENOSYS;
+}
+static inline bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
+{
+	WARN_ON(desc);
+	return false;
+}
+static inline u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
+{
+	WARN_ON(desc);
+	return 0;
+}
 static inline int gpiod_get_value(const struct gpio_desc *desc)
 {
 	/* GPIO can never have been requested */
diff --git a/include/linux/gpio/driver.h b/include/linux/gpio/driver.h
index 3a268781fcec..f343e8f54b08 100644
--- a/include/linux/gpio/driver.h
+++ b/include/linux/gpio/driver.h
@@ -10,6 +10,7 @@
 #include <linux/lockdep.h>
 #include <linux/pinctrl/pinctrl.h>
 #include <linux/pinctrl/pinconf-generic.h>
+#include <linux/hte.h> /* For hardware timestamping */
 
 struct gpio_desc;
 struct of_phandle_args;
@@ -304,6 +305,10 @@ struct gpio_irq_chip {
  * @add_pin_ranges: optional routine to initialize pin ranges, to be used when
  *	requires special mapping of the pins that provides GPIO functionality.
  *	It is called after adding GPIO chip and before adding IRQ chip.
+ * @timestamp_control: Dependent on GPIO chip, an optional routine to
+ * 	enable/disable hardware assisted timestamp.
+ * @get_hw_timestamp: Retrieves hardware timestamp. The consumer can specify
+ * 	block parameter if it wishes to block till timestamp is available. 
  * @base: identifies the first GPIO number handled by this chip;
  *	or, if negative during registration, requests dynamic ID allocation.
  *	DEPRECATION: providing anything non-negative and nailing the base
@@ -396,6 +401,14 @@ struct gpio_chip {
 
 	int			(*add_pin_ranges)(struct gpio_chip *gc);
 
+	int			(*timestamp_control)(struct gpio_chip *gc,
+						     unsigned int offset,
+						     struct hte_ts_desc **hdesc,
+						     bool enable);
+	int			(*get_hw_timestamp)(struct gpio_chip *gc,
+						    bool block,
+						    struct hte_ts_desc *hdesc,
+						    u64 *ts);
 	int			base;
 	u16			ngpio;
 	const char		*const *names;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 07/11] gpio: tegra186: Add HTE in gpio-tegra186 driver
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (5 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 06/11] gpiolib: Add HTE support Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Tegra194 AON GPIO controller with the use of its internal hardware
timestamping engine (HTE) also known as GTE can timestamp its GPIO
lines through system counter. This patch implements two callbacks
which are essential for the gpio consumers that want such HTE
functionality. The callbacks details can be found at
include/gpio/driver.h.

Since AON GPIO controller depends on HTE engine, it creates hardware
dependency between controller and AON HTE provider. To express that,
the optional devicetree property is introduced for AON GPIO controller.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 .../bindings/gpio/nvidia,tegra186-gpio.txt    |  7 ++
 drivers/gpio/gpio-tegra186.c                  | 78 +++++++++++++++++++
 2 files changed, 85 insertions(+)

diff --git a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
index adff16c71d21..00a3e47ab560 100644
--- a/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
+++ b/Documentation/devicetree/bindings/gpio/nvidia,tegra186-gpio.txt
@@ -127,6 +127,12 @@ Required properties:
             - 8: Active low level-sensitive.
             Valid combinations are 1, 2, 3, 4, 8.
 
+Optional properties:
+- timestamp-engine
+    AON GPIO controller has timestamp engine which can hardware timestamp
+    GPIO configured as input and IRQ. This property specifies hardware
+    timestamp engine (HTE) device-tree node.
+
 Example:
 
 #include <dt-bindings/interrupt-controller/irq.h>
@@ -162,4 +168,5 @@ gpio@c2f0000 {
 	#gpio-cells = <2>;
 	interrupt-controller;
 	#interrupt-cells = <2>;
+	timestamp-engine = <&tegra_hte_aon>;
 };
diff --git a/drivers/gpio/gpio-tegra186.c b/drivers/gpio/gpio-tegra186.c
index 05974b760796..4962d7de73f1 100644
--- a/drivers/gpio/gpio-tegra186.c
+++ b/drivers/gpio/gpio-tegra186.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/platform_device.h>
+#include <linux/hte.h>
 
 #include <dt-bindings/gpio/tegra186-gpio.h>
 #include <dt-bindings/gpio/tegra194-gpio.h>
@@ -34,6 +35,7 @@
 #define  TEGRA186_GPIO_ENABLE_CONFIG_TRIGGER_LEVEL BIT(4)
 #define  TEGRA186_GPIO_ENABLE_CONFIG_DEBOUNCE BIT(5)
 #define  TEGRA186_GPIO_ENABLE_CONFIG_INTERRUPT BIT(6)
+#define  TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC BIT(7)
 
 #define TEGRA186_GPIO_DEBOUNCE_CONTROL 0x04
 #define  TEGRA186_GPIO_DEBOUNCE_CONTROL_THRESHOLD(x) ((x) & 0xff)
@@ -79,6 +81,7 @@ struct tegra_gpio {
 	struct irq_chip intc;
 	unsigned int num_irq;
 	unsigned int *irq;
+	struct device_node *hte_nd;
 
 	const struct tegra_gpio_soc *soc;
 
@@ -188,6 +191,65 @@ static int tegra186_gpio_direction_output(struct gpio_chip *chip,
 	return 0;
 }
 
+static int tegra186_gpio_timestamp_control(struct gpio_chip *chip,
+					   unsigned int offset,
+					   struct hte_ts_desc **hdesc,
+					   bool enable)
+{
+	struct tegra_gpio *gpio = gpiochip_get_data(chip);
+	void __iomem *base;
+	int value;
+	struct hte_ts_desc *desc;
+
+	if (!gpio->hte_nd)
+		return -ENOTSUPP;
+
+	base = tegra186_gpio_get_base(gpio, offset);
+	if (WARN_ON(base == NULL))
+		return -EINVAL;
+
+	value = readl(base + TEGRA186_GPIO_ENABLE_CONFIG);
+
+	if (enable) {
+		desc = hte_req_ts_by_dt_node(gpio->hte_nd, offset, NULL);
+		if (IS_ERR(desc))
+			return PTR_ERR(desc);
+
+		*hdesc = desc;
+		value |= TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC;
+	} else {
+		desc = *hdesc;
+		hte_release_ts(desc);
+		value &= ~TEGRA186_GPIO_ENABLE_CONFIG_TIMESTAMP_FUNC;
+	}
+
+	writel(value, base + TEGRA186_GPIO_ENABLE_CONFIG);
+	return 0;
+}
+
+static int tegra186_gpio_get_hw_timestamp(struct gpio_chip *chip, bool block,
+					  struct hte_ts_desc *hdesc, u64 *ts)
+{
+	struct hte_ts_data el;
+	int ret;
+
+	if (!hdesc || !ts)
+		return -EINVAL;
+
+	if (!block)
+		ret = hte_retrieve_ts_ns(hdesc, &el, 1);
+	else
+		/* Wait till timestamp is available */
+		ret = hte_retrieve_ts_ns_wait(hdesc, &el, 1);
+
+	if (ret)
+		return ret;
+
+	*ts = el.tsc;
+
+	return 0;
+}
+
 static int tegra186_gpio_get(struct gpio_chip *chip, unsigned int offset)
 {
 	struct tegra_gpio *gpio = gpiochip_get_data(chip);
@@ -605,6 +667,7 @@ static int tegra186_gpio_probe(struct platform_device *pdev)
 	struct device_node *np;
 	char **names;
 	int err;
+	phandle hte_ph;
 
 	gpio = devm_kzalloc(&pdev->dev, sizeof(*gpio), GFP_KERNEL);
 	if (!gpio)
@@ -730,6 +793,21 @@ static int tegra186_gpio_probe(struct platform_device *pdev)
 		offset += port->pins;
 	}
 
+	err = of_property_read_u32(gpio->gpio.of_node,
+				   "timestamp-engine", &hte_ph);
+	if (!err) {
+		gpio->hte_nd = of_find_node_by_phandle(hte_ph);
+		if (!gpio->hte_nd)
+			return -ENOSYS;
+
+		gpio->gpio.timestamp_control = tegra186_gpio_timestamp_control;
+		gpio->gpio.get_hw_timestamp = tegra186_gpio_get_hw_timestamp;
+
+		of_node_put(gpio->hte_nd);
+	} else {
+		gpio->hte_nd = NULL;
+	}
+
 	platform_set_drvdata(pdev, gpio);
 
 	err = devm_gpiochip_add_data(&pdev->dev, &gpio->gpio, gpio);
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (6 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 07/11] gpio: tegra186: Add HTE in gpio-tegra186 driver Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 11:38   ` Linus Walleij
                     ` (3 more replies)
  2021-06-25 23:55 ` [RFC 09/11] tools: gpio: Add new hardware " Dipen Patel
                   ` (3 subsequent siblings)
  11 siblings, 4 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

This patch adds new clock type for the GPIO controller which can
timestamp gpio lines using hardware means. To expose such
functionalities to the userspace, code has been added in this patch
where during line create call, it checks for new clock type and if
requested, calls hardware timestamp related API from gpiolib.c.
During line change event, it retrieves timestamp in nano seconds by
calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
it disables this functionality by calling gpiod_hw_timestamp_control.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
 include/uapi/linux/gpio.h   |  1 +
 2 files changed, 64 insertions(+), 2 deletions(-)

diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
index 1631727bf0da..9f98c727e937 100644
--- a/drivers/gpio/gpiolib-cdev.c
+++ b/drivers/gpio/gpiolib-cdev.c
@@ -518,6 +518,7 @@ struct linereq {
 	 GPIO_V2_LINE_DRIVE_FLAGS | \
 	 GPIO_V2_LINE_EDGE_FLAGS | \
 	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
+	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
 	 GPIO_V2_LINE_BIAS_FLAGS)
 
 static void linereq_put_event(struct linereq *lr,
@@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
 
 static u64 line_event_timestamp(struct line *line)
 {
+	bool block;
+
 	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
 		return ktime_get_real_ns();
 
+	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
+		if (irq_count())
+			block = false;
+		else
+			block = true;
+
+		return gpiod_get_hw_timestamp(line->desc, block);
+	}
+
 	return ktime_get_ns();
 }
 
@@ -828,6 +840,7 @@ static int edge_detector_setup(struct line *line,
 		return ret;
 
 	line->irq = irq;
+
 	return 0;
 }
 
@@ -891,7 +904,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
 	/* Return an error if an unknown flag is set */
 	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
 		return -EINVAL;
-
 	/*
 	 * Do not allow both INPUT and OUTPUT flags to be set as they are
 	 * contradictory.
@@ -900,6 +912,14 @@ static int gpio_v2_line_flags_validate(u64 flags)
 	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
 		return -EINVAL;
 
+	/*
+	 * Do not mix with any other clocks if hardware assisted timestamp is
+	 * asked.
+	 */
+	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
+	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
+		return -EINVAL;
+
 	/* Edge detection requires explicit input. */
 	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
 	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
@@ -992,6 +1012,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
 
 	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
 		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
+	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
+		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
 }
 
 static long linereq_get_values(struct linereq *lr, void __user *ip)
@@ -1139,6 +1161,18 @@ static long linereq_set_config_unlocked(struct linereq *lr,
 			int val = gpio_v2_line_config_output_value(lc, i);
 
 			edge_detector_stop(&lr->lines[i]);
+
+			/*
+			 * Assuming line was input before and hardware
+			 * assisted timestamp only timestamps the input
+			 * lines.
+			 */
+			if (gpiod_is_hw_timestamp_enabled(desc)) {
+				ret = gpiod_hw_timestamp_control(desc, false);
+				if (ret)
+					return ret;
+			}
+
 			ret = gpiod_direction_output(desc, val);
 			if (ret)
 				return ret;
@@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
 					polarity_change);
 			if (ret)
 				return ret;
+
+			/* Check if new config sets hardware assisted clock */
+			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
+				ret = gpiod_hw_timestamp_control(desc, true);
+				if (ret)
+					return ret;
+			}
 		}
 
 		blocking_notifier_call_chain(&desc->gdev->notifier,
@@ -1281,8 +1322,12 @@ static void linereq_free(struct linereq *lr)
 
 	for (i = 0; i < lr->num_lines; i++) {
 		edge_detector_stop(&lr->lines[i]);
-		if (lr->lines[i].desc)
+		if (lr->lines[i].desc) {
+			if (gpiod_is_hw_timestamp_enabled(lr->lines[i].desc))
+				gpiod_hw_timestamp_control(lr->lines[i].desc,
+							   false);
 			gpiod_free(lr->lines[i].desc);
+		}
 	}
 	kfifo_free(&lr->events);
 	kfree(lr->label);
@@ -1409,6 +1454,15 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
 					flags & GPIO_V2_LINE_EDGE_FLAGS);
 			if (ret)
 				goto out_free_linereq;
+
+			/*
+			 * Check if hardware assisted timestamp is requested
+			 */
+			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
+				ret = gpiod_hw_timestamp_control(desc, true);
+				if (ret)
+					goto out_free_linereq;
+			}
 		}
 
 		blocking_notifier_call_chain(&desc->gdev->notifier,
@@ -1956,8 +2010,15 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
 	if (test_bit(FLAG_EDGE_FALLING, &desc->flags))
 		info->flags |= GPIO_V2_LINE_FLAG_EDGE_FALLING;
 
+	/*
+	 * Practically it is possible that user will want both the real time
+	 * and hardware timestamps on GPIO events, for now however lets just
+	 * work with either clocks
+	 */
 	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
 		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
+	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
+		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
 
 	debounce_period_us = READ_ONCE(desc->debounce_period_us);
 	if (debounce_period_us) {
diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
index eaaea3d8e6b4..d360545b4c21 100644
--- a/include/uapi/linux/gpio.h
+++ b/include/uapi/linux/gpio.h
@@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
 	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
 	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
 	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
+	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
 };
 
 /**
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 09/11] tools: gpio: Add new hardware clock type
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (7 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 11:36   ` Linus Walleij
  2021-06-25 23:55 ` [RFC 10/11] hte: Add tegra GPIO HTE test driver Dipen Patel
                   ` (2 subsequent siblings)
  11 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

gpiolib-cdev is extended to support hardware clock type, this
patch reflects that fact.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 tools/gpio/gpio-event-mon.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/gpio/gpio-event-mon.c b/tools/gpio/gpio-event-mon.c
index a2b233fdb572..bed52333698d 100644
--- a/tools/gpio/gpio-event-mon.c
+++ b/tools/gpio/gpio-event-mon.c
@@ -149,6 +149,7 @@ void print_usage(void)
 		"  -r         Listen for rising edges\n"
 		"  -f         Listen for falling edges\n"
 		"  -w         Report the wall-clock time for events\n"
+		"  -t         Report the hardware timestamp for events\n"
 		"  -b <n>     Debounce the line with period n microseconds\n"
 		" [-c <n>]    Do <n> loops (optional, infinite loop if not stated)\n"
 		"  -?         This helptext\n"
@@ -174,7 +175,7 @@ int main(int argc, char **argv)
 
 	memset(&config, 0, sizeof(config));
 	config.flags = GPIO_V2_LINE_FLAG_INPUT;
-	while ((c = getopt(argc, argv, "c:n:o:b:dsrfw?")) != -1) {
+	while ((c = getopt(argc, argv, "c:n:o:b:dsrfwt?")) != -1) {
 		switch (c) {
 		case 'c':
 			loops = strtoul(optarg, NULL, 10);
@@ -208,6 +209,9 @@ int main(int argc, char **argv)
 		case 'w':
 			config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
 			break;
+		case 't':
+			config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
+			break;
 		case '?':
 			print_usage();
 			return -1;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 10/11] hte: Add tegra GPIO HTE test driver
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (8 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 09/11] tools: gpio: Add new hardware " Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 17:43   ` Randy Dunlap
  2021-06-25 23:55 ` [RFC 11/11] MAINTAINERS: Added HTE Subsystem Dipen Patel
  2021-06-27 13:07 ` [RFC 00/11] Intro to Hardware timestamping engine Andy Shevchenko
  11 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Tegra194 GPIO controller and HTE supports AON GPIO lines for realtime
timestamp using hardware means. This in kernel gpio consumer test
driver demonstrates that functionality indirectly using HTE subsytem
through GPIOLIB framework. During probe it also registers sysfs
interface /sys/kernel/tegra_hte_gpio_test/gpio_en_dis. The value 1
enables gpio line for the HTE functionality while the value 0
disables that.

The test driver can be compiled as a module and takes optional
parameters gpio_out and gpio_in of type uint that specifies GPIO
numbers.

This patch also adds compilation support in Kconfig and Makefile.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 drivers/hte/Kconfig                  |   8 +
 drivers/hte/Makefile                 |   3 +-
 drivers/hte/hte-tegra194-gpio-test.c | 255 +++++++++++++++++++++++++++
 3 files changed, 265 insertions(+), 1 deletion(-)
 create mode 100644 drivers/hte/hte-tegra194-gpio-test.c

diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index c4d335c41254..e62077c1024c 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -38,4 +38,12 @@ config HTE_TEGRA194_IRQ_TEST
           The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
 	  usage for the LIC IRQ hardware timestamp.
 
+config HTE_TEGRA194_GPIO_TEST
+        tristate "NVIDIA Tegra194 HTE GPIO Test"
+        depends on HTE_TEGRA194
+        help
+          The NVIDIA Tegra194 GTE GPIO test driver demonstrates how to use HTE
+	  subsystem indirectly through gpiolib API calls for GPIO line for the
+	  hardware assisted timestamping.
+
 endif
diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
index b1cde6bc939b..3f2f3a3ac4d4 100644
--- a/drivers/hte/Makefile
+++ b/drivers/hte/Makefile
@@ -1,3 +1,4 @@
 obj-$(CONFIG_HTE)		+= hte.o
 obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
-obj-$(CONFIG_HTE_TEGRA194_IRQ_TEST) += hte-tegra194-irq-test.o
\ No newline at end of file
+obj-$(CONFIG_HTE_TEGRA194_IRQ_TEST) += hte-tegra194-irq-test.o
+obj-$(CONFIG_HTE_TEGRA194_GPIO_TEST) += hte-tegra194-gpio-test.o
diff --git a/drivers/hte/hte-tegra194-gpio-test.c b/drivers/hte/hte-tegra194-gpio-test.c
new file mode 100644
index 000000000000..b2e66d5c8700
--- /dev/null
+++ b/drivers/hte/hte-tegra194-gpio-test.c
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 NVIDIA Corporation
+ *
+ * Author: Dipen Patel <dipenp@nvidia.com>
+ */
+
+#include <linux/version.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
+#include <linux/gpio.h>
+#include <linux/timer.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+
+/*
+ * Tegra194 On chip HTE (hardware timestamping engine) also known as GTE
+ * (generic timestamping engine) can monitor subset of GPIO lines for the event
+ * and timestamp accordingly.
+ *
+ * This sample HTE GPIO test driver demonstrates HTE API usage indirectly
+ * through GPIOLIB framework. It enables hardware timestamp on gpio_in line.
+ *
+ * Note: gpio_out and gpio_in need to be shorted externally in order for this
+ * test driver to work for the GPIO monitoring. This test driver has been
+ * tested on Jetson AGX platform by shorting pin 32 and 16 on 40 pin header.
+ */
+
+static unsigned int gpio_in = 322;
+module_param(gpio_in, uint, 0660);
+
+static unsigned int gpio_out = 321;
+module_param(gpio_out, uint, 0660);
+
+static struct tegra_hte_test {
+	bool is_ts_en;
+	int gpio_in_irq;
+	struct gpio_desc *gpio_in;
+	struct gpio_desc *gpio_out;
+	struct timer_list timer;
+	struct work_struct ev_work;
+	struct kobject *kobj;
+} hte;
+
+static void hte_print_ts(struct work_struct *data)
+{
+	(void) data;
+	/*
+	 * We are called from workqueue, it is ok to block in case ts is not
+	 * yet available.
+	 */
+	pr_info("GPIO HW Timestamp: %llu\n",
+		gpiod_get_hw_timestamp(hte.gpio_in, true));
+}
+
+/*
+ * Sysfs attribute to request/release HTE gpio line
+ */
+static ssize_t store_gpio_en_dis(struct kobject *kobj,
+				 struct kobj_attribute *attr,
+				 const char *buf, size_t count)
+{
+	int ret = count;
+	unsigned long val = 0;
+
+	if (kstrtoul(buf, 10, &val) < 0) {
+		ret = -EINVAL;
+		goto error;
+	}
+
+	if (val == 1) {
+		if (hte.is_ts_en) {
+			ret = -EEXIST;
+			goto error;
+		}
+
+		ret = gpiod_hw_timestamp_control(hte.gpio_in, true);
+		if (ret)
+			goto error;
+
+		hte.is_ts_en = true;
+	} else if (val == 0) {
+		if (!hte.is_ts_en) {
+			ret = -EINVAL;
+			goto error;
+		}
+		ret = gpiod_hw_timestamp_control(hte.gpio_in, false);
+		if (ret)
+			goto error;
+
+		hte.is_ts_en = false;
+	}
+
+	ret = count;
+
+error:
+	return ret;
+}
+
+struct kobj_attribute gpio_en_dis_attr =
+		__ATTR(gpio_en_dis, 0220, NULL, store_gpio_en_dis);
+
+static struct attribute *attrs[] = {
+	&gpio_en_dis_attr.attr,
+	NULL,
+};
+
+static struct attribute_group tegra_hte_test_attr_group = {
+	.attrs = attrs,
+};
+
+static int tegra_hte_test_sysfs_create(void)
+{
+	int ret;
+
+	/* Creates /sys/kernel/tegra_hte_gpio_test */
+	hte.kobj = kobject_create_and_add("tegra_hte_gpio_test", kernel_kobj);
+	if (!hte.kobj)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(hte.kobj, &tegra_hte_test_attr_group);
+	if (ret)
+		kobject_put(hte.kobj);
+	return ret;
+}
+
+static void gpio_timer_cb(struct timer_list *t)
+{
+	gpiod_set_value(hte.gpio_out, !gpiod_get_value(hte.gpio_out));
+	mod_timer(&hte.timer, jiffies + msecs_to_jiffies(8000));
+}
+
+static irqreturn_t tegra_hte_test_gpio_isr(int irq, void *data)
+{
+	struct tegra_hte_test *hte = data;
+	u64 ts;
+
+	ts = gpiod_get_hw_timestamp(hte->gpio_in, false);
+	if (!ts)
+		schedule_work(&hte->ev_work);
+	else
+		pr_info("GPIO HW timestamp(ISR): %llu", ts);
+
+	return IRQ_HANDLED;
+}
+
+static int __init tegra_hte_gpio_test_init(void)
+{
+	int ret = 0;
+
+	ret = gpio_request(gpio_out, "gte_test_gpio_out");
+	if (ret) {
+                pr_err("failed request gpio out\n");
+                return -EINVAL;
+        }
+
+	ret = gpio_request(gpio_in, "gte_test_gpio_in");
+	if (ret) {
+                pr_err("failed request gpio in\n");
+		ret = -EINVAL;
+		goto free_gpio_out;
+        }
+
+	hte.gpio_out = gpio_to_desc(gpio_out);
+	if (!hte.gpio_out) {
+		pr_err("failed convert gpio out to desc\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	hte.gpio_in = gpio_to_desc(gpio_in);
+	if (!hte.gpio_in) {
+		pr_err("failed convert gpio in to desc\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	ret = gpiod_direction_output(hte.gpio_out, 0);
+	if (ret) {
+		pr_err("failed to set output\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	ret = gpiod_direction_input(hte.gpio_in);
+	if (ret) {
+		pr_err("failed to set input\n");
+		ret = -EINVAL;
+		goto free_gpio_in;
+	}
+
+	/* IRQ setup */
+	ret = gpiod_to_irq(hte.gpio_in);
+	if (ret < 0) {
+		pr_err("failed to map GPIO to IRQ: %d\n", ret);
+		ret = -ENXIO;
+		goto free_gpio_in;
+	}
+
+	hte.gpio_in_irq = ret;
+
+	ret = request_irq(ret, tegra_hte_test_gpio_isr,
+			  IRQF_TRIGGER_RISING,
+			  "tegra_hte_gpio_test_isr", &hte);
+	if (ret) {
+		pr_err("failed to acquire IRQ\n");
+		ret = -ENXIO;
+		goto free_irq;
+	}
+
+	ret = tegra_hte_test_sysfs_create();
+	if (ret != 0) {
+		pr_err("sysfs creation failed\n");
+		ret = -ENXIO;
+		goto free_irq;
+	}
+
+	timer_setup(&hte.timer, gpio_timer_cb, 0);
+
+	INIT_WORK(&hte.ev_work, hte_print_ts);
+	mod_timer(&hte.timer, jiffies + msecs_to_jiffies(5000));
+
+	return 0;
+
+free_irq:
+	free_irq(hte.gpio_in_irq, &hte);
+free_gpio_in:
+	gpio_free(gpio_in);
+free_gpio_out:
+	gpio_free(gpio_out);
+
+	return ret;
+}
+
+static void __exit tegra_hte_gpio_test_exit(void)
+{
+	int ret;
+
+	cancel_work_sync(&hte.ev_work);
+	free_irq(hte.gpio_in_irq, &hte);
+	gpio_free(gpio_in);
+	gpio_free(gpio_out);
+	ret = gpiod_hw_timestamp_control(hte.gpio_in, false);
+	if (ret)
+		pr_err("failed to disable hw control\n");
+	kobject_put(hte.kobj);
+	del_timer(&hte.timer);
+}
+
+module_init(tegra_hte_gpio_test_init);
+module_exit(tegra_hte_gpio_test_exit);
+MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
+MODULE_LICENSE("GPL v2");
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* [RFC 11/11] MAINTAINERS: Added HTE Subsystem
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (9 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 10/11] hte: Add tegra GPIO HTE test driver Dipen Patel
@ 2021-06-25 23:55 ` Dipen Patel
  2021-06-27 13:07 ` [RFC 00/11] Intro to Hardware timestamping engine Andy Shevchenko
  11 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-06-25 23:55 UTC (permalink / raw)
  To: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	dipenp, robh+dt

Added myself as a maintainer for this new Hardware Timestamping Engine
(HTE) subsystem.

Signed-off-by: Dipen Patel <dipenp@nvidia.com>
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index ba0cc0a67b32..29e79e7f5a50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8390,6 +8390,14 @@ L:	linux-input@vger.kernel.org
 S:	Maintained
 F:	drivers/input/touchscreen/htcpen.c
 
+HTE SUBSYSTEM
+M:	dipenp@nvidia.com
+S:	Maintained
+F:	drivers/hte/*
+F:	include/linux/hte.h
+F:	Documentation/hte/*
+F:	Documentation/devicetree/bindings/hte/*
+
 HTS221 TEMPERATURE-HUMIDITY IIO DRIVER
 M:	Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
 L:	linux-iio@vger.kernel.org
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
@ 2021-06-27 10:56   ` Linus Walleij
  2021-07-30  1:32     ` Dipen Patel
  2021-07-01 14:02   ` Rob Herring
  2021-07-01 15:54   ` Rob Herring
  2 siblings, 1 reply; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 10:56 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

Hi Dipen,

thanks a lot for this very interesting patch set!

I'm gonna try to review properly, just pointing out some conceptual
things to begin with. Bindings is a good place to start.

On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:

> +description: |
> +  HTE properties should be named "htes". The exact meaning of each htes
> +  property must be documented in the device tree binding for each device.
> +  An optional property "hte-names" may contain a list of strings to label
> +  each of the HTE devices listed in the "htes" property.

I think this is a bit over-abbreviated. IIO has:
io-channels =...
io-channel-names =...

Given DT:s infatuation with using english plural I would opt for:
hardware-timestamps = ..
hardware-timestamp-names = ...

The "engine" part is a bit of an nVidia:ism I think and a too generic
term. Could as well be "processor" or "automata" but nVidia just
happened to name it an engine. (DMA engine would be a precedent
though, so no hard preference from my side.)

When reading this it is pretty intuitively evident what is going on.

Other than that it looks really good!

> +++ b/Documentation/devicetree/bindings/hte/hte.yaml

I would name this hardware-timestamp-common.yamp or so.

> +title: HTE providers

Spell this out: Hardware timestamp providers

> +properties:
> +  $nodename:
> +    pattern: "^hte(@.*|-[0-9a-f])*$"

Likewise:
hardware-timestamp@ ...

I think this is good because it is very unambiguous.

> +examples:
> +  - |
> +    tegra_hte_aon: hte@c1e0000 {
> +              compatible = "nvidia,tegra194-gte-aon";
> +              reg = <0xc1e0000 0x10000>;
> +              interrupts = <0 13 0x4>;
> +              int-threshold = <1>;
> +              slices = <3>;
> +              #hte-cells = <1>;
> +    };

The examples can be kept to the tegra194 bindings I think, this
generic binding doesn't need an example as such.

> +$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#

This one should be named like this, that is great.

> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Tegra194 on chip generic hardware timestamping engine (HTE)

This is clear and nice.

> +  int-threshold:
> +    description:
> +      HTE device generates its interrupt based on this u32 FIFO threshold
> +      value. The recommended value is 1.
> +    minimum: 1
> +    maximum: 256

Does this mean a single timestamp in the FIFO will generate an IRQ?
Then spell that out so it is clear.

> +  slices:
> +   description:
> +    HTE lines are arranged in 32 bit slice where each bit represents different
> +    line/signal that it can enable/configure for the timestamp. It is u32
> +    property and depends on the HTE instance in the chip.
> +   oneOf:
> +    - items:
> +        - const: 3
> +    - items:
> +        - const: 11

Can't you just use
enum: [3, 11]
?

> +  '#hte-cells':
> +    const: 1

So IMO this would be something like
#hardware-timestamp-cells

Other than this it overall looks very nice to me!

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 09/11] tools: gpio: Add new hardware clock type
  2021-06-25 23:55 ` [RFC 09/11] tools: gpio: Add new hardware " Dipen Patel
@ 2021-06-27 11:36   ` Linus Walleij
  2021-07-30  3:17     ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 11:36 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:

> gpiolib-cdev is extended to support hardware clock type, this
> patch reflects that fact.
>
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
(...)
>                 case 'w':
>                         config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>                         break;
> +               case 't':
> +                       config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
> +                       break;

After the checking of the command line options we need a small sanity
check so we don't try to enable both realtime and hardware clock
at the same time, we will only be able to request one of them.

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
@ 2021-06-27 11:38   ` Linus Walleij
  2021-06-27 11:49   ` Linus Walleij
                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 11:38 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:

> This patch adds new clock type for the GPIO controller which can
> timestamp gpio lines using hardware means. To expose such
> functionalities to the userspace, code has been added in this patch
> where during line create call, it checks for new clock type and if
> requested, calls hardware timestamp related API from gpiolib.c.
> During line change event, it retrieves timestamp in nano seconds by
> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
> it disables this functionality by calling gpiod_hw_timestamp_control.
>
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>

This looks good to me, pretty much exactly as I imagine it should be
done, and it is also nice that we only implement it
for the v2 UAPI.

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 06/11] gpiolib: Add HTE support
  2021-06-25 23:55 ` [RFC 06/11] gpiolib: Add HTE support Dipen Patel
@ 2021-06-27 11:41   ` Linus Walleij
  2021-07-01 14:24   ` Kent Gibson
  1 sibling, 0 replies; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 11:41 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:

> Some GPIO chip can provide hardware timestamp support on its GPIO lines
> , in order to support that additional functions needs to be added which
> can talk to both GPIO chip and HTE (hardware timestamping engine)
> subsystem. This patch introduces functions which gpio consumer can use
> to request hardware assisted timestamping. Below is the list of the APIs
> that are added in gpiolib subsystem.
>
> - gpiod_hw_timestamp_control - to enable/disable HTE on specified GPIO
> line. This API will return HTE specific descriptor for the specified
> GPIO line during the enable call, it will be stored as pointer in the
> gpio_desc structure as hw_ts_data.
> - gpiod_is_hw_timestamp_enabled - to query if HTE is enabled on
> specified GPIO line.
> - gpiod_get_hw_timestamp - to retrieve hardware timestamps.
>
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>

This looks good to me.

The chip driver can look up and provide a timestamp provider for a
certain line, which is proper since the GPIO hardware will be tightly
coupled with the timestamp hardware so we need to ask the hardware
about this directly and delegate it to the GPIO driver.

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
  2021-06-27 11:38   ` Linus Walleij
@ 2021-06-27 11:49   ` Linus Walleij
  2021-07-30  3:16     ` Dipen Patel
  2021-07-01 14:24   ` Kent Gibson
  2021-07-09  8:30   ` Jon Hunter
  3 siblings, 1 reply; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 11:49 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:

Just a quick question about this:

> +        GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \

Is the usage intended to be such that since hardware timestamp
can not be guaranteed we need to ask for it and fail and if that
fails maybe the software wants to fall back to the realtime or
common timestamp?

I'm thinking from the view of libgpiod or similar apps that abstract
this and they will be "I want to use hardware timestamps if and
only if it is available, otherwise I want to use this other timestamp"
or is that use case uncommon, such that either you know exactly
what you want or you should not be messing with hardware
timestamps?

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 00/11] Intro to Hardware timestamping engine
  2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
                   ` (10 preceding siblings ...)
  2021-06-25 23:55 ` [RFC 11/11] MAINTAINERS: Added HTE Subsystem Dipen Patel
@ 2021-06-27 13:07 ` Andy Shevchenko
  2021-06-27 14:40   ` Linus Walleij
  11 siblings, 1 reply; 75+ messages in thread
From: Andy Shevchenko @ 2021-06-27 13:07 UTC (permalink / raw)
  To: Dipen Patel, Jarkko Nikula
  Cc: Thierry Reding, Jon Hunter, Linux Kernel Mailing List,
	linux-tegra, open list:GPIO SUBSYSTEM, Linus Walleij,
	Bartosz Golaszewski, Kent Gibson, devicetree,
	Linux Documentation List, Rob Herring

On Sat, Jun 26, 2021 at 2:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
>
> This patch series introduces new subsystem called hardware timestamping
> engine (HTE). It offers functionality such as timestamping through hardware
> means in realtime. The HTE subsystem centralizes HTE provider and consumers
> where providers can register themselves with subsystem and the consumers can
> request interested entity which could be lines, GPIO, signals or buses. The
> HTE subsystem provides timestamp in nano seconds, having said that the provider
> need to convert the timestamp if its not in that unit. There was upstream
> discussion about the same at
> https://lore.kernel.org/lkml/4c46726d-fa35-1a95-4295-bca37c8b6fe3@nvidia.com/
>
> To summarize upstream discussion:
> - It was heavily favoured by Linus and Kent to extend GPIOLIB and supporting
> GPIO drivers to add HTE functionality and I agreed to experiment with it.

I guess this series should include more people from different
companies, especially documentation parts. This may be used by
different hardware and quite different vendors. Developing a framework
like this for only one vendor is no go in general.

> This patch series implements and extends GPIOLIB and GPIO tegra driver.
> - Discussed possibility to add HTE provider as irqchip instead which
> was argued against as HTE devices are not necessarily event emitting
> devices.
> - Discussed other possibility if HTE device can be added as posix clock
> type like PTP clocks. That was also argues against since HTE devices
> are not necessarily tightly coupled with hardware clock.
>
> Typical HTE provider does following:
> - Register itself with HTE subsystem
> - Provide *request, *release, *enable, *disable timestamp callbacks and
> optional get_clk_src_info callback to HTE subsystem.
> - Provide optional xlate callback to the subsystem which can translate
> consumer provided logical ids into actual ids of the entity, where entity here
> is the provider dependent and could be GPIO, in chip lines or signals, buses
> etc...This converted id will be used between HTE subsystem and the provider for
> below bullet point.
> - Push timestamps to the subsystem. This happens when HTE provider has
> timestamp data available and willing to push it to HTE subsystem. The HTE
> subsystem stores it into software buffer for the consumers.
> - Unregister itself
>
> Typical HTE consumer does following:
> - Request interested entity it wishes to timestamp in realtime to the
> subsystem. During this call HTE subsystem allocates software buffer to
> store timestamps data.
> - The subsystem does necessary communications with the provider to
> complete the request, which includes translating logical id of the entity to
> provider dependent physical/actual id and enabling hardware timestamping on
> requested id.
> - It can optionally specify callback during registration, this cb will
> be called when provider pushes timestamps. Once notified through cb, the
> consumer can call retrieve API to read the data from the software buffer.
> If cb is not provided, the consumers can elect to call blocking version of
> retrieve API.
> - Manage pre allocated software buffer if needed. It includes changing buffer
> length and watermark/threshold. The subsystem automatically sets watermark or
> threshold at 1, consumers can later change it to any other value it wishes. The
> main purpose for having threshold functionality is to notify consumer either
> through callback if provided or unblock waiting consumer when threshold is
> reached.
> - Retrieve timestamp using various means provided by subsystem.
> - Release entity and its resources.
>
> HTE and GPIOLIB:
> - For the HTE provider which can timestamp GPIO lines.
> - For the GPIO consumers, either in kernel or userspace, The GPIOLIB and its
> CDEV framework are extended as frontend to the HTE by introducing new APIs.
> - Tegra194 AON GPIO controller has HTE support also known as GTE
> (Generic Timestamping Engine). The tegra gpio driver is modified to accommodate
> HTE functionality.
>
> Dipen Patel (11):
>   Documentation: Add HTE subsystem guide
>   drivers: Add HTE subsystem
>   hte: Add tegra194 HTE kernel provider
>   dt-bindings: Add HTE bindings
>   hte: Add Tegra194 IRQ HTE test driver
>   gpiolib: Add HTE support
>   gpio: tegra186: Add HTE in gpio-tegra186 driver
>   gpiolib: cdev: Add hardware timestamp clock type
>   tools: gpio: Add new hardware clock type
>   hte: Add tegra GPIO HTE test driver
>   MAINTAINERS: Added HTE Subsystem
>
>  .../bindings/gpio/nvidia,tegra186-gpio.txt    |    7 +
>  .../devicetree/bindings/hte/hte-consumer.yaml |   47 +
>  .../devicetree/bindings/hte/hte.yaml          |   34 +
>  .../bindings/hte/nvidia,tegra194-hte.yaml     |   83 +
>  Documentation/hte/hte.rst                     |  198 +++
>  Documentation/hte/index.rst                   |   21 +
>  Documentation/hte/tegra194-hte.rst            |   65 +
>  Documentation/index.rst                       |    1 +
>  MAINTAINERS                                   |    8 +
>  drivers/Kconfig                               |    2 +
>  drivers/Makefile                              |    1 +
>  drivers/gpio/gpio-tegra186.c                  |   78 +
>  drivers/gpio/gpiolib-cdev.c                   |   65 +-
>  drivers/gpio/gpiolib.c                        |   92 ++
>  drivers/gpio/gpiolib.h                        |   11 +
>  drivers/hte/Kconfig                           |   49 +
>  drivers/hte/Makefile                          |    4 +
>  drivers/hte/hte-tegra194-gpio-test.c          |  255 +++
>  drivers/hte/hte-tegra194-irq-test.c           |  400 +++++
>  drivers/hte/hte-tegra194.c                    |  554 +++++++
>  drivers/hte/hte.c                             | 1368 +++++++++++++++++
>  include/linux/gpio/consumer.h                 |   21 +-
>  include/linux/gpio/driver.h                   |   13 +
>  include/linux/hte.h                           |  278 ++++
>  include/uapi/linux/gpio.h                     |    1 +
>  tools/gpio/gpio-event-mon.c                   |    6 +-
>  26 files changed, 3657 insertions(+), 5 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>  create mode 100644 Documentation/hte/hte.rst
>  create mode 100644 Documentation/hte/index.rst
>  create mode 100644 Documentation/hte/tegra194-hte.rst
>  create mode 100644 drivers/hte/Kconfig
>  create mode 100644 drivers/hte/Makefile
>  create mode 100644 drivers/hte/hte-tegra194-gpio-test.c
>  create mode 100644 drivers/hte/hte-tegra194-irq-test.c
>  create mode 100644 drivers/hte/hte-tegra194.c
>  create mode 100644 drivers/hte/hte.c
>  create mode 100644 include/linux/hte.h
>
> --
> 2.17.1
>


-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 00/11] Intro to Hardware timestamping engine
  2021-06-27 13:07 ` [RFC 00/11] Intro to Hardware timestamping engine Andy Shevchenko
@ 2021-06-27 14:40   ` Linus Walleij
  2021-06-28 12:02     ` Andy Shevchenko
  0 siblings, 1 reply; 75+ messages in thread
From: Linus Walleij @ 2021-06-27 14:40 UTC (permalink / raw)
  To: Andy Shevchenko, Ye Xiang, Drew Fustini, Sandeep Singh
  Cc: Dipen Patel, Jarkko Nikula, Thierry Reding, Jon Hunter,
	Linux Kernel Mailing List, linux-tegra, open list:GPIO SUBSYSTEM,
	Bartosz Golaszewski, Kent Gibson, devicetree,
	Linux Documentation List, Rob Herring, Jonathan Cameron,
	linux-iio

On Sun, Jun 27, 2021 at 3:08 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:

> > To summarize upstream discussion:
> > - It was heavily favoured by Linus and Kent to extend GPIOLIB and supporting
> > GPIO drivers to add HTE functionality and I agreed to experiment with it.
>
> I guess this series should include more people from different
> companies, especially documentation parts. This may be used by
> different hardware and quite different vendors. Developing a framework
> like this for only one vendor is no go in general.

I forwarded patch 00 to the IIO list and Jonathan Cameron,
and let's page Ye Xiang who made a bunch of contributions
from Intel's side to IIO directly. (Hi Ye, please check this concept
if you have time!)

The actually most important target group would be people
doing things like sensor fusion where a common timebase is
important, I don't know who does really, but Sandeep Singh from
AMD has contributed the AMD Sensor Fusion hub in
drivers/hid/amd-sfh-hid and might know a few things about this
though I don't think SFH would need this directly.
https://en.wikipedia.org/wiki/Sensor_fusion

Also Paging Drew Fustini, who knows a lot of maker and tinker
people, he might know a bit about this or know someone who
knows.

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
@ 2021-06-27 17:40   ` Randy Dunlap
  2021-07-04 20:15   ` Jonathan Cameron
  2021-07-05  7:30   ` Greg KH
  2 siblings, 0 replies; 75+ messages in thread
From: Randy Dunlap @ 2021-06-27 17:40 UTC (permalink / raw)
  To: Dipen Patel, thierry.reding, jonathanh, linux-kernel,
	linux-tegra, linux-gpio, linus.walleij, bgolaszewski, warthog618,
	devicetree, linux-doc, robh+dt

On 6/25/21 4:55 PM, Dipen Patel wrote:
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> new file mode 100644
> index 000000000000..394e112f7dfb
> --- /dev/null
> +++ b/drivers/hte/Kconfig
> @@ -0,0 +1,22 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig HTE
> +        bool "Hardware Timestamping Engine (HTE) Support"
> +        help
> +          Hardware Timestamping Engine (HTE) Support.
> +
> +          Some devices provide hardware timestamping engine which can timestamp
> +	  certain device lines/signals in realtime. This way to provide
> +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
> +	  comes with benefit for the applications like autonomous machines
> +	  needing accurate timestamping event with less jitter.
> +
> +	  This framework provides a generic interface to such HTE devices
> +          within the Linux kernel. It provides an API to register and
> +	  unregister a HTE provider chip, configurable sw buffer to
> +	  store the timestamps, push the timestamp from the HTE providers and
> +	  retrieve timestamps for the consumers. It also provides means for the
> +	  consumers to request signals it wishes to hardware timestamp and
> +	  release them if not required.
> +
> +          If unsure, say no.

semi-bot:

Please follow coding-style for Kconfig files:

(from Documentation/process/coding-style.rst, section 10):

For all of the Kconfig* configuration files throughout the source tree,
the indentation is somewhat different.  Lines under a ``config`` definition
are indented with one tab, while help text is indented an additional two
spaces.


thanks.
-- 
~Randy


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver
  2021-06-25 23:55 ` [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver Dipen Patel
@ 2021-06-27 17:42   ` Randy Dunlap
  0 siblings, 0 replies; 75+ messages in thread
From: Randy Dunlap @ 2021-06-27 17:42 UTC (permalink / raw)
  To: Dipen Patel, thierry.reding, jonathanh, linux-kernel,
	linux-tegra, linux-gpio, linus.walleij, bgolaszewski, warthog618,
	devicetree, linux-doc, robh+dt

On 6/25/21 4:55 PM, Dipen Patel wrote:
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> index f7b01fcc7190..c4d335c41254 100644
> --- a/drivers/hte/Kconfig
> +++ b/drivers/hte/Kconfig
> @@ -31,4 +31,11 @@ config HTE_TEGRA194
>  	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
>  	  lines for timestamping in realtime.
>  
> +config HTE_TEGRA194_IRQ_TEST
> +        tristate "NVIDIA Tegra194 HTE LIC IRQ Test"
> +        depends on HTE_TEGRA194
> +        help
> +          The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
> +	  usage for the LIC IRQ hardware timestamp.
> +
>  endif

Fix indentation, please.


-- 
~Randy


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 10/11] hte: Add tegra GPIO HTE test driver
  2021-06-25 23:55 ` [RFC 10/11] hte: Add tegra GPIO HTE test driver Dipen Patel
@ 2021-06-27 17:43   ` Randy Dunlap
  0 siblings, 0 replies; 75+ messages in thread
From: Randy Dunlap @ 2021-06-27 17:43 UTC (permalink / raw)
  To: Dipen Patel, thierry.reding, jonathanh, linux-kernel,
	linux-tegra, linux-gpio, linus.walleij, bgolaszewski, warthog618,
	devicetree, linux-doc, robh+dt

On 6/25/21 4:55 PM, Dipen Patel wrote:
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> index c4d335c41254..e62077c1024c 100644
> --- a/drivers/hte/Kconfig
> +++ b/drivers/hte/Kconfig
> @@ -38,4 +38,12 @@ config HTE_TEGRA194_IRQ_TEST
>            The NVIDIA Tegra194 GTE IRQ test driver demonstrates HTE subsystem
>  	  usage for the LIC IRQ hardware timestamp.
>  
> +config HTE_TEGRA194_GPIO_TEST
> +        tristate "NVIDIA Tegra194 HTE GPIO Test"
> +        depends on HTE_TEGRA194
> +        help
> +          The NVIDIA Tegra194 GTE GPIO test driver demonstrates how to use HTE
> +	  subsystem indirectly through gpiolib API calls for GPIO line for the
> +	  hardware assisted timestamping.
> +
>  endif

Fix indentation, please.

-- 
~Randy


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 00/11] Intro to Hardware timestamping engine
  2021-06-27 14:40   ` Linus Walleij
@ 2021-06-28 12:02     ` Andy Shevchenko
  0 siblings, 0 replies; 75+ messages in thread
From: Andy Shevchenko @ 2021-06-28 12:02 UTC (permalink / raw)
  To: Linus Walleij
  Cc: Ye Xiang, Drew Fustini, Sandeep Singh, Dipen Patel,
	Jarkko Nikula, Thierry Reding, Jon Hunter,
	Linux Kernel Mailing List, linux-tegra, open list:GPIO SUBSYSTEM,
	Bartosz Golaszewski, Kent Gibson, devicetree,
	Linux Documentation List, Rob Herring, Jonathan Cameron,
	linux-iio

On Sun, Jun 27, 2021 at 5:41 PM Linus Walleij <linus.walleij@linaro.org> wrote:
> On Sun, Jun 27, 2021 at 3:08 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
>
> > > To summarize upstream discussion:
> > > - It was heavily favoured by Linus and Kent to extend GPIOLIB and supporting
> > > GPIO drivers to add HTE functionality and I agreed to experiment with it.
> >
> > I guess this series should include more people from different
> > companies, especially documentation parts. This may be used by
> > different hardware and quite different vendors. Developing a framework
> > like this for only one vendor is no go in general.
>
> I forwarded patch 00 to the IIO list and Jonathan Cameron,
> and let's page Ye Xiang who made a bunch of contributions
> from Intel's side to IIO directly. (Hi Ye, please check this concept
> if you have time!)
>
> The actually most important target group would be people
> doing things like sensor fusion where a common timebase is
> important, I don't know who does really, but Sandeep Singh from
> AMD has contributed the AMD Sensor Fusion hub in
> drivers/hid/amd-sfh-hid and might know a few things about this
> though I don't think SFH would need this directly.
> https://en.wikipedia.org/wiki/Sensor_fusion
>
> Also Paging Drew Fustini, who knows a lot of maker and tinker
> people, he might know a bit about this or know someone who
> knows.

Thank you!

-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
  2021-06-27 10:56   ` Linus Walleij
@ 2021-07-01 14:02   ` Rob Herring
  2021-07-30  1:56     ` Dipen Patel
  2021-07-01 15:54   ` Rob Herring
  2 siblings, 1 reply; 75+ messages in thread
From: Rob Herring @ 2021-07-01 14:02 UTC (permalink / raw)
  To: Dipen Patel
  Cc: devicetree, linux-tegra, linux-gpio, linux-kernel, robh+dt,
	linus.walleij, thierry.reding, bgolaszewski, linux-doc,
	warthog618, jonathanh

On Fri, 25 Jun 2021 16:55:25 -0700, Dipen Patel wrote:
> Introduces HTE devicetree binding details for the HTE subsystem. It
> includes examples for the consumers, binding details for the providers
> and specific binding details for the Tegra194 based HTE providers.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  .../devicetree/bindings/hte/hte-consumer.yaml | 47 +++++++++++
>  .../devicetree/bindings/hte/hte.yaml          | 34 ++++++++
>  .../bindings/hte/nvidia,tegra194-hte.yaml     | 83 +++++++++++++++++++
>  3 files changed, 164 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
> 

My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
on your patch (DT_CHECKER_FLAGS is new in v5.13):

yamllint warnings/errors:
./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:40:4: [warning] wrong indentation: expected 4 but found 3 (indentation)
./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:41:5: [warning] wrong indentation: expected 5 but found 4 (indentation)
./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:45:5: [warning] wrong indentation: expected 5 but found 4 (indentation)
./Documentation/devicetree/bindings/hte/hte.yaml:34:7: [error] no new line character at the end of file (new-line-at-end-of-file)

dtschema/dtc warnings/errors:
\ndoc reference errors (make refcheckdocs):

See https://patchwork.ozlabs.org/patch/1497480

This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.

If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:

pip3 install dtschema --upgrade

Please check and re-submit.


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
@ 2021-07-01 14:21   ` Kent Gibson
  2021-07-28 23:59     ` Dipen Patel
  2021-07-04 20:27   ` Jonathan Cameron
  2021-07-08 23:33   ` Michał Mirosław
  2 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-07-01 14:21 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
> Tegra194 device has multiple HTE instances also known as GTE
> (Generic hardware Timestamping Engine) which can timestamp subset of
> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> and exposes timestamping ability on those lines to the consumers
> through HTE subsystem.
> 
> Also, with this patch, added:
> - documentation about this provider and its capabilities at
> Documentation/hte.
> - Compilation support in Makefile and Kconfig
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  Documentation/hte/index.rst        |  21 ++
>  Documentation/hte/tegra194-hte.rst |  65 ++++
>  Documentation/index.rst            |   1 +
>  drivers/hte/Kconfig                |  12 +
>  drivers/hte/Makefile               |   1 +
>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>  6 files changed, 654 insertions(+)
>  create mode 100644 Documentation/hte/index.rst
>  create mode 100644 Documentation/hte/tegra194-hte.rst
>  create mode 100644 drivers/hte/hte-tegra194.c
> 
> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
> new file mode 100644
> index 000000000000..f311ebec6b47
> --- /dev/null
> +++ b/Documentation/hte/index.rst
> @@ -0,0 +1,21 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +============================================
> +The Linux Hardware Timestamping Engine (HTE)
> +============================================
> +
> +The HTE Subsystem
> +=================
> +
> +.. toctree::
> +   :maxdepth: 1
> +
> +   hte
> +
> +HTE Tegra Provider
> +==================
> +
> +.. toctree::
> +   :maxdepth: 1
> +
> +   tegra194-hte
> \ No newline at end of file
> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> new file mode 100644
> index 000000000000..c23eaafcf080
> --- /dev/null
> +++ b/Documentation/hte/tegra194-hte.rst
> @@ -0,0 +1,65 @@
> +HTE Kernel provider driver
> +==========================
> +
> +Description
> +-----------
> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
> +known as generic timestamping engine (GTE). This provider driver implements
> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> +driver converts clock tick rate to nano seconds before storing it as timestamp
> +value.
> +
> +GPIO GTE
> +--------
> +
> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
> +needs to be configured as input and IRQ needs to ba enabled as well. The only
> +always on (AON) gpio controller instance supports timestamping GPIOs in
> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
> +controller as it requires very specific bits to be set in GPIO config register.
> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
> +subsystem and GPIO GTE for in kernel consumers.
> +
> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
> +
> +	To enable HTE on given GPIO line.
> +
> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
> +
> +	To retrieve hardwre timestamp in nano seconds.
> +
> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
> +
> +	To query if HTE is enabled on the given GPIO.
> +
> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
> +in nano second.
> +

<snip>

> +
> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
> +{
> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
> +	u64 tsc;
> +	int dir;
> +	struct hte_ts_data el;
> +
> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
> +		HTE_TESTATUS_OCCUPANCY_MASK) {
> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
> +		tsc = (((u64)tsh << 32) | tsl);
> +
> +		src = tegra_hte_readl(gs, HTE_TESRC);
> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
> +
> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
> +		cv = tegra_hte_readl(gs, HTE_TECCV);
> +		acv = pv ^ cv;
> +		while (acv) {
> +			bit_index = __builtin_ctz(acv);
> +			if ((pv >> bit_index) & BIT(0))
> +				dir = HTE_EVENT_RISING_EDGE;
> +			else
> +				dir = HTE_EVENT_FALLING_EDGE;
> +
> +			line_id = bit_index + (slice << 5);
> +			el.dir = dir;
> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
> +					      sizeof(el));
> +			acv &= ~BIT(bit_index);
> +		}
> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
> +	}
> +}

What happens when the hte_push_ts_ns_atomic() fails?
The timestamp will be quietly dropped?
What happens when the interrupt corresponding to that dropped timestamp
asks for it?  The irq handler thread will block until it can get a
timestamp from the subsequent interrupt?

Which brings me back to the concern I have with the approach used in
the hte/gpiolib integration - how do you guarantee that the timestamp
returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
being handled, particularly in the face of errors such as:
 - overflows of the timestamp FIFO in the chip
 - overflows of software FIFOs as here
 - lost interupts (if the hw generates interrupts faster than the CPU
   can service them)
?

Cheers,
Kent.


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 06/11] gpiolib: Add HTE support
  2021-06-25 23:55 ` [RFC 06/11] gpiolib: Add HTE support Dipen Patel
  2021-06-27 11:41   ` Linus Walleij
@ 2021-07-01 14:24   ` Kent Gibson
  2021-07-30  2:25     ` Dipen Patel
  1 sibling, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-07-01 14:24 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Jun 25, 2021 at 04:55:27PM -0700, Dipen Patel wrote:
> Some GPIO chip can provide hardware timestamp support on its GPIO lines
> , in order to support that additional functions needs to be added which
> can talk to both GPIO chip and HTE (hardware timestamping engine)
> subsystem. This patch introduces functions which gpio consumer can use
> to request hardware assisted timestamping. Below is the list of the APIs
> that are added in gpiolib subsystem.
> 
> - gpiod_hw_timestamp_control - to enable/disable HTE on specified GPIO
> line. This API will return HTE specific descriptor for the specified
> GPIO line during the enable call, it will be stored as pointer in the
> gpio_desc structure as hw_ts_data.
> - gpiod_is_hw_timestamp_enabled - to query if HTE is enabled on
> specified GPIO line.
> - gpiod_get_hw_timestamp - to retrieve hardware timestamps.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  drivers/gpio/gpiolib.c        | 92 +++++++++++++++++++++++++++++++++++
>  drivers/gpio/gpiolib.h        | 11 +++++
>  include/linux/gpio/consumer.h | 21 +++++++-
>  include/linux/gpio/driver.h   | 13 +++++
>  4 files changed, 135 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
> index 220a9d8dd4e3..335eaddfde98 100644
> --- a/drivers/gpio/gpiolib.c
> +++ b/drivers/gpio/gpiolib.c
> @@ -2361,6 +2361,98 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
>  }
>  EXPORT_SYMBOL_GPL(gpiod_direction_output);
>  
> +/**
> + * gpiod_hw_timestamp_control - set the hardware assisted timestamp control.
> + * @desc:	GPIO to set
> + * @enable:	Set true to enable the hardware timestamp, false otherwise.
> + *
> + * Certain GPIO chip can rely on hardware assisted timestamp engines which can
> + * record timestamp at the occurance of the configured events on selected GPIO
> + * lines. This is helper API to control such engine.
> + *
> + * Return 0 in case of success, else an error code.
> + */
> +int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable)
> +{
> +	struct gpio_chip	*gc;
> +	int			ret = 0;
> +
> +	VALIDATE_DESC(desc);
> +	gc = desc->gdev->chip;
> +
> +	if (!gc->timestamp_control) {
> +		gpiod_warn(desc,
> +			   "%s: Hardware assisted ts not supported\n",
> +			   __func__);
> +		return -ENOTSUPP;
> +	}
> +
> +	ret = gc->timestamp_control(gc, gpio_chip_hwgpio(desc),
> +				    &desc->hdesc, enable);
> +
> +	if (ret) {
> +		gpiod_warn(desc,
> +			   "%s: ts control operation failed\n", __func__);
> +		return ret;
> +	}
> +
> +	if (!enable)
> +		desc->hdesc = NULL;
> +
> +	return ret;
> +}

Last I checked, pointer accesses are not guaranteed atomic, so how is
hdesc protected from concurrent access?
Here is it modified unprotected.
Below it is read unprotected.

> +EXPORT_SYMBOL_GPL(gpiod_hw_timestamp_control);
> +
> +/**
> + * gpiod_is_hw_timestamp_enabled - check if hardware assisted timestamp is
> + * enabled.
> + * @desc:	GPIO to check
> + *
> + * Return true in case of success, false otherwise.
> + */
> +bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
> +{
> +	if (!desc)
> +		return false;
> +
> +	return (desc->hdesc) ? true : false;
> +}
> +EXPORT_SYMBOL_GPL(gpiod_is_hw_timestamp_enabled);
> +
> +/**
> + * gpiod_get_hw_timestamp - Get hardware timestamp in nano seconds.
> + * @desc:	GPIO to get the timestamp.
> + * @block:	Set true to block until data is available.
> + *
> + * Return non-zero on success, else 0.
> + */
> +u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
> +{
> +	struct gpio_chip	*gc;
> +	int			ret = 0;
> +	u64 ts;
> +
> +	VALIDATE_DESC(desc);
> +	gc = desc->gdev->chip;
> +
> +	if (!gc->get_hw_timestamp) {
> +		gpiod_warn(desc,
> +			   "%s: Hardware assisted ts not supported\n",
> +			   __func__);
> +		return -ENOTSUPP;
> +	}
> +

Can't return an error code here.  Return value is u64, so this will look
like a valid ts.

Just return 0 on error, as you do immediately below...

> +	ret = gc->get_hw_timestamp(gc, block, desc->hdesc, &ts);
> +	if (ret) {
> +		gpiod_warn(desc,
> +			   "%s: get timestamp operation failed\n", __func__);
> +		return 0;
> +	}
> +
> +	return ts;
> +}
> +EXPORT_SYMBOL_GPL(gpiod_get_hw_timestamp);
> +
>  /**
>   * gpiod_set_config - sets @config for a GPIO
>   * @desc: descriptor of the GPIO for which to set the configuration
> diff --git a/drivers/gpio/gpiolib.h b/drivers/gpio/gpiolib.h
> index 30bc3f80f83e..5393e1d90f61 100644
> --- a/drivers/gpio/gpiolib.h
> +++ b/drivers/gpio/gpiolib.h
> @@ -15,6 +15,7 @@
>  #include <linux/device.h>
>  #include <linux/module.h>
>  #include <linux/cdev.h>
> +#include <linux/hte.h>
>  
>  #define GPIOCHIP_NAME	"gpiochip"
>  
> @@ -117,6 +118,7 @@ struct gpio_desc {
>  #define FLAG_EDGE_RISING     16	/* GPIO CDEV detects rising edge events */
>  #define FLAG_EDGE_FALLING    17	/* GPIO CDEV detects falling edge events */
>  #define FLAG_EVENT_CLOCK_REALTIME	18 /* GPIO CDEV reports REALTIME timestamps in events */
> +#define FLAG_EVENT_CLOCK_HARDWARE	19 /* GPIO CDEV reports hardware timestamps in events */
>  
>  	/* Connection label */
>  	const char		*label;
> @@ -129,6 +131,15 @@ struct gpio_desc {
>  	/* debounce period in microseconds */
>  	unsigned int		debounce_period_us;
>  #endif
> +	/*
> +	 * Hardware timestamp engine related internal data structure.
> +	 * This gets set when the consumer calls gpiod_hw_timestamp_control to enable
> +	 * hardware timestamping on the specified GPIO line. The API calls into HTE
> +	 * subsystem, in turns HTE subsystem return the HTE descriptor for the GPIO
> +	 * line. The hdesc will be later used with gpiod_is_hw_timestamp_enabled
> +	 * and gpiod_get_hw_timestamp API calls.
> +	 */
> +	struct hte_ts_desc *hdesc;
>  };
>  
>  #define gpiod_not_found(desc)		(IS_ERR(desc) && PTR_ERR(desc) == -ENOENT)
> diff --git a/include/linux/gpio/consumer.h b/include/linux/gpio/consumer.h
> index c73b25bc9213..476ee04de7d0 100644
> --- a/include/linux/gpio/consumer.h
> +++ b/include/linux/gpio/consumer.h
> @@ -112,6 +112,9 @@ int gpiod_get_direction(struct gpio_desc *desc);
>  int gpiod_direction_input(struct gpio_desc *desc);
>  int gpiod_direction_output(struct gpio_desc *desc, int value);
>  int gpiod_direction_output_raw(struct gpio_desc *desc, int value);
> +int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable);
> +bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc);
> +u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block);
>  
>  /* Value get/set from non-sleeping context */
>  int gpiod_get_value(const struct gpio_desc *desc);
> @@ -353,8 +356,22 @@ static inline int gpiod_direction_output_raw(struct gpio_desc *desc, int value)
>  	WARN_ON(desc);
>  	return -ENOSYS;
>  }
> -
> -
> +static inline int gpiod_hw_timestamp_control(struct gpio_desc *desc,
> +					     bool enable)
> +{
> +	WARN_ON(desc);
> +	return -ENOSYS;
> +}
> +static inline bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
> +{
> +	WARN_ON(desc);
> +	return false;
> +}
> +static inline u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
> +{
> +	WARN_ON(desc);
> +	return 0;
> +}
>  static inline int gpiod_get_value(const struct gpio_desc *desc)
>  {
>  	/* GPIO can never have been requested */
> diff --git a/include/linux/gpio/driver.h b/include/linux/gpio/driver.h
> index 3a268781fcec..f343e8f54b08 100644
> --- a/include/linux/gpio/driver.h
> +++ b/include/linux/gpio/driver.h
> @@ -10,6 +10,7 @@
>  #include <linux/lockdep.h>
>  #include <linux/pinctrl/pinctrl.h>
>  #include <linux/pinctrl/pinconf-generic.h>
> +#include <linux/hte.h> /* For hardware timestamping */
>  
>  struct gpio_desc;
>  struct of_phandle_args;
> @@ -304,6 +305,10 @@ struct gpio_irq_chip {
>   * @add_pin_ranges: optional routine to initialize pin ranges, to be used when
>   *	requires special mapping of the pins that provides GPIO functionality.
>   *	It is called after adding GPIO chip and before adding IRQ chip.
> + * @timestamp_control: Dependent on GPIO chip, an optional routine to
> + * 	enable/disable hardware assisted timestamp.
> + * @get_hw_timestamp: Retrieves hardware timestamp. The consumer can specify
> + * 	block parameter if it wishes to block till timestamp is available. 
>   * @base: identifies the first GPIO number handled by this chip;
>   *	or, if negative during registration, requests dynamic ID allocation.
>   *	DEPRECATION: providing anything non-negative and nailing the base
> @@ -396,6 +401,14 @@ struct gpio_chip {
>  
>  	int			(*add_pin_ranges)(struct gpio_chip *gc);
>  
> +	int			(*timestamp_control)(struct gpio_chip *gc,
> +						     unsigned int offset,
> +						     struct hte_ts_desc **hdesc,
> +						     bool enable);
> +	int			(*get_hw_timestamp)(struct gpio_chip *gc,
> +						    bool block,
> +						    struct hte_ts_desc *hdesc,
> +						    u64 *ts);
>  	int			base;
>  	u16			ngpio;
>  	const char		*const *names;
> -- 
> 2.17.1
> 

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
  2021-06-27 11:38   ` Linus Walleij
  2021-06-27 11:49   ` Linus Walleij
@ 2021-07-01 14:24   ` Kent Gibson
  2021-07-30  3:07     ` Dipen Patel
  2021-07-09  8:30   ` Jon Hunter
  3 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-07-01 14:24 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Jun 25, 2021 at 04:55:29PM -0700, Dipen Patel wrote:
> This patch adds new clock type for the GPIO controller which can
> timestamp gpio lines using hardware means. To expose such
> functionalities to the userspace, code has been added in this patch
> where during line create call, it checks for new clock type and if
> requested, calls hardware timestamp related API from gpiolib.c.
> During line change event, it retrieves timestamp in nano seconds by
> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
> it disables this functionality by calling gpiod_hw_timestamp_control.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>  include/uapi/linux/gpio.h   |  1 +
>  2 files changed, 64 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> index 1631727bf0da..9f98c727e937 100644
> --- a/drivers/gpio/gpiolib-cdev.c
> +++ b/drivers/gpio/gpiolib-cdev.c
> @@ -518,6 +518,7 @@ struct linereq {
>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>  	 GPIO_V2_LINE_BIAS_FLAGS)
>  
>  static void linereq_put_event(struct linereq *lr,
> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>  
>  static u64 line_event_timestamp(struct line *line)
>  {
> +	bool block;
> +
>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>  		return ktime_get_real_ns();
>  
> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> +		if (irq_count())
> +			block = false;
> +		else
> +			block = true;
> +
> +		return gpiod_get_hw_timestamp(line->desc, block);
> +	}
> +

Use in_task() instead of block?

>  	return ktime_get_ns();
>  }
>  
> @@ -828,6 +840,7 @@ static int edge_detector_setup(struct line *line,
>  		return ret;
>  
>  	line->irq = irq;
> +
>  	return 0;
>  }
>  

Remove gratuitous whitespace changes.
If you dislike the formatting then suggest it in a separate patch.

> @@ -891,7 +904,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>  	/* Return an error if an unknown flag is set */
>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>  		return -EINVAL;
> -
>  	/*
>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>  	 * contradictory.
> @@ -900,6 +912,14 @@ static int gpio_v2_line_flags_validate(u64 flags)
>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>  		return -EINVAL;
>  

Same here.

> +	/*
> +	 * Do not mix with any other clocks if hardware assisted timestamp is
> +	 * asked.
> +	 */
> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
> +		return -EINVAL;
> +

The comment is very hw timestamp centric. It should just be something
along the lines of "only allow one event clock source".

>  	/* Edge detection requires explicit input. */
>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
> @@ -992,6 +1012,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>  
>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>  }
>  
>  static long linereq_get_values(struct linereq *lr, void __user *ip)
> @@ -1139,6 +1161,18 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>  			int val = gpio_v2_line_config_output_value(lc, i);
>  
>  			edge_detector_stop(&lr->lines[i]);
> +
> +			/*
> +			 * Assuming line was input before and hardware
> +			 * assisted timestamp only timestamps the input
> +			 * lines.
> +			 */
> +			if (gpiod_is_hw_timestamp_enabled(desc)) {
> +				ret = gpiod_hw_timestamp_control(desc, false);
> +				if (ret)
> +					return ret;
> +			}
> +

So if you fail to disable the hw timestamp then you fail the set_config?
Does that make sense?
It should be impossible to fail, as per the preceding edge_detector_stop(),
or any failure in this context is irrelevant and so can be ignored.

>  			ret = gpiod_direction_output(desc, val);
>  			if (ret)
>  				return ret;
> @@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>  					polarity_change);
>  			if (ret)
>  				return ret;
> +
> +			/* Check if new config sets hardware assisted clock */
> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> +				ret = gpiod_hw_timestamp_control(desc, true);
> +				if (ret)
> +					return ret;
> +			}
>  		}
>  

The error code here can come from the pinctrl timestamp_control(), so it
should be sanitised before being returned to userspace.

>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> @@ -1281,8 +1322,12 @@ static void linereq_free(struct linereq *lr)
>  
>  	for (i = 0; i < lr->num_lines; i++) {
>  		edge_detector_stop(&lr->lines[i]);
> -		if (lr->lines[i].desc)
> +		if (lr->lines[i].desc) {
> +			if (gpiod_is_hw_timestamp_enabled(lr->lines[i].desc))
> +				gpiod_hw_timestamp_control(lr->lines[i].desc,
> +							   false);
>  			gpiod_free(lr->lines[i].desc);
> +		}

Potential race on gpiod_is_hw_timestamp_enabled() and the call to
gpiod_hw_timestamp_control()?
Why not put the gpiod_is_hw_timestamp_enabled() check inside
gpiod_hw_timestamp_control()?

And the gpiod_hw_timestamp_control() call should be moved inside
gpiod_free(), or more correctly gpiod_free_commit().
i.e. whenever you free the gpio you release any associated hw timestamp.

>  	}
>  	kfifo_free(&lr->events);
>  	kfree(lr->label);
> @@ -1409,6 +1454,15 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>  			if (ret)
>  				goto out_free_linereq;
> +
> +			/*
> +			 * Check if hardware assisted timestamp is requested
> +			 */
> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> +				ret = gpiod_hw_timestamp_control(desc, true);
> +				if (ret)
> +					goto out_free_linereq;
> +			}
>  		}
>  

Comment can fit on one line, and probably isn't even necessary - the
code is clear enough.

>  		blocking_notifier_call_chain(&desc->gdev->notifier,
> @@ -1956,8 +2010,15 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>  	if (test_bit(FLAG_EDGE_FALLING, &desc->flags))
>  		info->flags |= GPIO_V2_LINE_FLAG_EDGE_FALLING;
>  
> +	/*
> +	 * Practically it is possible that user will want both the real time
> +	 * and hardware timestamps on GPIO events, for now however lets just
> +	 * work with either clocks
> +	 */
>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>

If there is any need or intent to support multiple clock sources then
avoid creeping API changes and add it now.
Either way, drop the comment.

>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>  	if (debounce_period_us) {
> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
> index eaaea3d8e6b4..d360545b4c21 100644
> --- a/include/uapi/linux/gpio.h
> +++ b/include/uapi/linux/gpio.h
> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>  };
>  
>  /**
> -- 
> 2.17.1
> 

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
  2021-06-27 10:56   ` Linus Walleij
  2021-07-01 14:02   ` Rob Herring
@ 2021-07-01 15:54   ` Rob Herring
  2021-07-30  1:58     ` Dipen Patel
  2 siblings, 1 reply; 75+ messages in thread
From: Rob Herring @ 2021-07-01 15:54 UTC (permalink / raw)
  To: Dipen Patel
  Cc: Thierry Reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Linus Walleij, Bartosz Golaszewski,
	warthog618, devicetree, Linux Doc Mailing List

On Fri, Jun 25, 2021 at 5:48 PM Dipen Patel <dipenp@nvidia.com> wrote:
>
> Introduces HTE devicetree binding details for the HTE subsystem. It
> includes examples for the consumers, binding details for the providers
> and specific binding details for the Tegra194 based HTE providers.
>
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  .../devicetree/bindings/hte/hte-consumer.yaml | 47 +++++++++++
>  .../devicetree/bindings/hte/hte.yaml          | 34 ++++++++
>  .../bindings/hte/nvidia,tegra194-hte.yaml     | 83 +++++++++++++++++++
>  3 files changed, 164 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>
> diff --git a/Documentation/devicetree/bindings/hte/hte-consumer.yaml b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
> new file mode 100644
> index 000000000000..79ae1f7d5185
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
> @@ -0,0 +1,47 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/hte/hte-consumer.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: HTE Consumer Device Tree Bindings
> +
> +maintainers:
> +  - Dipen Patel <dipenp@nvidia.com>
> +
> +description: |
> +  HTE properties should be named "htes". The exact meaning of each htes
> +  property must be documented in the device tree binding for each device.
> +  An optional property "hte-names" may contain a list of strings to label
> +  each of the HTE devices listed in the "htes" property.
> +
> +  The "hte-names" property if specified is used to map the name of the HTE
> +  device requested by the devm_of_hte_request_ts() or of_hte_request_ts
> +  call to an index into the list given by the "htes" property.
> +
> +properties:
> +  htes:
> +    $ref: /schemas/types.yaml#/definitions/phandle-array
> +    description:
> +      The list of HTE provider phandle. The provider must document the number
> +      of cell that must be passed in this property along with phandle.
> +
> +  hte-names:
> +    $ref: /schemas/types.yaml#/definitions/string-array
> +    description:
> +      An optional string property.
> +
> +required:
> +  - "htes"
> +
> +dependencies:
> +  hte-names: [ htes ]
> +
> +additionalProperties: true
> +
> +examples:
> +  - |
> +    hte_irq_consumer {
> +              htes = <&tegra_hte_lic 0x19>;
> +              hte-names = "hte-irq";
> +    };
> diff --git a/Documentation/devicetree/bindings/hte/hte.yaml b/Documentation/devicetree/bindings/hte/hte.yaml
> new file mode 100644
> index 000000000000..e285c38f1a05
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/hte/hte.yaml
> @@ -0,0 +1,34 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/hte/hte.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: HTE providers
> +
> +maintainers:
> +  - Dipen Patel <dipenp@nvidia.com>
> +
> +properties:
> +  $nodename:
> +    pattern: "^hte(@.*|-[0-9a-f])*$"
> +
> +  "#hte-cells":
> +    description:
> +      Number of cells in a HTE specifier.
> +
> +required:
> +  - "#hte-cells"
> +
> +additionalProperties: true
> +
> +examples:
> +  - |
> +    tegra_hte_aon: hte@c1e0000 {
> +              compatible = "nvidia,tegra194-gte-aon";
> +              reg = <0xc1e0000 0x10000>;
> +              interrupts = <0 13 0x4>;
> +              int-threshold = <1>;
> +              slices = <3>;
> +              #hte-cells = <1>;
> +    };
> \ No newline at end of file
> diff --git a/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
> new file mode 100644
> index 000000000000..bb76cc1971f0
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
> @@ -0,0 +1,83 @@
> +# SPDX-License-Identifier: GPL-2.0
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Tegra194 on chip generic hardware timestamping engine (HTE)

I had to read until here to know what HTE is.

Is there another example of this type of h/w that this should be a
generic binding?

Rob

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 01/11] Documentation: Add HTE subsystem guide
  2021-06-25 23:55 ` [RFC 01/11] Documentation: Add HTE subsystem guide Dipen Patel
@ 2021-07-04 18:55   ` Jonathan Cameron
  2021-07-27 23:44     ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Jonathan Cameron @ 2021-07-04 18:55 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Fri, 25 Jun 2021 16:55:22 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> Adding hte document which can help understand various APIs implemented
> in HTE framework for the HTE producers and the consumers.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
Some editorial stuff inline. (I can't resist even on RFCs)

Certainly interesting. I'm running a bit tight on time today, so not sure how
much of the code I'll get a chance to look at.  Will try to get to it soon though.

Jonathan

> ---
>  Documentation/hte/hte.rst | 198 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 198 insertions(+)
>  create mode 100644 Documentation/hte/hte.rst
> 
> diff --git a/Documentation/hte/hte.rst b/Documentation/hte/hte.rst
> new file mode 100644
> index 000000000000..11744dbc6d16
> --- /dev/null
> +++ b/Documentation/hte/hte.rst
> @@ -0,0 +1,198 @@
> +============================================
> +The Linux Hardware Timestamping Engine (HTE)
> +============================================
> +
> +:Author: Dipen Patel
> +
> +Introduction
> +------------
> +
> +The certain devices have the built in hardware timestamping engine which can

Certain devices have built in hardware timestamping engines which can

> +monitor sets of system signals, lines, buses etc... in realtime for the state

for state changes;

> +change; upon detecting the change it can automatically store the timestamp at

they can

> +the moment of occurrence. Such functionality may help achieve better accuracy
> +in obtaining timestamp than using software counterparts i.e. ktime and friends.
> +
> +This document describes the API that can be used by hardware timestamping
> +engine provider and consumer drivers that want to use the hardware timestamping
> +engine (HTE) framework.
> +
> +The HTE framework APIs for the providers
> +----------------------------------------
> +Each driver must #include <linux/hte.h>. The ``linux/hte.h`` declares the
> +following functions for the provider:
> +
> +.. c:function:: int hte_register_chip( struct hte_chip *chip )
> +		int hte_unregister_chip( struct hte_chip *chip )
> +
> +	The provider uses these APIs to un/register itself with HTE framework.
> +
> +.. c:function:: int hte_push_ts_ns_atomic( const struct hte_chip *chip, u32 xlated_id, struct hte_ts_data *data, size_t n )
> +
> +	The provider pushes timestamp data in nano seconds unit using this API.
> +
> +The detail about parameters and API usage are described in each functions
> +definitions in ``drivers/hte/hte.c`` file.
> +
> +The HTE framework APIs for the consumers
> +----------------------------------------
> +The consumers use following APIs to control the line for the timestamp:
> +

When documenting APIs you may well be better including a reference to the files
themselves and using kernel doc there.  The documentation build can then pull that
in when creating the html docs etc (and crucially you don't have to provide the
same docs in two places.).   Having them here is very convenient for the RFC however :)

> +.. c:function:: int hte_release_ts( struct hte_ts_desc *desc )
> +		int devm_hte_release_ts( struct device *dev, struct hte_ts_desc *desc )
> +
> +	The consumer uses API to release specified desc from timestamping.
> +	The API frees resources associated with the desc and disables the
> +	timestamping on it. The later is managed version of the same API.
> +
> +.. c:function:: struct hte_ts_desc *of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
> +		struct hte_ts_desc *devm_of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
> +
> +	The consumers can use above request APIs to request real timestamp
> +	capability on specified entity. The later is resource managed version
> +	of the of_hte_request_ts API. Both the APIs expect consumer to follow
> +	device tree bindings for the HTE consumer. The details about binding
> +	is in ``Documentation/devicetree/bindings/hte/hte-consumer.yaml``.
> +
> +.. c:function:: struct hte_ts_desc *hte_req_ts_by_dt_node( struct device_node *of_node, unsigned int id, void (*cb)(enum hte_notify n) )
> +
> +	The consumer can request timestamping directly specifying provider
> +	device tree node.

When does this make sense?

> +
> +.. c:function:: int hte_enable_ts( struct hte_ts_desc *desc )
> +.. c:function:: int hte_disable_ts( struct hte_ts_desc *desc )
> +
> +	The consumer can enable/disable timestamping on given desc.
> +
> +.. c:function:: int hte_retrieve_ts_ns( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
> +		int hte_retrieve_ts_ns_wait( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
> +
> +	The consumer uses above two API versions to get/retrieve timestamp data
> +	for the given desc. The later is blocking version.
> +
> +.. c:function:: hte_get_clk_src_info(const struct hte_line_desc *desc, struct hte_clk_info *ci)
> +
> +	The consumer retrieves clock source information that provider uses to
> +	timestamp entity in the structure hte_clk_info. This information
> +	specifies clock rate in HZ and clock.
> +
> +The details on struct hte_clk_info
> +-----------------------------------
> +This structure presents detail of the hardware clock that provider uses for
> +realtime timestamping purposes. The consumer can use hte_get_clk_src_info API
> +to get the information in hte_clk_info structure. It has hz and type parameters
> +where hz represents clock rate in HZ and type is clock type of clockid_t and
> +of CLOCK_* family (for example, CLOCK_MONOTONIC).
> +
> +The consumers calling of_hte_request_ts or hte_req_ts_by_dt_node APIs with
> +cb parameter set, usually will call hte_retrieve_ts (non blocking
> +version) after being notified by the callbacks from HTE subsystem. The
> +consumers calling those requests APIs with cb parameter NULL, usually will call
> +hte_retrieve_ts_wait API.
> +
> +The HTE subsystem provides software buffer per requested id/entity to store
> +timestamp data (struct hte_ts_data type). The consumers can manage the buffer.
> +It also provides buffer watermark which can notify (if cb parameter is provided
> +during request API call) consumer or unblock consumers calling
> +hte_retrieve_ts_wait API. The following APIs are used to manipulate the
> +software buffer:

Have you come across any devices that have a hardware fifo for these timestamps?
It's moderately common on sensor hubs to do so, and then you get into a fun question
of how to manage the watermark.  You don't want to pull from the hardware too early,
but conversely you can get out of sync between the software and hardware buffers if
someone reasons less than 'watermark' samples from the software buffer.

Anyhow, it can be entertaining.  So in those cases it can be simpler to explicitly provide
control of two separate watermarks.

> +
> +.. c:function:: int hte_set_buf_len( const struct hte_ts_desc *desc,unsigned int len )
> +		int hte_get_buf_len( const struct hte_ts_desc *desc )
> +
> +	The consumer uses above APIs to set/get software buffer depth.

What happens if there is content when it is resized?

> +
> +.. c:function:: int hte_set_buf_watermark( const struct hte_ts_desc *desc, unsigned int val )
> +		int hte_get_buf_watermark( const struct hte_ts_desc *desc )
> +
> +	The consumer uses above APIs to set/get software threshold, threshold
> +	can be used to notity or unblock waiting consumer when data becomes
> +	available equal or above to threshold value.
> +
> +.. c:function:: size_t hte_available_ts( const struct hte_ts_desc *desc )
> +
> +	The consumer uses above API to get available timestamp data stored
> +	in the software buffer for the desc.
> +
> +The detail about parameters and API usage are described in each functions
> +definitions in ``drivers/hte/hte.c`` file.
> +
> +The HTE timestamp element detail
> +--------------------------------
> +The struct hte_ts_data, declared at ``include/linux/hte.h``, is used to pass
> +timestamp details between the consumers and the providers. It expresses
> +timestamp data in nano second in u64 data type.

I'd suggest s64 to match with kernel timestamp format.

> For now all the HTE APIs
> +using struct hte_ts_data requires tsc to be in nano seconds. The timestamp
> +element structure stores below information along with timestamp data::
> +
> + struct hte_ts_data {
> +	/*
> +	 * Timestamp value
> +	 */
> +	u64 tsc;
> +	/*
> +	 * The sequence counter, keep track of the number of timestamps.
> +	 * It can be used to check if data is dropped in between.
> +	 */

Is this a hardware feature?  A bit unusual to have this rather than simple
overflow flag to indicate we dropped an unknown number of samples.

> +	u64 seq;
> +	/* Direction of the event, i.e. falling or rising */
> +	int dir;

Given an even could do more than that potentially, or indeed not be able to
tell if it was rising or falling, I would suggest an enum to which we can add
more options as needed.

> + };
> +
> +The typical hte_ts_data data life cycle::
> +In this example the provider provides timestamp in nano seconds and for the
> +GPIO line::
> +
> + - Monitors GPIO line change.
> + - Detects the state change on GPIO line.
> + - Converts timestamps in nano seconds and stores it in tsc.
> + - Stores GPIO direction in dir variable if the provider has that hardware
> + capability.

We definitely want to know if it does or not.  How does an application query that?

> + - Pushes this hte_timestamp_el object to HTE subsystem.
> + - HTE subsystem increments seq counter and stores it in software buffer
> + dedicated to requested GPIO line.

Ah. So that seq counter is only for software drops if the fifo fills up.

> + - Waiting consumer gets notified.
> + - The consumer calls the retrieve timestamp API.
> +
> +HTE subsystem debugfs attributes
> +--------------------------------
> +HTE subsystem creates debugfs attributes at ``/sys/kernel/debug/hte/``.
> +It also creates line/signal related debugfs attributes at
> +``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> +
> +`ts_requested`
> +		The total number of entities requested from the given provider,
> +		where entity is the provider specific and could represent
> +		lines, GPIO, chip signals, buses etc...
> +                The attribute will be availble at
> +		``/sys/kernel/debug/hte/<provider>/``.
> +
> +		Read only value
> +
> +`total_ts`
> +		The total number of entities supported by the provider.
> +                The attribute will be availble at
> +		``/sys/kernel/debug/hte/<provider>/``.
> +
> +		Read only value
> +
> +`ts_buffer_depth`
> +		The software buffer lenth to store timestamp data.
> +                The attribute will be availble at
> +		``/sys/kernel/debug/hte/<provider>/<label or id>/``.
> +
> +		Read only value
> +
> +`ts_buffer_watermark`
> +		The software buffer watermark or threshold.
> +                The attribute will be availble at
> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> +
> +		Read only value
> +
> +`dropped_timestamps`
> +		The dropped timestamps for a given line.
> +                The attribute will be availble at
> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> +
> +		Read only value


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
  2021-06-27 17:40   ` Randy Dunlap
@ 2021-07-04 20:15   ` Jonathan Cameron
  2021-07-04 20:45     ` Jonathan Cameron
  2021-07-28  4:38     ` Dipen Patel
  2021-07-05  7:30   ` Greg KH
  2 siblings, 2 replies; 75+ messages in thread
From: Jonathan Cameron @ 2021-07-04 20:15 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Fri, 25 Jun 2021 16:55:23 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> Some devices can timestamp system lines/signals/Buses in real-time
> using the hardware counter or other hardware means which can give
> finer granularity and help avoid jitter introduced by software means
> of timestamping. To utilize such functionality there has to be
> framework where such devices can register themselves as producers or
> providers so that the consumers or clients devices can request specific
> line from the providers. This patch introduces such subsystem as
> hardware timestamping engine (HTE).
> 
> It provides below APIs for the provider:
> - hte_register_chip() -- To register the HTE chip.
> - hte_unregister_chip() -- To unregister the HTE chip.
> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
> 
> It provides below APIs for the consumer:
> - of_hte_request_ts() -- To request timestamp functionality.
> - devm_of_hte_request_ts() -- Managed version of the above.
> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
> using HTE provider dt node.
> - devm_hte_release_ts() -- The managed version to release timestamp
> functionality and associated resources.
> - hte_retrieve_ts_ns() -- To retrieve timestamps.
> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
> - hte_enable_ts() -- To disable timestamp functionality.
> - hte_disable_ts() -- To enable timestamp functionality.
> - hte_available_ts() -- To query available timestamp data.
> - hte_release_ts() -- To release timestamp functionality and its
> associated resources.
> - hte_get_clk_src_info() -- To query clock source information from
> the provider
> 
> It provides centralized software buffer management per requested id to
> store the timestamp data for the consumers as below:
> - hte_set_buf_len() -- To set the buffer length.
> - hte_get_buf_len() -- To get the buffer length.
> - hte_set_buf_watermark() -- To set the software threshold/watermark.
> - hte_get_buf_watermark() -- To get the software threshold/watermark.
> 
> The detail about parameters and API usage are described in each
> functions definitions in drivers/hte/hte.c file.
> 
> The patch adds compilation support in Makefile and menu options in
> Kconfig.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>

Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
around what this is doing + it is an RFC :)

> ---
>  drivers/Kconfig      |    2 +
>  drivers/Makefile     |    1 +
>  drivers/hte/Kconfig  |   22 +
>  drivers/hte/Makefile |    1 +
>  drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
>  include/linux/hte.h  |  278 +++++++++
>  6 files changed, 1672 insertions(+)
>  create mode 100644 drivers/hte/Kconfig
>  create mode 100644 drivers/hte/Makefile
>  create mode 100644 drivers/hte/hte.c
>  create mode 100644 include/linux/hte.h
> 
> diff --git a/drivers/Kconfig b/drivers/Kconfig
> index 47980c6b1945..9b078964974b 100644
> --- a/drivers/Kconfig
> +++ b/drivers/Kconfig
> @@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
>  source "drivers/counter/Kconfig"
>  
>  source "drivers/most/Kconfig"
> +
> +source "drivers/hte/Kconfig"
>  endmenu
> diff --git a/drivers/Makefile b/drivers/Makefile
> index 5a6d613e868d..0a996a698e4c 100644
> --- a/drivers/Makefile
> +++ b/drivers/Makefile
> @@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
>  obj-$(CONFIG_COUNTER)		+= counter/
>  obj-$(CONFIG_MOST)		+= most/
> +obj-$(CONFIG_HTE)		+= hte/
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> new file mode 100644
> index 000000000000..394e112f7dfb
> --- /dev/null
> +++ b/drivers/hte/Kconfig
> @@ -0,0 +1,22 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig HTE
> +        bool "Hardware Timestamping Engine (HTE) Support"
> +        help
> +          Hardware Timestamping Engine (HTE) Support.

Tidy this up, but think that's already been commented on.

> +
> +          Some devices provide hardware timestamping engine which can timestamp
> +	  certain device lines/signals in realtime. This way to provide
> +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
> +	  comes with benefit for the applications like autonomous machines
> +	  needing accurate timestamping event with less jitter.
> +
> +	  This framework provides a generic interface to such HTE devices
> +          within the Linux kernel. It provides an API to register and
> +	  unregister a HTE provider chip, configurable sw buffer to
> +	  store the timestamps, push the timestamp from the HTE providers and
> +	  retrieve timestamps for the consumers. It also provides means for the
> +	  consumers to request signals it wishes to hardware timestamp and
> +	  release them if not required.
> +
> +          If unsure, say no.
> +
> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
> new file mode 100644
> index 000000000000..9899dbe516f7
> --- /dev/null
> +++ b/drivers/hte/Makefile
> @@ -0,0 +1 @@
> +obj-$(CONFIG_HTE)		+= hte.o
> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
> new file mode 100644
> index 000000000000..c53260d1e250
> --- /dev/null
> +++ b/drivers/hte/hte.c
> @@ -0,0 +1,1368 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 NVIDIA Corporation
> + *
> + * Author: Dipen Patel <dipenp@nvidia.com>
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/err.h>
> +#include <linux/slab.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/kfifo.h>
> +#include <linux/mutex.h>
> +#include <linux/sched.h>
> +#include <linux/uaccess.h>
> +#include <linux/hte.h>
> +#include <linux/delay.h>
> +#include <linux/debugfs.h>
> +
> +/* Global list of the HTE devices */
> +static DEFINE_SPINLOCK(hte_lock);
> +static LIST_HEAD(hte_devices);
> +
> +enum {
> +	HTE_TS_REGISTERED,
> +	HTE_TS_DISABLE,
> +};
> +
> +/* Default FIFO depth */
> +#define HTE_EV_FIFO_EL		32
> +
> +#define HTE_TS_NAME_LEN		10
> +
> +struct hte_ts_buf;
> +
> +/**
> + * struct hte_ts_buf_acc_func - Software buffer management functions.
> + * @store: Store timestamp from atomic context as providers most likely
> + * be pushing timestamps from their interrupt handlers.
> + * @read: Read timestamps from the buffer.
> + * @el_available: Available timestamps to retrieve. The client can use this to
> + * query available elements so that it can pre-allocate internal buffer to send
> + * to during hte_retrieve_ts_ns API.
> + * @set_length: Set length/depth of the buffer.
> + * @get_length: Get length/depth of the buffer.
> + * @set_watermark: Set software threshold of the buffer.
> + * @get_watermark: Get software threshold of the buffer.
> + * @release: Release/free buffer.
> + * @reset: Reset the buffer.
> + */
> +struct hte_ts_buf_acc_func {
> +	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
> +	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
> +		    size_t *copied);
> +	size_t (*el_available)(struct hte_ts_buf *buf);
> +	int (*set_length)(struct hte_ts_buf *buf,
> +			  size_t length, size_t bpd);
> +	size_t (*get_length)(struct hte_ts_buf *buf);
> +	int (*set_watermark)(struct hte_ts_buf *buf,
> +			     size_t val);
> +	size_t (*get_watermark)(struct hte_ts_buf *buf);
> +	void (*release)(struct hte_ts_buf *buf);
> +	void (*reset)(struct hte_ts_buf *buf);
> +};
> +
> +/**
> + * struct hte_ts_buf - Software buffer per requested id or entity to store
> + * timestamps.
> + *
> + * @datum_len: Buffer depth or number of elements.
> + * @bytes_per_datum: Element size in bytes.
> + * @watermark: Software threshold at which client will be notified.
> + * @valid: Validity of the buffer.
> + * @pollq: Waitqueue for the blocking clients.
> + * @access: Various buffer management functions.
> + */
> +struct hte_ts_buf {
> +	size_t datum_len;
> +	size_t bytes_per_datum;
> +	size_t watermark;
> +	bool valid;
> +	wait_queue_head_t pollq;
> +	const struct hte_ts_buf_acc_func *access;
> +};
> +
> +/**
> + * struct hte_ts_info - Information related to requested timestamp.
> + *
> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
> + * See xlate callback API.
> + * @flags: Flags holding state informations.
> + * @seq: Timestamp sequence counter.
> + * @dropped_ts: Dropped timestamps.
> + * @cb: Callback to notify clients.
> + * @mlock: Lock during timestamp request/release APIs.
> + * @ts_dbg_root: Root for the debug fs.
> + * @gdev: HTE abstract device that this timestamp belongs to.
> + * @buf: Per requested timestamp software buffer.
> + * @desc: Timestamp descriptor understood between clients and HTE subsystem.
> + */
> +struct hte_ts_info {
> +	u32 xlated_id;
> +	unsigned long flags;
> +	u64 seq;
> +	atomic_t dropped_ts;
> +	void (*cb)(enum hte_notify n);
> +	struct mutex mlock;
> +	struct dentry *ts_dbg_root;
> +	struct hte_device *gdev;
> +	struct hte_ts_buf *buf;
> +	struct hte_ts_desc *desc;
> +};
> +
> +/**
> + * struct hte_device - HTE abstract device
> + * @nlines: Number of entities this device supports.
> + * @ts_req: Total number of entities requested.
> + * @ei: Timestamp information.
> + * @sdev: Device used at various debug prints.
> + * @dbg_root: Root directory for debug fs.
> + * @list: List node for internal use.

Be more specific of what sort of internal use.

> + * @chip: HTE chip providing this HTE device.
> + * @owner: helps prevent removal of modules when in use.
> + */
> +struct hte_device {
> +	u32 nlines;
> +	atomic_t ts_req;
> +	struct hte_ts_info *ei;
> +	struct device *sdev;
> +	struct dentry *dbg_root;
> +	struct list_head list;
> +	struct hte_chip *chip;
> +	struct module *owner;
> +};
> +
> +/* Buffer management functions */
> +
> +/**
> + * struct hte_kfifo - Software buffer wrapper.
> + * @buffer: Abstract buffer device.
> + * @gkf: Actual software buffer type, this case its FIFO.
> + */
> +struct hte_kfifo {
> +	struct hte_ts_buf buffer;
> +	struct kfifo gkf;
> +};
> +
> +#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
> +
> +static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
> +					size_t n)
> +{
> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> +
> +	if (unlikely(!r->valid))
> +		return 0;
> +
> +	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
> +}
> +
> +static inline int hte_ts_buf_read(struct hte_ts_buf *r,
> +				  unsigned char *buf, size_t n,
> +				  size_t *copied)
> +{
> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> +
> +	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
> +		return -EINVAL;
> +
> +	*copied = kfifo_out(&kf->gkf, buf, n);
> +
> +	return 0;
> +}
> +
> +static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
> +{
> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> +
> +	if (!r->valid)
> +		return 0;
> +
> +	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
> +}
> +
> +static int hte_ts_buf_set_length(struct hte_ts_buf *r,
> +				 size_t length, size_t bpd)
> +{
> +	int ret = 0;
> +	struct hte_kfifo *buf;
> +
> +	if ((length == 0) || (bpd == 0) || !r)
> +		return -EINVAL;
> +
> +	buf = buf_to_kfifo(r);
> +
> +	if (r->datum_len != length) {
> +		if (r->valid)
> +			kfifo_free(&buf->gkf);
> +		r->valid = false;
> +		r->datum_len = length;
> +		r->bytes_per_datum = bpd;
> +		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
> +		if (!ret)
> +			r->valid = true;
> +	}
> +
> +	return ret;
> +}
> +
> +static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
> +{
> +	if ((!r->valid) || !r->datum_len)
> +		return 0;
> +
> +	return r->datum_len;
> +}
> +
> +static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
> +{
> +	if ((!r->valid) || (val > r->datum_len))
> +		return -EINVAL;
> +
> +	r->watermark = val;
> +
> +	return 0;
> +}
> +
> +static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
> +{
> +	if (!r->valid)
> +		return 0;
> +
> +	return r->watermark;
> +}
> +
> +static inline void hte_ts_buf_release(struct hte_ts_buf *r)
> +{
> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> +
> +	r->valid = false;
> +	kfifo_free(&kf->gkf);
> +	kfree(kf);
> +}
> +
> +static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
> +{
> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> +
> +	if (!r->valid)
> +		return;
> +
> +	kfifo_reset(&kf->gkf);
> +}
> +
> +static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
> +	.store = &hte_ts_store_to_buf,
> +	.read = &hte_ts_buf_read,
> +	.el_available = &hte_ts_buf_el_available,
> +	.set_length = &hte_ts_buf_set_length,
> +	.get_length = &hte_ts_buf_get_length,
> +	.set_watermark = &hte_ts_buf_set_watermark,
> +	.get_watermark = &hte_ts_buf_get_watermark,
> +	.release = &hte_ts_buf_release,
> +	.reset = &hte_ts_buf_reset,
> +};
> +
> +static struct hte_ts_buf *hte_ts_buf_allocate(void)
> +{
> +	struct hte_kfifo *kf;
> +
> +	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
> +	if (!kf)
> +		return ERR_PTR(-ENOMEM);
> +
> +	init_waitqueue_head(&kf->buffer.pollq);
> +	kf->buffer.watermark = 1;
> +	kf->buffer.datum_len = 0;
> +	kf->buffer.valid = false;
> +	kf->buffer.access = &kfifo_access_funcs;
> +
> +	return &kf->buffer;
> +}
> +/* End of buffer management */
> +
> +/* Debugfs management */
> +
> +#ifdef CONFIG_DEBUG_FS
> +
> +static struct dentry *hte_root;
> +
> +static void __init hte_subsys_dbgfs_init(void)
> +{
> +	/* creates /sys/kernel/debug/hte/ */
> +	hte_root = debugfs_create_dir("hte", NULL);
> +}
> +subsys_initcall(hte_subsys_dbgfs_init);
> +
> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> +{
> +	const struct hte_chip *chip = gdev->chip;
> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
> +
> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
> +	if (!gdev->dbg_root)
> +		return;
> +
> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
> +				&gdev->ts_req);
> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
> +			   &gdev->nlines);
> +}
> +
> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> +{
> +	if (!ei->gdev->dbg_root || !name)
> +		return;
> +
> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
> +	if (!ei->ts_dbg_root)
> +		return;
> +
> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
> +			      &ei->buf->datum_len);
> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
> +			      &ei->buf->watermark);
> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
> +				&ei->dropped_ts);
> +}
> +
> +static inline void hte_dbgfs_deinit(struct dentry *root)
> +{
> +	if (!root)
> +		return;
> +
> +	debugfs_remove_recursive(root);
> +}
> +
> +#else
> +
> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> +{
> +}
> +
> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> +{
> +}
> +
> +static inline void hte_dbgfs_deinit(struct dentry *root)
> +{
> +}
> +
> +#endif
> +/* end of debugfs management*/
> +
> +/* Driver APIs */
> +
> +/**
> + * hte_release_ts() - Consumer calls this API to release the entity, where
> + * entity could be anything providers support, like lines, signals, buses,
> + * etc...
> + *
> + * The correct sequence to call this API is as below:
> + * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
> + * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
> + * still care about the data.
> + * 3) Call this API.
> + * Above sequence makes sure that entity gets released race free.
> + *
> + * @desc: timestamp descriptor, this is the same as returned by the request API.
> + *
> + * Context: hte_dbgfs_deinit() function call may use sleeping locks,
> + *	    not suitable from atomic context in that case.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_release_ts(struct hte_ts_desc *desc)
> +{
> +	u32 id;
> +	int ret = 0;
> +	struct hte_device *gdev;
> +	struct hte_ts_info *ei;
> +	struct hte_ts_buf *buf;
> +
> +	if (!desc)
> +		return -EINVAL;
> +
> +	ei = (struct hte_ts_info *)desc->data_subsys;

As data_subsys is void * you don't need to explicitly cast it to another pointer type.

> +
> +	if (!ei || !ei->gdev || !ei->buf)
> +		return -EINVAL;
> +
> +	gdev = ei->gdev;
> +	buf = ei->buf;
> +	id = desc->con_id;
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_info(gdev->sdev, "id:%d is not registered", id);
> +		return -EUSERS;
> +	}
> +
> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
> +	if (ret) {
> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
> +		goto out;
> +	}
> +
> +	atomic_dec(&gdev->ts_req);
> +	atomic_set(&ei->dropped_ts, 0);
> +
> +	kfree(desc->name);
> +	kfree(desc);
> +	ei->desc = NULL;
> +	ei->seq = 0;
> +	buf->access->release(buf);
> +
> +	hte_dbgfs_deinit(ei->ts_dbg_root);
> +	module_put(gdev->owner);
> +
> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
> +
> +out:
> +	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hte_release_ts);
> +
> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
> +{
> +	u32 ts_id;
> +	struct hte_device *gdev;
> +	struct hte_ts_info *ei;
> +	int ret;
> +
> +	if (!desc)
> +		return -EINVAL;
> +
> +	ei = (struct hte_ts_info *)desc->data_subsys;

As above, no need to cast - though it rather implies the type of data_subsys
should not be void *.

> +
> +	if (!ei || !ei->gdev)
> +		return -EINVAL;
> +
> +	gdev = ei->gdev;
> +	ts_id = desc->con_id;
> +
> +	mutex_lock(&ei->mlock);
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
> +		ret = -EUSERS;
> +		goto out;
> +	}
> +
> +	if (en) {
> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +			ret = 0;
> +			goto out;
> +		}
> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
> +		if (ret) {
> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
> +				 ts_id);
> +			goto out;
> +		}
> +
> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
> +		ret = 0;

ret is already 0 so no point in setting it again.

> +	} else {
> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +			ret = 0;
> +			goto out;
> +		}
> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
> +		if (ret) {
> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
> +				 ts_id);
> +			goto out;
> +		}
> +
> +		set_bit(HTE_TS_DISABLE, &ei->flags);
> +		ret = 0;
> +	}
> +
> +out:
> +	mutex_unlock(&ei->mlock);
> +	return ret;
> +}
> +
> +/**
> + * hte_disable_ts() - Disable timestamp on given descriptor.
> + *
> + * @desc: ts descriptor, this is the same as returned by the request API.
> + *
> + * Context: Holds mutex lock, not suitable from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_disable_ts(struct hte_ts_desc *desc)
> +{
> +	return hte_ts_dis_en_common(desc, false);
> +}
> +EXPORT_SYMBOL_GPL(hte_disable_ts);
> +
> +/**
> + * hte_enable_ts() - Enable timestamp on given descriptor.
> + *
> + * @desc: ts descriptor, this is the same as returned by the request API.
> + *
> + * Context: Holds mutex lock, not suitable from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_enable_ts(struct hte_ts_desc *desc)
> +{
> +	return hte_ts_dis_en_common(desc, true);
> +}
> +EXPORT_SYMBOL_GPL(hte_enable_ts);
> +
> +static int hte_simple_xlate(struct hte_chip *gc,
> +			    const struct of_phandle_args *args,
> +			    struct hte_ts_desc *desc,
> +			    u32 *id)
> +{
> +	if (!id || !desc || !gc)
> +		return -EINVAL;
> +
> +	/*
> +	 * For the providers which do not have any internal mappings between
> +	 * logically exposed ids and actual ids, will set both
> +	 * the same.
> +	 *
> +	 * In case there is a internal mapping needed, providers will need to
> +	 * provide its own xlate function where con_id will be sent as
> +	 * args[0] and it will return xlated id. Later xlated id will be
> +	 * used for any future exchanges between provider and subsystems.
> +	 */
> +
> +	if (args) {
> +		if (gc->of_hte_n_cells < 1)
> +			return -EINVAL;
> +
> +		if (args->args_count != gc->of_hte_n_cells)
> +			return -EINVAL;
> +
> +		*id = args->args[0];
> +		desc->con_id = *id;
> +	} else {
> +		*id = desc->con_id;
> +	}
> +
> +	if (desc->con_id > gc->nlines)
> +		return -EINVAL;
> +
> +	desc->data_subsys = NULL;
> +
> +	return 0;
> +}
> +
> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
> +{
> +	struct hte_device *gdev;
> +
> +	spin_lock(&hte_lock);
> +
> +	list_for_each_entry(gdev, &hte_devices, list)
> +		if (gdev->chip && gdev->chip->dev &&
> +		    gdev->chip->dev->of_node == np) {
> +			spin_unlock(&hte_lock);
> +			return gdev;
> +		}
> +
> +	spin_unlock(&hte_lock);
> +
> +	return ERR_PTR(-ENODEV);
> +}
> +
> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
> +			 u32 xlated_id, void (*cb)(enum hte_notify n))
> +{
> +	struct hte_ts_info *ei;
> +	struct hte_ts_buf *buf;
> +	int ret;
> +	u32 con_id = desc->con_id;
> +
> +	if (!try_module_get(gdev->owner))
> +		return -ENODEV;
> +
> +	ei = &gdev->ei[xlated_id];
> +	ei->xlated_id = xlated_id;
> +
> +	/*
> +	 * There a chance that multiple consumers requesting same entity,
> +	 * lock here.
> +	 */
> +	mutex_lock(&ei->mlock);
> +
> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
> +			xlated_id);
> +		ret = -EUSERS;
> +		goto unlock;
> +	}
> +
> +	buf = hte_ts_buf_allocate();
> +	if (IS_ERR(buf)) {
> +		dev_err(gdev->chip->dev, "Buffer allocation failed");
> +		ret = PTR_ERR(buf);
> +		goto unlock;
> +	}
> +
> +	/* Set default here, let consumer decide how much to set later */
> +	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
> +				      sizeof(struct hte_ts_data));
> +

It's good to keep to consistent style of no line break between a statement
and it's error check.

> +	if (ret) {
> +		dev_err(gdev->chip->dev, "Fifo set length failed");
> +		goto buf_rel;
> +	}
> +
> +	buf->access->reset(buf);
> +	buf->valid = true;
> +
> +	ei->buf = buf;
> +	ei->cb = cb;
> +
> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev, "ts request failed\n");
> +		goto buf_rel;
> +	}
> +
> +	desc->data_subsys = ei;
> +	ei->desc = desc;
> +
> +	atomic_inc(&gdev->ts_req);
> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
> +	mutex_unlock(&ei->mlock);
> +
> +	if (!desc->name) {
> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
> +		if (desc->name)
> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
> +				  con_id);
> +	}
> +
> +	hte_ts_dbgfs_init(desc->name, ei);
> +
> +	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
> +		__func__, con_id, xlated_id);
> +
> +	return 0;
> +
> +buf_rel:
> +	buf->access->release(buf);
> +unlock:
> +	module_put(gdev->owner);
> +	mutex_unlock(&ei->mlock);
> +
> +	return ret;
> +}
> +
> +static struct hte_device *of_hte_dev_get(struct device *dev,
> +					 struct device_node *np,
> +					 const char *label,
> +					 struct of_phandle_args *args)
> +{
> +	struct hte_device *gdev = NULL;
> +	int index = 0;
> +	int err;
> +
> +	if (label) {
> +		index = of_property_match_string(np, "hte-names", label);
> +		if (index < 0)
> +			return ERR_PTR(index);
> +	}
> +
> +	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
> +					 args);
> +	if (err) {
> +		pr_err("%s(): can't parse \"htes\" property\n", __func__);
> +		return ERR_PTR(err);
> +	}
> +
> +	gdev = of_node_to_htedevice(args->np);
> +	if (IS_ERR(gdev)) {
> +		pr_err("%s(): HTE chip not found\n", __func__);
> +		of_node_put(args->np);
> +		return gdev;
> +	}
> +
> +	return gdev;
> +}
> +
> +static struct hte_ts_desc *__hte_req_ts(struct device *dev,
> +					struct device_node *np,
> +					const char *label,
> +					void (*cb)(enum hte_notify n))
> +{
> +	struct hte_device *gdev = NULL;
> +	struct hte_ts_desc *desc;
> +	struct of_phandle_args args;
> +	int ret;
> +	u32 xlated_id;
> +
> +	gdev = of_hte_dev_get(dev, np, label, &args);
> +	if (IS_ERR(gdev))
> +		return ERR_CAST(gdev);
> +
> +	if (!gdev->chip) {
> +		pr_debug("requested id does not have provider\n");
> +		return ERR_PTR(-ENODEV);
> +	}
> +
> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> +	if (!desc)
> +		return ERR_PTR(-ENOMEM);
> +
> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
> +	if (ret < 0)
> +		goto put;
> +
> +	desc->name = NULL;
> +	if (label)
> +		desc->name = kstrdup(label, GFP_KERNEL);
> +
> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> +	if (ret < 0)
> +		goto put;
> +
> +	return desc;
> +
> +put:
> +	of_node_put(args.np);
> +	kfree(desc);
> +
> +	return ERR_PTR(ret);
> +}
> +
> +/**
> + * of_hte_request_ts() - Consumer calls this API to request the HTE facility
> + * on the specified entity, where entity is provider specific for example,
> + * GPIO lines, signals, buses etc...
> + *
> + * @dev: Consumer device.
> + * @label: Optional label.
> + * @cb: Optional notify callback to consumer when data is pushed by the
> + * provider.
> + *
> + * Context: Holds mutex lock, not suitable from atomic context.
> + * Returns: Timestamp descriptor on success or error ptr on failure.
> + */
> +struct hte_ts_desc *of_hte_request_ts(struct device *dev,
> +				      const char *label,
> +				      void (*cb)(enum hte_notify n))
> +{
> +
> +	if (dev && dev->of_node)
> +		return __hte_req_ts(dev, dev->of_node, label, cb);
> +	else
> +		return ERR_PTR(-EOPNOTSUPP);
> +}
> +EXPORT_SYMBOL_GPL(of_hte_request_ts);
> +
> +static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)

I'm not seeing what is devm about this.

> +{
> +	struct hte_ts_desc **p = res;
> +
> +	if (WARN_ON(!p || !*p))
> +		return 0;
> +
> +	return *p == data;
> +}
> +
> +static void __devm_hte_release_ts(struct device *dev, void *res)
> +{
> +	hte_release_ts(*(struct hte_ts_desc **)res);
> +}
> +
> +/**
> + * devm_hte_release_ts() - Resource managed hte_release_ts().

I'd not introduce this until you have a user.  It very rarely actually makes
sense to call a devm release manually. Not having one makes people think harder
about it.

> + * @dev: HTE consumer/client device.
> + * @desc: HTE ts descriptor.
> + *
> + * Release timestamp functionality and its resources previously allocated using
> + * of_hte_request_ts(). Calling this function is usually not needed because
> + * devm-allocated resources are automatically released on driver detach.
> + *
> + * Context: Same as hte_release_ts() function.
> + * Returns: 0 on success otherwise negative error code.
> + */
> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
> +{
> +	return devres_release(dev, __devm_hte_release_ts,
> +			      devm_hte_ts_match_desc, desc);
> +}
> +EXPORT_SYMBOL_GPL(devm_hte_release_ts);
> +
> +/**
> + * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().

If it's kernel-doc it needs to give no warnings when you point the kernel-doc
scripts at it.  They insist on full parameter documentation.

> + */
> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> +					   const char *label,
> +					   void (*cb)(enum hte_notify n))
> +{
> +
> +	struct hte_ts_desc **ptr, *desc;
> +
> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);

Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
last kernel cycle.


> +	if (!ptr)
> +		return ERR_PTR(-ENOMEM);
> +
> +	desc = of_hte_request_ts(dev, label, cb);
> +	if (!IS_ERR(desc)) {
> +		*ptr = desc;
> +		devres_add(dev, ptr);
> +	} else {
> +		devres_free(ptr);
> +	}
> +	return desc;
> +}
> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
> +
> +static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
> +					  size_t val)

Not a good name or indeed combination of different things.
hte_desc_to_info() and some separate check on val would be better.

> +{
> +	struct hte_ts_info *ei;
> +
> +	if (!desc || !desc->data_subsys || !val) {
> +		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
> +		return NULL;
> +	}
> +
> +	ei = desc->data_subsys;
> +	if (!ei || !ei->buf) {
> +		pr_debug("%s:%d\n", __func__, __LINE__);
> +		return NULL;
> +	}
> +
> +	return ei;
> +}
> +
> +static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
> +{
> +	size_t el_avail;
> +
> +	el_avail = buffer->access->el_available(buffer);
> +
> +	return (el_avail >= to_read) ? false : true;

return el_avail < to_read;

> +}
> +
> +static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> +			       struct hte_ts_data *el, size_t n, bool block)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +	int ret;
> +	size_t to_read, copied;
> +
> +	ei = hte_para_check(desc, n);
> +	if (!ei)
> +		return -EINVAL;
> +
> +	buffer = ei->buf;
> +
> +	to_read = min_t(size_t, n, buffer->watermark);

Needs a comment as not obvious why you'd read the min of that requested or
the watermark if there might be more available.

> +
> +	do {
> +		if (hte_ts_buf_wait(buffer, to_read)) {
> +			if (!block) {
> +				/* Possibly early here to retrieve, try again */
> +				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
> +					__func__, ret);
> +				return -EAGAIN;
> +			}
> +			ret = wait_event_interruptible(buffer->pollq,
> +					!hte_ts_buf_wait(buffer, to_read));
> +			if (ret)
> +				return ret;
> +		}
> +		ret = buffer->access->read(buffer, (void *)el,

If you have to cast to a void * that usually means something is wrong in your definitions.
Why is it needed here?  Looks like read has an inappropriate definition.

> +					   n * buffer->bytes_per_datum,
> +					   &copied);
> +		if (ret < 0)
> +			return ret;
> +
> +		if (copied > 0)
> +			return 0;
> +		else if (copied == 0 && !block)
> +			return -EAGAIN;
> +	} while (copied == 0);
> +
> +	return 0;
> +}
> +
> +/**
> + * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
> + * nano seconds i.e. el->tsc will be in ns.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + * @el: buffer to store the timestamp details.
> + * @n: Number of struct hte_timestamp_el elements.
> + *
> + * Context: Can be called from the atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> +		       struct hte_ts_data *el, size_t n)
> +{
> +	return _hte_retrieve_ts_ns(desc, el, n, false);
> +}
> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
> +
> +/**
> + * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
> + * @desc: ts descriptor, same as returned from request API.
> + * @el: buffer to store the timestamp data.
> + * @n: Number of struct hte_ts_data data.
> + *
> + * Context: Can not be called from the atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> +			    struct hte_ts_data *el, size_t n)
> +{
> +	return _hte_retrieve_ts_ns(desc, el, n, true);
> +}
> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
> +
> +/**
> + * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
> + * depth.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + * @len: New length/depth.
> + *
> + * The correct sequence to set buffer length is as below:
> + * 1) Disable timestamp by calling hte_disable_ts API.
> + * 2) Optionally retrieve all the timestamps by calling non blocking
> + *    hte_retrieve_ts_ns() API. This step only needed if you still care about
> + *    the data.
> + * 3) Call this API.
> + * 4) Enable timestamp by calling hte_enable_ts API.
> + *
> + * This API destroys previously allocated buffer and creates new one, because
> + * of that, it is mandatory to follow above sequence to make sure there is no
> + * race between various other APIs in the subsystem.

Good docs.  This is why I mentioned in review of docs patch that it is better
to just have that refer to the kernel-doc in these files.  Keep all this good
information in one place.

> + *
> + * By default during the request API call, HTE subsystem allocates software
> + * buffer with predefined length, this API gives flexibility to adjust the
> + * length according to consumer's need.
> + *
> + * Context: Can not be called from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +	int ret;
> +
> +	ei = hte_para_check(desc, len);
> +	if (!ei)
> +		return -EINVAL;
> +
> +	buffer = ei->buf;
> +	ret = buffer->access->set_length(buffer, len,
> +					 sizeof(struct hte_ts_data));
> +	if (ret)
> +		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);

Not point in printing things line __func__ manually in dev_err() etc.
Dynamic debug includes that and gives far more information + control of this.

> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hte_set_buf_len);
> +
> +/**
> + * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
> + * depth or length.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + *
> + * Context: Any context.
> + * Returns: Positive length on success or 0 on failure.
> + */
> +size_t hte_get_buf_len(const struct hte_ts_desc *desc)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +
> +	ei = hte_para_check(desc, 1);
> +	if (!ei)
> +		return 0;
> +
> +	buffer = ei->buf;
> +
> +	return buffer->access->get_length(buffer);
> +}
> +EXPORT_SYMBOL_GPL(hte_get_buf_len);
> +
> +/**
> + * hte_available_ts() - Returns total available timestamps.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + *
> + * The API helps consumers to pre-allocate its internal buffer required
> + * during hte_retrieve_ts_ns call.
> + *
> + * Context: Any context.
> + * Returns: Positive value if elements are available else 0. The value is
> + * number of total available struct hte_timestamp_el elements available not
> + * the size in bytes.
> + */
> +size_t hte_available_ts(const struct hte_ts_desc *desc)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +
> +	ei = hte_para_check(desc, 1);
> +	if (!ei)
> +		return 0;
> +
> +	buffer = ei->buf;
> +
> +	return buffer->access->el_available(buffer);
> +}
> +EXPORT_SYMBOL_GPL(hte_available_ts);
> +
> +/**
> + * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
> + * buffer watermark. The correct sequence to call this API is as below:
> + * 1) Disable timestamp by calling hte_disable_ts API.
> + * 2) Call this API.
> + * 3) Enable timestamp by calling hte_enable_ts API.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + * @val: New watermark.
> + *
> + * By default during the request API call, HTE subsystem sets watermark as 1,
> + * this API gives flexibility to adjust the watermark according to consumer's
> + * need. The consumers will get notification through callback registered during
> + * request API either when timestamp is dropped or watermark is reached or will
> + * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
> + * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
> + *
> + * Context: Any context.

You have no way of knowing that as will depend on the driver - I'd definitely
suggest not from atomic context, but then that would be crazy so you are better
 off not documenting any specific requirement at all.

> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +	int ret;
> +
> +	ei = hte_para_check(desc, val);
> +	if (!ei)
> +		return -EINVAL;
> +
> +	buffer = ei->buf;
> +	ret = buffer->access->set_watermark(buffer, val);
> +	if (ret)
> +		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
> +
> +/**
> + * hte_get_buf_watermark() - Consumer calls this API to get software
> + * buffer watermark.
> + * @desc: ts descriptor, same as returned from request API.
> + *
> + * Context: Any context.
> + * Returns: Positive current watermark on success or 0 on failure.
> + */
> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
> +{
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +
> +	ei = hte_para_check(desc, 1);
> +	if (!ei)
> +		return 0;
> +
> +	buffer = ei->buf;
> +
> +	return buffer->access->get_watermark(buffer);
> +}
> +EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
> +
> +/**
> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
> + * node directly, where meaning of the entity is provider specific, for example
> + * lines, signals, GPIOs, buses etc...
> + *
> + * @of_node: HTE provider device node.
> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
> + * @cb: Optional callback to notify.
> + *
> + * Context: Holds mutex lock, can not be called from atomic context.

What mutex and why?  If it is one you can check is held even better.

> + * Returns: ts descriptor on success or error pointers.
> + */
> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> +					  unsigned int id,
> +					  void (*cb)(enum hte_notify n))
> +{
> +	struct hte_device *gdev;
> +	struct hte_ts_desc *desc;
> +	int ret;
> +	u32 xlated_id;
> +
> +	gdev = of_node_to_htedevice(of_node);
> +	if (IS_ERR(gdev))
> +		return ERR_PTR(-ENOTSUPP);
> +
> +	if (!gdev->chip || !gdev->chip->ops)
> +		return ERR_PTR(-ENOTSUPP);
> +
> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> +	if (!desc) {
> +		ret = -ENOMEM;
> +		goto out_put_device;
> +	}

Pass a desc pointer into this function rather than allocating the structure
in here.  That lets the caller embed that structure inside one of it's own
structures if it wants to, resulting in fewer small allocations which is always good.

It's far from obvious that the caller needs to free desc.

> +
> +	desc->con_id = id;
> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev,
> +			"failed to xlate id: %d\n", id);
> +		goto out_free_desc;
> +	}
> +
> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> +	if (ret < 0) {
> +		dev_err(gdev->chip->dev,
> +			"failed to request id: %d\n", id);
> +		goto out_free_desc;
> +	}
> +
> +	return desc;
> +
> +out_free_desc:
> +	kfree(desc);
> +
> +out_put_device:
> +	return ERR_PTR(ret);
> +}
> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
> +
> +/**
> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
> + * information of the desc.
> + *
> + * @desc: ts descriptor, same as returned from request API.
> + *
> + * Context: Any context.
> + * Returns: 0 on success else negative error code on failure.
> + */
> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> +			 struct hte_clk_info *ci)
> +{
> +	struct hte_chip *chip;
> +	struct hte_ts_info *ei;
> +
> +	if (!desc || !desc->data_subsys || !ci) {
> +		pr_debug("%s:%d\n", __func__, __LINE__);
> +		return -EINVAL;
> +	}
> +
> +	ei = desc->data_subsys;
> +	if (!ei || !ei->gdev || !ei->gdev->chip)
> +		return -EINVAL;
> +
> +	chip = ei->gdev->chip;
> +	if (!chip->ops->get_clk_src_info)
> +		return -ENOTSUPP;
> +
> +	return chip->ops->get_clk_src_info(chip, ci);
> +}
> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
> +
> +static inline void hte_add_to_device_list(struct hte_device *gdev)
> +{
> +	struct hte_device *prev;

Needs to take an appropriate lock as you may have concurrent calls.

> +
> +	if (list_empty(&hte_devices)) {
> +		list_add_tail(&gdev->list, &hte_devices);

Needs a comment. I've no idea why you might want to only add it if there were
no other hte_devices already there.

> +		return;
> +	}
> +
> +	prev = list_last_entry(&hte_devices, struct hte_device, list);
Why woud you do this?

> +	list_add_tail(&gdev->list, &hte_devices);
> +}
> +
> +/**
> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
> + * using this API from its ISR or atomic context.
> + *
> + * @chip: The HTE chip, used during the registration.
> + * @xlated_id: entity id understood by both subsystem and provider, usually this
> + * is obtained from xlate callback during request API.
> + * @data: timestamp data.
> + * @n: Size of the data.
> + *
> + * Context: Atomic.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> +			  struct hte_ts_data *data, size_t n)
> +{
> +	unsigned int ret;
> +	bool notify;
> +	size_t el_avail;
> +	struct hte_ts_buf *buffer;
> +	struct hte_ts_info *ei;
> +
> +	if (!chip || !data || !chip->gdev)
> +		return -EINVAL;
> +
> +	if (xlated_id > chip->nlines)
> +		return -EINVAL;
> +
> +	ei = &chip->gdev->ei[xlated_id];
> +
> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
> +		return -EINVAL;
> +	}
> +
> +	/* timestamp sequence counter, start from 0 */
> +	data->seq = ei->seq++;
> +
> +	buffer = ei->buf;
> +	el_avail = buffer->access->el_available(buffer);

> +	ret = buffer->access->store(buffer, data, n);

If we are doing this from the hte core, why is buffer definition in the scope of the
drivers rather than the core?  That seems backwards to me.

> +	if (ret != n) {
> +		atomic_inc(&ei->dropped_ts);
> +		if (ei->cb)
> +			ei->cb(HTE_TS_DROPPED);
> +		return -ENOMEM;
> +	}
> +
> +	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;

You push n but only check on el_avail + 1 here.  
Also, this is the same as

	notify = ((el_avail + 1) >= buffer->watermark;


> +
> +	/*
> +	 * If there is a callback, its consumer's job to retrieve the timestamp.
> +	 * For the rest, wake up the process.
> +	 */
> +	if (notify && ei->cb) {
> +		ei->cb(HTE_TS_AVAIL);
> +		return 0;

Given you return 0 anyway, might as well not have this line.

> +	} else if (notify) {
> +		wake_up_interruptible(&buffer->pollq);
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
> +
> +/**
> + * hte_register_chip() - Used by provider to register a HTE chip.
> + * @chip: the HTE chip to add to subsystem.
> + *
> + * Context: Can not be called from atomic context.

Whilst true, I'd think that was common sense as it would be insane
to register something like this from atomic context.  So I'd say no
need to comment on it!  Keep those comments for things that
might be used like that.

> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_register_chip(struct hte_chip *chip)
> +{
> +	struct hte_device *gdev;
> +	int ret;
> +	u32 i;
> +
> +	if (!chip || !chip->dev || !chip->dev->of_node)
> +		return -EINVAL;
> +
> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
> +		dev_err(chip->dev, "Driver needs to provide ops\n");
> +		return -EINVAL;
> +	}
> +
> +	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
> +	if (!gdev)
> +		return -ENOMEM;
> +
> +	gdev->chip = chip;
> +	chip->gdev = gdev;
> +	gdev->nlines = chip->nlines;
> +	gdev->sdev = chip->dev;
> +
> +	/*
> +	 * Allocate all the supported entities here at once, this will have
> +	 * following advantages:
> +	 * When provider pushes timestamp, it can then just send the
> +	 * xlated_id, subsystem will use it as an index which
> +	 * gives us the constant time access; this is important as mostly
> +	 * providers will be pushing the timestamps from their ISR.
> +	 */
> +	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
> +			   GFP_KERNEL);

I'd be tempted to do this as a 0 length element at the end of gdev
then do the allocation in one go use struct_size() etc to work out
how long it is.  Cuts down on allocations + error paths to deal with
for no obvious disadvantage.

> +	if (!gdev->ei) {
> +		ret = -ENOMEM;
> +		goto err_free_gdev;
> +	}
> +
> +	for (i = 0; i < chip->nlines; i++) {
> +		gdev->ei[i].flags = 0;

zero allocated, so don't bother setting things to 0 where it's a fairly obvious
base state.  If you set something to 0 to act as some form of documentation then
that's fine, but I don't think that's true here.

> +		gdev->ei[i].gdev = gdev;
> +		gdev->ei[i].seq = 0;
> +		mutex_init(&gdev->ei[i].mlock);
> +	}
> +
> +	if (chip->dev->driver)
> +		gdev->owner = chip->dev->driver->owner;
> +	else
> +		gdev->owner = THIS_MODULE;
> +
> +	if (!chip->xlate) {
> +		chip->xlate = hte_simple_xlate;
> +		/* Just a id number to monitor */
> +		chip->of_hte_n_cells = 1;
> +	}
> +
> +	of_node_get(chip->dev->of_node);
> +
> +	INIT_LIST_HEAD(&gdev->list);
> +
> +	spin_lock(&hte_lock);
> +	hte_add_to_device_list(gdev);
> +	spin_unlock(&hte_lock);
> +
> +	hte_chip_dbgfs_init(gdev);
> +
> +	dev_dbg(chip->dev, "Added hte chip\n");
> +	return 0;
> +
> +err_free_gdev:
> +	kfree(gdev);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hte_register_chip);
> +
> +/**
> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
> + * @chip: the HTE chip to remove.
> + *
> + * Context: Can not be called from atomic context.
> + * Returns: 0 on success or a negative error code on failure.
> + */
> +int hte_unregister_chip(struct hte_chip *chip)
> +{
> +	struct hte_device *gdev = chip->gdev;
> +
> +	spin_lock(&hte_lock);
> +	list_del(&gdev->list);
> +	spin_unlock(&hte_lock);
> +
> +	gdev->chip = NULL;
> +
> +	of_node_put(chip->dev->of_node);
> +	hte_dbgfs_deinit(gdev->dbg_root);
> +	kfree(gdev->ei);
> +	kfree(gdev);
> +
> +	dev_dbg(chip->dev, "Removed hte chip\n");
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(hte_unregister_chip);
> +
> +/* Driver APIs ends */

Don't bother with file layout type comments.  They don't add that much and tend
to rot horribly over time as people move code around in files.

> diff --git a/include/linux/hte.h b/include/linux/hte.h
> new file mode 100644
> index 000000000000..e1737579d4c4
> --- /dev/null
> +++ b/include/linux/hte.h
> @@ -0,0 +1,278 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 NVIDIA Corporation
> + *
> + * Author: Dipen Patel <dipenp@nvidia.com>
> + */
> +
> +#ifndef __LINUX_HTE_H
> +#define __LINUX_HTE_H
> +
> +struct hte_chip;
> +struct hte_device;
> +struct of_phandle_args;
> +
> +/**
> + * Used by providers to indicate the direction of the timestamp.
> + */
> +#define HTE_EVENT_RISING_EDGE          0x1
> +#define HTE_EVENT_FALLING_EDGE         0x2

Use an enum rather than a define for this as it's a value that can take a
set of distinct values.  Also, provide a name for 'I've no idea' which
I'm guessing is 0 currently.

> +
> +/**
> + * struct hte_ts_data - HTE timestamp data.
> + * The provider uses and fills timestamp related details during push_timestamp
> + * API call. The consumer uses during retrieve_timestamp API call.
> + *
> + * @tsc: Timestamp value.
> + * @seq: Sequence counter of the timestamps.
> + * @dir: Direction of the event at the time of timestamp.
> + */
> +struct hte_ts_data {
> +	u64 tsc;
> +	u64 seq;
> +	int dir;
> +};
> +
> +/**
> + * struct hte_clk_info - Clock source info that HTE provider uses.
> + * The provider uses hardware clock as a source to timestamp real time. This
> + * structure presents the clock information to consumers. 
> + *
> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
> + * @type: Clock type. CLOCK_* types.

So this is something we got a it wrong in IIO. It's much better to define
a subset of clocks that can be potentially used.  There are some that make
absolutely no sense and consumers really don't want to have to deal with them.
 
> + */
> +struct hte_clk_info {
> +	u64 hz;
> +	clockid_t type;
> +};
> +
> +/**
> + * HTE subsystem notifications for the consumers.
> + *
> + * @HTE_TS_AVAIL: Timestamps available notification.
> + * @HTE_TS_DROPPED: Timestamps dropped notification.

Something I've missed so far is whether drops are in a kfifo or a ring
fashion.  I'm guess that's stated somewhere, but it might be useful to have
it here.

> + */
> +enum hte_notify {
> +	HTE_TS_AVAIL = 1,
> +	HTE_TS_DROPPED,
> +	HTE_NUM_NOTIFIER,
> +};
> +
> +/**
> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
> + * communication token between consumers to subsystem and subsystem to
> + * providers.
> + *
> + * @con_id: This is the same id sent in request APIs.
> + * @name: Descriptive name of the entity that is being monitored for the
> + * realtime timestamping.
> + * @data_subsys: Subsystem's private data relate to requested con_id.
> + */
> +struct hte_ts_desc {
> +	u32 con_id;
> +	char *name;
> +	void *data_subsys;
> +};
> +
> +/**
> + * struct hte_ops - HTE operations set by providers.
> + *
> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
> + * non-zero for failures.
> + * @get_clk_src_info: Optional hook to get the clock information provider uses
> + * to timestamp. Returns 0 for success and negative error code for failure. On
> + * success HTE subsystem fills up provided struct hte_clk_info.

Why optional?  Consumers will probably need that information.

> + *
> + * xlated_id parameter is used to communicate between HTE subsystem and the
> + * providers. It is the same id returned during xlate API call and translated
> + * by the provider. This may be helpful as both subsystem and provider locate
> + * the requested entity in constant time, where entity could be anything from
> + * lines, signals, events, buses etc.. that providers support.
> + */
> +struct hte_ops {
> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
> +	int (*get_clk_src_info)(struct hte_chip *chip,
> +				struct hte_clk_info *ci);
> +};
> +
> +/**
> + * struct hte_chip - Abstract HTE chip structure.
> + * @name: functional name of the HTE IP block.
> + * @dev: device providing the HTE.

Unclear naming.  Is this the parent device, or one associated with the HTE itself?
I'm guessing today you don't have one associated with the HTE, but it is plausible you
might gain on in future to make it fit nicely in the device model as a function of another
device.

> + * @ops: callbacks for this HTE.
> + * @nlines: number of lines/signals supported by this chip.
> + * @xlate: Callback which translates consumer supplied logical ids to
> + * physical ids, return from 0 for the success and negative for the
> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
> + * @data: chip specific private data.
> + */
> +struct hte_chip {
> +	const char *name;
> +	struct device *dev;
> +	const struct hte_ops *ops;
> +	u32 nlines;
> +	int (*xlate)(struct hte_chip *gc,
> +		     const struct of_phandle_args *args,
> +		     struct hte_ts_desc *desc, u32 *xlated_id);
> +	u8 of_hte_n_cells;
> +
> +	/* only used internally by the HTE framework */
> +	struct hte_device *gdev;
> +	void *data;
> +};
> +
> +#if IS_ENABLED(CONFIG_HTE)
> +/* HTE APIs for the providers */
> +int hte_register_chip(struct hte_chip *chip);
> +int hte_unregister_chip(struct hte_chip *chip);
> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> +			  struct hte_ts_data *data, size_t n);
> +
> +/* HTE APIs for the consumers */
> +
> +int hte_release_ts(struct hte_ts_desc *desc);
> +struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
> +				      void (*cb)(enum hte_notify n));
> +
> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> +					   const char *label,
> +					   void (*cb)(enum hte_notify n));
> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> +					  unsigned int id,
> +					  void (*cb)(enum hte_notify n));
> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
> +		       size_t n);
> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> +			    struct hte_ts_data *el, size_t n);
> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
> +size_t hte_get_buf_len(const struct hte_ts_desc *desc);
> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
> +size_t hte_available_ts(const struct hte_ts_desc *desc);
> +int hte_enable_ts(struct hte_ts_desc *desc);
> +int hte_disable_ts(struct hte_ts_desc *desc);
> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> +			 struct hte_clk_info *ci);
> +
>


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
  2021-07-01 14:21   ` Kent Gibson
@ 2021-07-04 20:27   ` Jonathan Cameron
  2021-07-29  2:42     ` Dipen Patel
  2021-07-08 23:33   ` Michał Mirosław
  2 siblings, 1 reply; 75+ messages in thread
From: Jonathan Cameron @ 2021-07-04 20:27 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Fri, 25 Jun 2021 16:55:24 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> Tegra194 device has multiple HTE instances also known as GTE
> (Generic hardware Timestamping Engine) which can timestamp subset of
> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> and exposes timestamping ability on those lines to the consumers
> through HTE subsystem.
> 
> Also, with this patch, added:
> - documentation about this provider and its capabilities at
> Documentation/hte.
> - Compilation support in Makefile and Kconfig
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>

A few comments inline,

J
> ---
>  Documentation/hte/index.rst        |  21 ++
>  Documentation/hte/tegra194-hte.rst |  65 ++++
>  Documentation/index.rst            |   1 +
>  drivers/hte/Kconfig                |  12 +
>  drivers/hte/Makefile               |   1 +
>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>  6 files changed, 654 insertions(+)
>  create mode 100644 Documentation/hte/index.rst
>  create mode 100644 Documentation/hte/tegra194-hte.rst
>  create mode 100644 drivers/hte/hte-tegra194.c
> 
> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
> new file mode 100644
> index 000000000000..f311ebec6b47
> --- /dev/null
> +++ b/Documentation/hte/index.rst
> @@ -0,0 +1,21 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +============================================
> +The Linux Hardware Timestamping Engine (HTE)
> +============================================
> +
> +The HTE Subsystem
> +=================
> +
> +.. toctree::
> +   :maxdepth: 1
> +
> +   hte
> +
> +HTE Tegra Provider
> +==================
> +
> +.. toctree::
> +   :maxdepth: 1
> +
> +   tegra194-hte
> \ No newline at end of file
> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> new file mode 100644
> index 000000000000..c23eaafcf080
> --- /dev/null
> +++ b/Documentation/hte/tegra194-hte.rst
> @@ -0,0 +1,65 @@
> +HTE Kernel provider driver
> +==========================
> +
> +Description
> +-----------
> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
> +known as generic timestamping engine (GTE). This provider driver implements
> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> +driver converts clock tick rate to nano seconds before storing it as timestamp
> +value.
> +
> +GPIO GTE
> +--------
> +
> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
> +needs to be configured as input and IRQ needs to ba enabled as well. The only
> +always on (AON) gpio controller instance supports timestamping GPIOs in
> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
> +controller as it requires very specific bits to be set in GPIO config register.
> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
> +subsystem and GPIO GTE for in kernel consumers.
> +
> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
> +
> +	To enable HTE on given GPIO line.
> +
> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
> +
> +	To retrieve hardwre timestamp in nano seconds.
> +
> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
> +
> +	To query if HTE is enabled on the given GPIO.
> +
> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
> +in nano second.
> +
> +IRQ GTE
> +--------
> +
> +This GTE instance helps timestamp LIC IRQ lines in real time. There are 352 IRQ
> +lines which this instance can help timestamp realtime. The hte devicetree
> +binding described at ``Documentation/devicetree/bindings/hte/`` gives out
> +example how consumer can request IRQ line, since it is one to one mapping,
> +consumers can simply specify IRQ number that they are interested in. There is
> +no userspace consumer support for this GTE instance. The sample test code
> +hte-tegra194-irq-test.c, located in ``drivers/hte/`` directory,
> +demonstrates how to use IRQ GTE instance. The below is sample device tree
> +snippet code for the test driver::
> +
> + tegra_hte_irq_test {
> +        compatible = "nvidia,tegra194-hte-irq-test";
> +        htes = <&tegra_hte_lic 0x19>;
> +        hte-names = "hte-lic";
> + };
> +
> +The source code of the driver both IRQ and GPIO GTE is locate at
> +``drivers/hte/hte-tegra194.c``.
> \ No newline at end of file
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 1b13c2445e87..b41118577fe6 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -138,6 +138,7 @@ needed).
>     misc-devices/index
>     scheduler/index
>     mhi/index
> +   hte/index
>  
>  Architecture-agnostic documentation
>  -----------------------------------
> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> index 394e112f7dfb..f7b01fcc7190 100644
> --- a/drivers/hte/Kconfig
> +++ b/drivers/hte/Kconfig
> @@ -20,3 +20,15 @@ menuconfig HTE
>  
>            If unsure, say no.
>  
> +if HTE
> +
> +config HTE_TEGRA194
> +	tristate "NVIDIA Tegra194 HTE Support"
> +	depends on ARCH_TEGRA_194_SOC
> +	help
> +	  Enable this option for integrated hardware timestamping engine also
> +	  known as generic timestamping engine (GTE) support on NVIDIA Tegra194
> +	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
> +	  lines for timestamping in realtime.
> +
> +endif
> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
> index 9899dbe516f7..52f978cfc913 100644
> --- a/drivers/hte/Makefile
> +++ b/drivers/hte/Makefile
> @@ -1 +1,2 @@
>  obj-$(CONFIG_HTE)		+= hte.o
> +obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
> \ No newline at end of file

fix that

> diff --git a/drivers/hte/hte-tegra194.c b/drivers/hte/hte-tegra194.c
> new file mode 100644
> index 000000000000..8ad10efd3641
> --- /dev/null
> +++ b/drivers/hte/hte-tegra194.c
> @@ -0,0 +1,554 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 NVIDIA Corporation
> + *
> + * Author: Dipen Patel <dipenp@nvidia.com>
> + */
> +
> +#include <linux/err.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <linux/stat.h>
> +#include <linux/interrupt.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/hte.h>
> +#include <linux/uaccess.h>
> +
> +#define HTE_SUSPEND	0
> +
> +/* HTE source clock TSC is 31.25MHz */
> +#define HTE_TS_CLK_RATE_HZ	31250000ULL
> +#define HTE_CLK_RATE_NS		32
> +#define HTE_TS_NS_SHIFT	__builtin_ctz(HTE_CLK_RATE_NS)
> +
> +#define NV_AON_SLICE_INVALID	-1
> +
> +/* AON HTE line map For slice 1 */
> +#define NV_AON_HTE_SLICE1_IRQ_GPIO_28	12
> +#define NV_AON_HTE_SLICE1_IRQ_GPIO_29	13
> +
> +/* AON HTE line map For slice 2 */
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_0	0
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_1	1
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_2	2
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_3	3
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_4	4
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_5	5
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_6	6
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_7	7
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_8	8
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_9	9
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_10	10
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_11	11
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_12	12
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_13	13
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_14	14
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_15	15
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_16	16
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_17	17
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_18	18
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_19	19
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_20	20
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_21	21
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_22	22
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_23	23
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_24	24
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_25	25
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_26	26
> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_27	27
> +
> +/* AON GPIO port AA pins */
> +#define NV_AON_GPIO_PORT_AA_0		0
> +#define NV_AON_GPIO_PORT_AA_1		1
> +#define NV_AON_GPIO_PORT_AA_2		2
> +#define NV_AON_GPIO_PORT_AA_3		3
> +#define NV_AON_GPIO_PORT_AA_4		4
> +#define NV_AON_GPIO_PORT_AA_5		5
> +#define NV_AON_GPIO_PORT_AA_6		6
> +#define NV_AON_GPIO_PORT_AA_7		7
> +
> +/* AON GPIO port BB pins */
> +#define NV_AON_GPIO_PORT_BB_0		8
> +#define NV_AON_GPIO_PORT_BB_1		9
> +#define NV_AON_GPIO_PORT_BB_2		10
> +#define NV_AON_GPIO_PORT_BB_3		11
> +
> +/* AON GPIO port CC pins */
> +#define NV_AON_GPIO_PORT_CC_0		16
> +#define NV_AON_GPIO_PORT_CC_1		17
> +#define NV_AON_GPIO_PORT_CC_2		18
> +#define NV_AON_GPIO_PORT_CC_3		19
> +#define NV_AON_GPIO_PORT_CC_4		20
> +#define NV_AON_GPIO_PORT_CC_5		21
> +#define NV_AON_GPIO_PORT_CC_6		22
> +#define NV_AON_GPIO_PORT_CC_7		23
> +
> +/* AON GPIO port DD pins */
> +#define NV_AON_GPIO_PORT_DD_0		24
> +#define NV_AON_GPIO_PORT_DD_1		25
> +#define NV_AON_GPIO_PORT_DD_2		26
> +
> +/* AON GPIO port EE pins */
> +#define NV_AON_GPIO_PORT_EE_0		32
> +#define NV_AON_GPIO_PORT_EE_1		33
> +#define NV_AON_GPIO_PORT_EE_2		34
> +#define NV_AON_GPIO_PORT_EE_3		35
> +#define NV_AON_GPIO_PORT_EE_4		36
> +#define NV_AON_GPIO_PORT_EE_5		37
> +#define NV_AON_GPIO_PORT_EE_6		38
> +
> +
> +#define HTE_TECTRL		0x0
> +#define HTE_TETSCH		0x4
> +#define HTE_TETSCL		0x8
> +#define HTE_TESRC		0xC
> +#define HTE_TECCV		0x10
> +#define HTE_TEPCV		0x14
> +#define HTE_TECMD		0x1C
> +#define HTE_TESTATUS		0x20
> +#define HTE_SLICE0_TETEN	0x40
> +#define HTE_SLICE1_TETEN	0x60
> +
> +#define HTE_SLICE_SIZE		(HTE_SLICE1_TETEN - HTE_SLICE0_TETEN)
> +
> +#define HTE_TECTRL_ENABLE_ENABLE	0x1
> +
> +#define HTE_TECTRL_OCCU_SHIFT		0x8
> +#define HTE_TECTRL_INTR_SHIFT		0x1
> +#define HTE_TECTRL_INTR_ENABLE		0x1
> +
> +#define HTE_TESRC_SLICE_SHIFT		16
> +#define HTE_TESRC_SLICE_DEFAULT_MASK	0xFF
> +
> +#define HTE_TECMD_CMD_POP		0x1
> +
> +#define HTE_TESTATUS_OCCUPANCY_SHIFT	8
> +#define HTE_TESTATUS_OCCUPANCY_MASK	0xFF
> +
> +struct hte_slices {
> +	u32 r_val;
> +	unsigned long flags;
> +	/* to prevent lines mapped to same slice updating its register */
> +	spinlock_t s_lock;
> +};
> +
> +struct tegra_hte_line_mapped {
> +	int slice;
> +	u32 bit_index;
> +};
> +
> +struct tegra_hte_line_table {
> +	int map_sz;
> +	const struct tegra_hte_line_mapped *map;
> +};
> +
> +struct tegra_hte_soc {
> +	int hte_irq;
> +	u32 itr_thrshld;
> +	u32 conf_rval;
> +	struct hte_slices *sl;
> +	const struct tegra_hte_line_table *line_map;
> +	struct hte_chip *chip;
> +	void __iomem *regs;
> +};
> +
> +static const struct tegra_hte_line_mapped tegra194_aon_gpio_map[] = {
> +	/* gpio, slice, bit_index */
> +	[NV_AON_GPIO_PORT_AA_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_11},
> +	[NV_AON_GPIO_PORT_AA_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_10},
> +	[NV_AON_GPIO_PORT_AA_2]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_9},
> +	[NV_AON_GPIO_PORT_AA_3]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_8},
> +	[NV_AON_GPIO_PORT_AA_4]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_7},
> +	[NV_AON_GPIO_PORT_AA_5]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_6},
> +	[NV_AON_GPIO_PORT_AA_6]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_5},
> +	[NV_AON_GPIO_PORT_AA_7]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_4},
> +	[NV_AON_GPIO_PORT_BB_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_3},
> +	[NV_AON_GPIO_PORT_BB_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_2},
> +	[NV_AON_GPIO_PORT_BB_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_1},
> +	[NV_AON_GPIO_PORT_BB_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_0},
> +	[12] = {NV_AON_SLICE_INVALID, 0},
> +	[13] = {NV_AON_SLICE_INVALID, 0},
> +	[14] = {NV_AON_SLICE_INVALID, 0},
> +	[15] = {NV_AON_SLICE_INVALID, 0},
> +	[NV_AON_GPIO_PORT_CC_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_22},
> +	[NV_AON_GPIO_PORT_CC_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_21},
> +	[NV_AON_GPIO_PORT_CC_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_20},
> +	[NV_AON_GPIO_PORT_CC_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_19},
> +	[NV_AON_GPIO_PORT_CC_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_18},
> +	[NV_AON_GPIO_PORT_CC_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_17},
> +	[NV_AON_GPIO_PORT_CC_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_16},
> +	[NV_AON_GPIO_PORT_CC_7] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_15},
> +	[NV_AON_GPIO_PORT_DD_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_14},
> +	[NV_AON_GPIO_PORT_DD_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_13},
> +	[NV_AON_GPIO_PORT_DD_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_12},
> +	[27] = {NV_AON_SLICE_INVALID, 0},
> +	[28] = {NV_AON_SLICE_INVALID, 0},
> +	[29] = {NV_AON_SLICE_INVALID, 0},
> +	[30] = {NV_AON_SLICE_INVALID, 0},
> +	[31] = {NV_AON_SLICE_INVALID, 0},
> +	[NV_AON_GPIO_PORT_EE_0] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_29},
> +	[NV_AON_GPIO_PORT_EE_1] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_28},
> +	[NV_AON_GPIO_PORT_EE_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_27},
> +	[NV_AON_GPIO_PORT_EE_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_26},
> +	[NV_AON_GPIO_PORT_EE_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_25},
> +	[NV_AON_GPIO_PORT_EE_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_24},
> +	[NV_AON_GPIO_PORT_EE_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_23},
> +};
> +
> +static const struct tegra_hte_line_table aon_hte_map = {
> +	.map_sz = ARRAY_SIZE(tegra194_aon_gpio_map),
> +	.map = tegra194_aon_gpio_map,
> +};
> +
> +static inline u32 tegra_hte_readl(struct tegra_hte_soc *hte, u32 reg)
> +{
> +	return readl(hte->regs + reg);
> +}
> +
> +static inline void tegra_hte_writel(struct tegra_hte_soc *hte, u32 reg,
> +				    u32 val)
> +{
> +	writel(val, hte->regs + reg);
> +}
> +
> +static inline int tegra_hte_map_to_line_id(u32 eid, struct tegra_hte_soc *gs,
> +					  u32 *mapped)
> +{
> +	const struct tegra_hte_line_mapped *m;
> +
> +	if (gs->line_map) {
> +		m = gs->line_map->map;
> +		if (eid > gs->line_map->map_sz)
> +			return -EINVAL;
> +		if (m[eid].slice == NV_AON_SLICE_INVALID)
> +			return -EINVAL;
> +
> +		*mapped = (m[eid].slice << 5) + m[eid].bit_index;
> +	} else {
> +		*mapped = eid;
> +	}
> +
> +	return 0;
> +}
> +
> +static int tegra_hte_line_xlate(struct hte_chip *gc,
> +				 const struct of_phandle_args *args,
> +				 struct hte_ts_desc *desc, u32 *xlated_id)
> +{
> +	int ret = 0;
> +
> +	if (!gc || !desc || !xlated_id)
> +		return -EINVAL;
> +
> +	if (args) {
> +		if (gc->of_hte_n_cells < 1)
> +			return -EINVAL;
> +
> +		if (args->args_count != gc->of_hte_n_cells)
> +			return -EINVAL;
> +
> +		desc->con_id = args->args[0];
> +	}
> +
> +	ret = tegra_hte_map_to_line_id(desc->con_id, gc->data,
> +				       xlated_id);
> +	if (ret < 0) {
> +		dev_dbg(gc->dev, "con_id:%u mapping failed\n",
> +			desc->con_id);
> +		return ret;
> +	}
> +
> +	if (*xlated_id > gc->nlines)
> +		return -EINVAL;
> +
> +	dev_dbg(gc->dev, "requested id:%u, xlated id:%u\n",
> +		desc->con_id, *xlated_id);
> +
> +	return 0;
> +}
> +
> +static int tegra_hte_en_dis_common(struct hte_chip *chip, u32 line_id, bool en)
> +{
> +	u32 slice, sl_bit_shift, line_bit, val, reg;
> +	struct tegra_hte_soc *gs;
> +
> +	sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
> +
> +	if (!chip)
> +		return -EINVAL;
> +
> +	gs = (struct tegra_hte_soc *)chip->data;
> +
> +	if (line_id > chip->nlines) {
> +		dev_err(chip->dev,
> +			"line id: %u is not supported by this controller\n",
> +			line_id);
> +		return -EINVAL;
> +	}
> +
> +	slice = line_id >> sl_bit_shift;
> +	line_bit = line_id & (HTE_SLICE_SIZE - 1);
> +	reg = (slice << sl_bit_shift) + HTE_SLICE0_TETEN;
> +
> +	spin_lock(&gs->sl[slice].s_lock);
> +
> +	if (test_bit(HTE_SUSPEND, &gs->sl[slice].flags)) {
> +		spin_unlock(&gs->sl[slice].s_lock);
> +		dev_dbg(chip->dev, "device suspended");
> +		return -EBUSY;
> +	}
> +
> +	val = tegra_hte_readl(gs, reg);
> +	if (en)
> +		val = val | (1 << line_bit);
> +	else
> +		val = val & (~(1 << line_bit));
> +	tegra_hte_writel(gs, reg, val);
> +
> +	spin_unlock(&gs->sl[slice].s_lock);
> +
> +	dev_dbg(chip->dev, "line: %u, slice %u, line_bit %u, reg:0x%x\n",
> +		line_id, slice, line_bit, reg);
> +
> +	return 0;
> +}
> +
> +static int tegra_hte_request(struct hte_chip *chip, u32 line_id)
> +{
> +	return tegra_hte_en_dis_common(chip, line_id, true);
> +}
> +
> +static int tegra_hte_release(struct hte_chip *chip, u32 line_id)
> +{
> +	return tegra_hte_en_dis_common(chip, line_id, false);
> +}
> +
> +static int tegra_hte_clk_src_info(struct hte_chip *chip,
> +				  struct hte_clk_info *ci)
> +{
> +	(void)chip;
> +
> +	ci->hz = HTE_TS_CLK_RATE_HZ;
> +	ci->type = CLOCK_MONOTONIC;
> +
> +	return 0;
> +}
> +
> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
> +{
> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
> +	u64 tsc;
> +	int dir;
> +	struct hte_ts_data el;
> +
> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
> +		HTE_TESTATUS_OCCUPANCY_MASK) {
> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
> +		tsc = (((u64)tsh << 32) | tsl);
> +
> +		src = tegra_hte_readl(gs, HTE_TESRC);
> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
> +
> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
> +		cv = tegra_hte_readl(gs, HTE_TECCV);
> +		acv = pv ^ cv;
> +		while (acv) {
> +			bit_index = __builtin_ctz(acv);
> +			if ((pv >> bit_index) & BIT(0))
> +				dir = HTE_EVENT_RISING_EDGE;
> +			else
> +				dir = HTE_EVENT_FALLING_EDGE;
> +
> +			line_id = bit_index + (slice << 5);
> +			el.dir = dir;
> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
> +					      sizeof(el));
> +			acv &= ~BIT(bit_index);
> +		}
> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
> +	}
> +}
> +
> +static irqreturn_t tegra_hte_isr(int irq, void *dev_id)
> +{
> +	struct tegra_hte_soc *gs = dev_id;
> +
> +	tegra_hte_read_fifo(gs);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static const struct of_device_id tegra_hte_of_match[] = {
> +	{ .compatible = "nvidia,tegra194-gte-lic"},
> +	{ .compatible = "nvidia,tegra194-gte-aon", .data = &aon_hte_map},
> +	{ }
> +};
> +MODULE_DEVICE_TABLE(of, tegra_hte_of_match);
> +
> +static const struct hte_ops g_ops = {
> +	.request = tegra_hte_request,
> +	.release = tegra_hte_release,
> +	.enable = tegra_hte_request,
> +	.disable = tegra_hte_release,
> +	.get_clk_src_info = tegra_hte_clk_src_info,
> +};
> +
> +static int tegra_hte_probe(struct platform_device *pdev)
> +{
> +	int ret;
> +	u32 i, slices, val = 0;
> +	struct device *dev;
> +	struct tegra_hte_soc *hte_dev;
> +	struct hte_chip *gc;
> +
> +	dev = &pdev->dev;
> +
> +	hte_dev = devm_kzalloc(dev, sizeof(*hte_dev), GFP_KERNEL);
> +	if (!hte_dev)
> +		return -ENOMEM;
> +
> +	gc = devm_kzalloc(dev, sizeof(*gc), GFP_KERNEL);
> +	if (!gc)
> +		return -ENOMEM;
> +
> +	dev_set_drvdata(&pdev->dev, hte_dev);
> +	hte_dev->line_map = of_device_get_match_data(&pdev->dev);
> +
> +	hte_dev->regs = devm_platform_ioremap_resource(pdev, 0);
> +	if (IS_ERR(hte_dev->regs))
> +		return PTR_ERR(hte_dev->regs);
> +
> +	ret = of_property_read_u32(dev->of_node, "int-threshold",
> +				   &hte_dev->itr_thrshld);
> +	if (ret != 0)
> +		hte_dev->itr_thrshld = 1;
> +
> +	ret = of_property_read_u32(dev->of_node, "slices", &slices);
> +	if (ret != 0) {
> +		dev_err(dev, "Could not read slices\n");
> +		return -EINVAL;
> +	}
> +
> +	hte_dev->sl = devm_kzalloc(dev, sizeof(struct hte_slices) * slices,

Preference for sizeof(*hte_dev->sl) as it saves me checking the size is correct
for the type.

> +				   GFP_KERNEL);
> +	if (!hte_dev->sl)
> +		return -ENOMEM;
> +
> +	ret = platform_get_irq(pdev, 0);
> +	if (ret < 0) {
> +		dev_err(dev, "get irq failed.\n");

dev_err_probe() probably so you don't print the message if deferred probing is
going on.

> +		return ret;
> +	}
> +	hte_dev->hte_irq = ret;
> +	ret = devm_request_irq(dev, hte_dev->hte_irq, tegra_hte_isr, 0,
> +			       dev_name(dev), hte_dev);
> +	if (ret < 0) {
> +		dev_err(dev, "request irq failed.\n");
> +		return ret;
> +	}
> +
> +	gc->nlines = slices << 5;
> +	gc->ops = &g_ops;
> +	gc->dev = dev;
> +	hte_dev->chip = gc;
> +	gc->data = (void *)hte_dev;

Don't case to void * - cast to the actual type if necessary.
If it is a void * then most likely it shouldn't be if we always put something
in particular it in it.

> +	gc->xlate = tegra_hte_line_xlate;
> +	gc->of_hte_n_cells = 1;
> +
> +	ret = hte_register_chip(hte_dev->chip);
> +

No blank line before error handler.  Under the circumstances is that not
fatal?

> +	if (ret)
> +		dev_err(gc->dev, "hte chip register failed");
> +
> +	for (i = 0; i < slices; i++) {
> +		hte_dev->sl[i].flags = 0;
> +		spin_lock_init(&hte_dev->sl[i].s_lock);
> +	}
> +
> +	val = HTE_TECTRL_ENABLE_ENABLE |
> +	      (HTE_TECTRL_INTR_ENABLE << HTE_TECTRL_INTR_SHIFT) |
> +	      (hte_dev->itr_thrshld << HTE_TECTRL_OCCU_SHIFT);
> +	tegra_hte_writel(hte_dev, HTE_TECTRL, val);

You could use a devm_add_action_or_reset() to deal with unwinding this
plus add a devm_hte_register_chip() and then you can get rid of remove
entirely which is always nice.

> +
> +	dev_dbg(gc->dev, "lines: %d, slices:%d", gc->nlines, slices);
> +	return 0;
> +}
> +
> +static int tegra_hte_remove(struct platform_device *pdev)
> +{
> +	struct tegra_hte_soc *gs = dev_get_drvdata(&pdev->dev);
> +
> +	tegra_hte_writel(gs, HTE_TECTRL, 0);
> +
> +	return hte_unregister_chip(gs->chip);
> +}
> +
> +#ifdef CONFIG_PM_SLEEP

Personally I prefer the approach of marking PM functions
__maybe_unused and dropping the ifdef protections.
There have been a lot of subtle issues in the build system in the
past around those and it's much easier to just let the compiler
drop them if they are unused.

> +static int tegra_hte_resume_early(struct device *dev)
> +{
> +	u32 i;
> +	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
> +	u32 slices = gs->chip->nlines >> 5;

Whilst it's the same thing, I'd prefer a divide there by however lines there are in a slice.

> +	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
> +
> +	tegra_hte_writel(gs, HTE_TECTRL, gs->conf_rval);
> +
> +	for (i = 0; i < slices; i++) {
> +		spin_lock(&gs->sl[i].s_lock);
> +		tegra_hte_writel(gs,
> +				 ((i << sl_bit_shift) + HTE_SLICE0_TETEN),
> +				 gs->sl[i].r_val);
> +		clear_bit(HTE_SUSPEND, &gs->sl[i].flags);
> +		spin_unlock(&gs->sl[i].s_lock);
> +	}
> +
> +	return 0;
> +}
> +
> +static int tegra_hte_suspend_late(struct device *dev)
> +{
> +	u32 i;
> +	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
> +	u32 slices = gs->chip->nlines >> 5;
> +	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
> +
> +	gs->conf_rval = tegra_hte_readl(gs, HTE_TECTRL);
> +	for (i = 0; i < slices; i++) {
> +		spin_lock(&gs->sl[i].s_lock);
> +		gs->sl[i].r_val = tegra_hte_readl(gs,
> +				((i << sl_bit_shift) + HTE_SLICE0_TETEN));
> +		set_bit(HTE_SUSPEND, &gs->sl[i].flags);
> +		spin_unlock(&gs->sl[i].s_lock);
> +	}
> +
> +	return 0;
> +}
> +#endif
> +
> +static const struct dev_pm_ops tegra_hte_pm = {
> +	SET_LATE_SYSTEM_SLEEP_PM_OPS(tegra_hte_suspend_late,
> +				     tegra_hte_resume_early)
> +};
> +
> +static struct platform_driver tegra_hte_driver = {
> +	.probe = tegra_hte_probe,
> +	.remove = tegra_hte_remove,
> +	.driver = {
> +		.name = "tegra_hte",
> +		.pm = &tegra_hte_pm,
> +		.of_match_table = tegra_hte_of_match,
> +	},
> +};
> +
> +module_platform_driver(tegra_hte_driver);
> +
> +MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
> +MODULE_DESCRIPTION("NVIDIA Tegra HTE (Hardware Timestamping Engine) driver");
> +MODULE_LICENSE("GPL v2");


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-04 20:15   ` Jonathan Cameron
@ 2021-07-04 20:45     ` Jonathan Cameron
  2021-07-28  5:12       ` Dipen Patel
  2021-07-28  4:38     ` Dipen Patel
  1 sibling, 1 reply; 75+ messages in thread
From: Jonathan Cameron @ 2021-07-04 20:45 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt, William Breathitt Gray

On Sun, 4 Jul 2021 21:15:25 +0100
Jonathan Cameron <jic23@kernel.org> wrote:

> On Fri, 25 Jun 2021 16:55:23 -0700
> Dipen Patel <dipenp@nvidia.com> wrote:
> 
> > Some devices can timestamp system lines/signals/Buses in real-time
> > using the hardware counter or other hardware means which can give
> > finer granularity and help avoid jitter introduced by software means
> > of timestamping. To utilize such functionality there has to be
> > framework where such devices can register themselves as producers or
> > providers so that the consumers or clients devices can request specific
> > line from the providers. This patch introduces such subsystem as
> > hardware timestamping engine (HTE).
> > 
> > It provides below APIs for the provider:
> > - hte_register_chip() -- To register the HTE chip.
> > - hte_unregister_chip() -- To unregister the HTE chip.
> > - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
> > 
> > It provides below APIs for the consumer:
> > - of_hte_request_ts() -- To request timestamp functionality.
> > - devm_of_hte_request_ts() -- Managed version of the above.
> > - hte_req_ts_by_dt_node() -- To request timestamp functionality by
> > using HTE provider dt node.
> > - devm_hte_release_ts() -- The managed version to release timestamp
> > functionality and associated resources.
> > - hte_retrieve_ts_ns() -- To retrieve timestamps.
> > - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
> > - hte_enable_ts() -- To disable timestamp functionality.
> > - hte_disable_ts() -- To enable timestamp functionality.
> > - hte_available_ts() -- To query available timestamp data.
> > - hte_release_ts() -- To release timestamp functionality and its
> > associated resources.
> > - hte_get_clk_src_info() -- To query clock source information from
> > the provider
> > 
> > It provides centralized software buffer management per requested id to
> > store the timestamp data for the consumers as below:
> > - hte_set_buf_len() -- To set the buffer length.
> > - hte_get_buf_len() -- To get the buffer length.
> > - hte_set_buf_watermark() -- To set the software threshold/watermark.
> > - hte_get_buf_watermark() -- To get the software threshold/watermark.
> > 
> > The detail about parameters and API usage are described in each
> > functions definitions in drivers/hte/hte.c file.
> > 
> > The patch adds compilation support in Makefile and menu options in
> > Kconfig.
> > 
> > Signed-off-by: Dipen Patel <dipenp@nvidia.com>  
> 
> Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
> around what this is doing + it is an RFC :)

Having read on through the rest of the series, one of the biggest things that became
clear is you have more layers of abstraction in here than make sense.  Squash things
together so you have fewer allocations (and hence fewer error paths etc).
Don't introduce ops functions unless you have more than one answer to what they
are (and a clear justification for those).   It's easy to add layers of indirection
later, but for now they just make your code harder to read.  Reality is that the buffer
is a kfifo, make it so everywhere, rather than pretending otherwise.

I thought for a while that you were allowing a different buffer implementation for
each of the your hte chips and was very confused as to why that would make sense.

Anyhow, it's interesting.  I'm not sure yet if the way it all fits together makes
sense and you will almost certainly want to support hardware FIFOs and those tend
to want to be partly exposed through to the consumer.

As to similar devices.  Lots of sensorhubs have timestamping facilities but it is
tightly coupled to the data streams, so probably doesn't make sense to map to a
generic subsystem like this.  You'll hit some of the same issues as those though
when you try to align these timestamps with system ones etc.

otherwise, some of the counter devices are closer to this.  Perhaps the ti ecap?
https://www.ti.com/lit/ug/spru807b/spru807b.pdf?ts=1625431159791&ref_url=https%253A%252F%252Fwww.google.com%252F

That has a short hardware buffer and can be used for absolute timestamp grabbing
on rising triggers etc.

Jonathan


> 
> > ---
> >  drivers/Kconfig      |    2 +
> >  drivers/Makefile     |    1 +
> >  drivers/hte/Kconfig  |   22 +
> >  drivers/hte/Makefile |    1 +
> >  drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
> >  include/linux/hte.h  |  278 +++++++++
> >  6 files changed, 1672 insertions(+)
> >  create mode 100644 drivers/hte/Kconfig
> >  create mode 100644 drivers/hte/Makefile
> >  create mode 100644 drivers/hte/hte.c
> >  create mode 100644 include/linux/hte.h
> > 
> > diff --git a/drivers/Kconfig b/drivers/Kconfig
> > index 47980c6b1945..9b078964974b 100644
> > --- a/drivers/Kconfig
> > +++ b/drivers/Kconfig
> > @@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
> >  source "drivers/counter/Kconfig"
> >  
> >  source "drivers/most/Kconfig"
> > +
> > +source "drivers/hte/Kconfig"
> >  endmenu
> > diff --git a/drivers/Makefile b/drivers/Makefile
> > index 5a6d613e868d..0a996a698e4c 100644
> > --- a/drivers/Makefile
> > +++ b/drivers/Makefile
> > @@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
> >  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
> >  obj-$(CONFIG_COUNTER)		+= counter/
> >  obj-$(CONFIG_MOST)		+= most/
> > +obj-$(CONFIG_HTE)		+= hte/
> > diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> > new file mode 100644
> > index 000000000000..394e112f7dfb
> > --- /dev/null
> > +++ b/drivers/hte/Kconfig
> > @@ -0,0 +1,22 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +menuconfig HTE
> > +        bool "Hardware Timestamping Engine (HTE) Support"
> > +        help
> > +          Hardware Timestamping Engine (HTE) Support.  
> 
> Tidy this up, but think that's already been commented on.
> 
> > +
> > +          Some devices provide hardware timestamping engine which can timestamp
> > +	  certain device lines/signals in realtime. This way to provide
> > +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
> > +	  comes with benefit for the applications like autonomous machines
> > +	  needing accurate timestamping event with less jitter.
> > +
> > +	  This framework provides a generic interface to such HTE devices
> > +          within the Linux kernel. It provides an API to register and
> > +	  unregister a HTE provider chip, configurable sw buffer to
> > +	  store the timestamps, push the timestamp from the HTE providers and
> > +	  retrieve timestamps for the consumers. It also provides means for the
> > +	  consumers to request signals it wishes to hardware timestamp and
> > +	  release them if not required.
> > +
> > +          If unsure, say no.
> > +
> > diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
> > new file mode 100644
> > index 000000000000..9899dbe516f7
> > --- /dev/null
> > +++ b/drivers/hte/Makefile
> > @@ -0,0 +1 @@
> > +obj-$(CONFIG_HTE)		+= hte.o
> > diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
> > new file mode 100644
> > index 000000000000..c53260d1e250
> > --- /dev/null
> > +++ b/drivers/hte/hte.c
> > @@ -0,0 +1,1368 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2021 NVIDIA Corporation
> > + *
> > + * Author: Dipen Patel <dipenp@nvidia.com>
> > + */
> > +
> > +#include <linux/kernel.h>
> > +#include <linux/module.h>
> > +#include <linux/err.h>
> > +#include <linux/slab.h>
> > +#include <linux/of.h>
> > +#include <linux/of_device.h>
> > +#include <linux/kfifo.h>
> > +#include <linux/mutex.h>
> > +#include <linux/sched.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/hte.h>
> > +#include <linux/delay.h>
> > +#include <linux/debugfs.h>
> > +
> > +/* Global list of the HTE devices */
> > +static DEFINE_SPINLOCK(hte_lock);
> > +static LIST_HEAD(hte_devices);
> > +
> > +enum {
> > +	HTE_TS_REGISTERED,
> > +	HTE_TS_DISABLE,
> > +};
> > +
> > +/* Default FIFO depth */
> > +#define HTE_EV_FIFO_EL		32
> > +
> > +#define HTE_TS_NAME_LEN		10
> > +
> > +struct hte_ts_buf;
> > +
> > +/**
> > + * struct hte_ts_buf_acc_func - Software buffer management functions.
> > + * @store: Store timestamp from atomic context as providers most likely
> > + * be pushing timestamps from their interrupt handlers.
> > + * @read: Read timestamps from the buffer.
> > + * @el_available: Available timestamps to retrieve. The client can use this to
> > + * query available elements so that it can pre-allocate internal buffer to send
> > + * to during hte_retrieve_ts_ns API.
> > + * @set_length: Set length/depth of the buffer.
> > + * @get_length: Get length/depth of the buffer.
> > + * @set_watermark: Set software threshold of the buffer.
> > + * @get_watermark: Get software threshold of the buffer.
> > + * @release: Release/free buffer.
> > + * @reset: Reset the buffer.
> > + */
> > +struct hte_ts_buf_acc_func {
> > +	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
> > +	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
> > +		    size_t *copied);
> > +	size_t (*el_available)(struct hte_ts_buf *buf);
> > +	int (*set_length)(struct hte_ts_buf *buf,
> > +			  size_t length, size_t bpd);
> > +	size_t (*get_length)(struct hte_ts_buf *buf);
> > +	int (*set_watermark)(struct hte_ts_buf *buf,
> > +			     size_t val);
> > +	size_t (*get_watermark)(struct hte_ts_buf *buf);
> > +	void (*release)(struct hte_ts_buf *buf);
> > +	void (*reset)(struct hte_ts_buf *buf);
> > +};
> > +
> > +/**
> > + * struct hte_ts_buf - Software buffer per requested id or entity to store
> > + * timestamps.
> > + *
> > + * @datum_len: Buffer depth or number of elements.
> > + * @bytes_per_datum: Element size in bytes.
> > + * @watermark: Software threshold at which client will be notified.
> > + * @valid: Validity of the buffer.
> > + * @pollq: Waitqueue for the blocking clients.
> > + * @access: Various buffer management functions.
> > + */
> > +struct hte_ts_buf {
> > +	size_t datum_len;
> > +	size_t bytes_per_datum;
> > +	size_t watermark;
> > +	bool valid;
> > +	wait_queue_head_t pollq;
> > +	const struct hte_ts_buf_acc_func *access;
> > +};
> > +
> > +/**
> > + * struct hte_ts_info - Information related to requested timestamp.
> > + *
> > + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
> > + * See xlate callback API.
> > + * @flags: Flags holding state informations.
> > + * @seq: Timestamp sequence counter.
> > + * @dropped_ts: Dropped timestamps.
> > + * @cb: Callback to notify clients.
> > + * @mlock: Lock during timestamp request/release APIs.
> > + * @ts_dbg_root: Root for the debug fs.
> > + * @gdev: HTE abstract device that this timestamp belongs to.
> > + * @buf: Per requested timestamp software buffer.
> > + * @desc: Timestamp descriptor understood between clients and HTE subsystem.
> > + */
> > +struct hte_ts_info {
> > +	u32 xlated_id;
> > +	unsigned long flags;
> > +	u64 seq;
> > +	atomic_t dropped_ts;
> > +	void (*cb)(enum hte_notify n);
> > +	struct mutex mlock;
> > +	struct dentry *ts_dbg_root;
> > +	struct hte_device *gdev;
> > +	struct hte_ts_buf *buf;

Where there is one instance, just embed it.  Lots of small allocations just make
for less readable code.

> > +	struct hte_ts_desc *desc;
> > +};
> > +
> > +/**
> > + * struct hte_device - HTE abstract device
> > + * @nlines: Number of entities this device supports.
> > + * @ts_req: Total number of entities requested.
> > + * @ei: Timestamp information.
> > + * @sdev: Device used at various debug prints.
> > + * @dbg_root: Root directory for debug fs.
> > + * @list: List node for internal use.  
> 
> Be more specific of what sort of internal use.
> 
> > + * @chip: HTE chip providing this HTE device.
> > + * @owner: helps prevent removal of modules when in use.
> > + */
> > +struct hte_device {
> > +	u32 nlines;
> > +	atomic_t ts_req;
> > +	struct hte_ts_info *ei;
> > +	struct device *sdev;
> > +	struct dentry *dbg_root;
> > +	struct list_head list;
> > +	struct hte_chip *chip;
> > +	struct module *owner;
> > +};
> > +
> > +/* Buffer management functions */
> > +
> > +/**
> > + * struct hte_kfifo - Software buffer wrapper.
> > + * @buffer: Abstract buffer device.
> > + * @gkf: Actual software buffer type, this case its FIFO.
> > + */
> > +struct hte_kfifo {
> > +	struct hte_ts_buf buffer;
> > +	struct kfifo gkf;
> > +};
> > +
> > +#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
> > +
> > +static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
> > +					size_t n)
> > +{
> > +	struct hte_kfifo *kf = buf_to_kfifo(r);
> > +
> > +	if (unlikely(!r->valid))
> > +		return 0;
> > +
> > +	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
> > +}
> > +
> > +static inline int hte_ts_buf_read(struct hte_ts_buf *r,
> > +				  unsigned char *buf, size_t n,
> > +				  size_t *copied)
> > +{
> > +	struct hte_kfifo *kf = buf_to_kfifo(r);
> > +
> > +	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
> > +		return -EINVAL;
> > +
> > +	*copied = kfifo_out(&kf->gkf, buf, n);
> > +
> > +	return 0;
> > +}
> > +
> > +static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
> > +{
> > +	struct hte_kfifo *kf = buf_to_kfifo(r);
> > +
> > +	if (!r->valid)
> > +		return 0;
> > +
> > +	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
> > +}
> > +
> > +static int hte_ts_buf_set_length(struct hte_ts_buf *r,
> > +				 size_t length, size_t bpd)
> > +{
> > +	int ret = 0;
> > +	struct hte_kfifo *buf;
> > +
> > +	if ((length == 0) || (bpd == 0) || !r)
> > +		return -EINVAL;
> > +
> > +	buf = buf_to_kfifo(r);
> > +
> > +	if (r->datum_len != length) {
> > +		if (r->valid)
> > +			kfifo_free(&buf->gkf);
> > +		r->valid = false;
> > +		r->datum_len = length;
> > +		r->bytes_per_datum = bpd;
> > +		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
> > +		if (!ret)
> > +			r->valid = true;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
> > +{
> > +	if ((!r->valid) || !r->datum_len)
> > +		return 0;
> > +
> > +	return r->datum_len;
> > +}
> > +
> > +static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
> > +{
> > +	if ((!r->valid) || (val > r->datum_len))
> > +		return -EINVAL;
> > +
> > +	r->watermark = val;
> > +
> > +	return 0;
> > +}
> > +
> > +static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
> > +{
> > +	if (!r->valid)
> > +		return 0;
> > +
> > +	return r->watermark;
> > +}
> > +
> > +static inline void hte_ts_buf_release(struct hte_ts_buf *r)
> > +{
> > +	struct hte_kfifo *kf = buf_to_kfifo(r);
> > +
> > +	r->valid = false;
> > +	kfifo_free(&kf->gkf);
> > +	kfree(kf);
> > +}
> > +
> > +static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
> > +{
> > +	struct hte_kfifo *kf = buf_to_kfifo(r);
> > +
> > +	if (!r->valid)
> > +		return;
> > +
> > +	kfifo_reset(&kf->gkf);
> > +}
> > +
> > +static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
> > +	.store = &hte_ts_store_to_buf,
> > +	.read = &hte_ts_buf_read,
> > +	.el_available = &hte_ts_buf_el_available,
> > +	.set_length = &hte_ts_buf_set_length,
> > +	.get_length = &hte_ts_buf_get_length,
> > +	.set_watermark = &hte_ts_buf_set_watermark,
> > +	.get_watermark = &hte_ts_buf_get_watermark,
> > +	.release = &hte_ts_buf_release,
> > +	.reset = &hte_ts_buf_reset,
> > +};
> > +
> > +static struct hte_ts_buf *hte_ts_buf_allocate(void)
> > +{
> > +	struct hte_kfifo *kf;
> > +
> > +	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
> > +	if (!kf)
> > +		return ERR_PTR(-ENOMEM);
> > +
> > +	init_waitqueue_head(&kf->buffer.pollq);
> > +	kf->buffer.watermark = 1;
> > +	kf->buffer.datum_len = 0;
> > +	kf->buffer.valid = false;
> > +	kf->buffer.access = &kfifo_access_funcs;

Why do you have this level abstraction?  I would suggest you flatten all this
into direct calls until you have some clear need for multiple buffer types.

A long long time ago (10 years or more) we had similar abstractions in IIO because
we though it would be helpful to support ring buffers and kfifos.  It wasn't, we ended
up ripping them all out because all they resulted in was a more complex code base
when in reality everyone was happy with a kfifo.

Jonathan


> > +
> > +	return &kf->buffer;
> > +}
> > +/* End of buffer management */
> > +
> > +/* Debugfs management */
> > +
> > +#ifdef CONFIG_DEBUG_FS
> > +
> > +static struct dentry *hte_root;
> > +
> > +static void __init hte_subsys_dbgfs_init(void)
> > +{
> > +	/* creates /sys/kernel/debug/hte/ */
> > +	hte_root = debugfs_create_dir("hte", NULL);
> > +}
> > +subsys_initcall(hte_subsys_dbgfs_init);
> > +
> > +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> > +{
> > +	const struct hte_chip *chip = gdev->chip;
> > +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
> > +
> > +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
> > +	if (!gdev->dbg_root)
> > +		return;
> > +
> > +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
> > +				&gdev->ts_req);
> > +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
> > +			   &gdev->nlines);
> > +}
> > +
> > +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> > +{
> > +	if (!ei->gdev->dbg_root || !name)
> > +		return;
> > +
> > +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
> > +	if (!ei->ts_dbg_root)
> > +		return;
> > +
> > +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
> > +			      &ei->buf->datum_len);
> > +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
> > +			      &ei->buf->watermark);
> > +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
> > +				&ei->dropped_ts);
> > +}
> > +
> > +static inline void hte_dbgfs_deinit(struct dentry *root)
> > +{
> > +	if (!root)
> > +		return;
> > +
> > +	debugfs_remove_recursive(root);
> > +}
> > +
> > +#else
> > +
> > +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> > +{
> > +}
> > +
> > +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> > +{
> > +}
> > +
> > +static inline void hte_dbgfs_deinit(struct dentry *root)
> > +{
> > +}
> > +
> > +#endif
> > +/* end of debugfs management*/
> > +
> > +/* Driver APIs */
> > +
> > +/**
> > + * hte_release_ts() - Consumer calls this API to release the entity, where
> > + * entity could be anything providers support, like lines, signals, buses,
> > + * etc...
> > + *
> > + * The correct sequence to call this API is as below:
> > + * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
> > + * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
> > + * still care about the data.
> > + * 3) Call this API.
> > + * Above sequence makes sure that entity gets released race free.
> > + *
> > + * @desc: timestamp descriptor, this is the same as returned by the request API.
> > + *
> > + * Context: hte_dbgfs_deinit() function call may use sleeping locks,
> > + *	    not suitable from atomic context in that case.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_release_ts(struct hte_ts_desc *desc)
> > +{
> > +	u32 id;
> > +	int ret = 0;
> > +	struct hte_device *gdev;
> > +	struct hte_ts_info *ei;
> > +	struct hte_ts_buf *buf;
> > +
> > +	if (!desc)
> > +		return -EINVAL;
> > +
> > +	ei = (struct hte_ts_info *)desc->data_subsys;  
> 
> As data_subsys is void * you don't need to explicitly cast it to another pointer type.
> 
> > +
> > +	if (!ei || !ei->gdev || !ei->buf)
> > +		return -EINVAL;
> > +
> > +	gdev = ei->gdev;
> > +	buf = ei->buf;
> > +	id = desc->con_id;
> > +
> > +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> > +		dev_info(gdev->sdev, "id:%d is not registered", id);
> > +		return -EUSERS;
> > +	}
> > +
> > +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
> > +	if (ret) {
> > +		dev_err(gdev->sdev, "id: %d free failed\n", id);
> > +		goto out;
> > +	}
> > +
> > +	atomic_dec(&gdev->ts_req);
> > +	atomic_set(&ei->dropped_ts, 0);
> > +
> > +	kfree(desc->name);
> > +	kfree(desc);
> > +	ei->desc = NULL;
> > +	ei->seq = 0;
> > +	buf->access->release(buf);
> > +
> > +	hte_dbgfs_deinit(ei->ts_dbg_root);
> > +	module_put(gdev->owner);
> > +
> > +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
> > +
> > +out:
> > +	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_release_ts);
> > +
> > +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
> > +{
> > +	u32 ts_id;
> > +	struct hte_device *gdev;
> > +	struct hte_ts_info *ei;
> > +	int ret;
> > +
> > +	if (!desc)
> > +		return -EINVAL;
> > +
> > +	ei = (struct hte_ts_info *)desc->data_subsys;  
> 
> As above, no need to cast - though it rather implies the type of data_subsys
> should not be void *.
> 
> > +
> > +	if (!ei || !ei->gdev)
> > +		return -EINVAL;
> > +
> > +	gdev = ei->gdev;
> > +	ts_id = desc->con_id;
> > +
> > +	mutex_lock(&ei->mlock);
> > +
> > +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> > +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
> > +		ret = -EUSERS;
> > +		goto out;
> > +	}
> > +
> > +	if (en) {
> > +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
> > +			ret = 0;
> > +			goto out;
> > +		}
> > +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
> > +		if (ret) {
> > +			dev_warn(gdev->sdev, "id: %d enable failed\n",
> > +				 ts_id);
> > +			goto out;
> > +		}
> > +
> > +		clear_bit(HTE_TS_DISABLE, &ei->flags);
> > +		ret = 0;  
> 
> ret is already 0 so no point in setting it again.
> 
> > +	} else {
> > +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
> > +			ret = 0;
> > +			goto out;
> > +		}
> > +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
> > +		if (ret) {
> > +			dev_warn(gdev->sdev, "id: %d disable failed\n",
> > +				 ts_id);
> > +			goto out;
> > +		}
> > +
> > +		set_bit(HTE_TS_DISABLE, &ei->flags);
> > +		ret = 0;
> > +	}
> > +
> > +out:
> > +	mutex_unlock(&ei->mlock);
> > +	return ret;
> > +}
> > +
> > +/**
> > + * hte_disable_ts() - Disable timestamp on given descriptor.
> > + *
> > + * @desc: ts descriptor, this is the same as returned by the request API.
> > + *
> > + * Context: Holds mutex lock, not suitable from atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_disable_ts(struct hte_ts_desc *desc)
> > +{
> > +	return hte_ts_dis_en_common(desc, false);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_disable_ts);
> > +
> > +/**
> > + * hte_enable_ts() - Enable timestamp on given descriptor.
> > + *
> > + * @desc: ts descriptor, this is the same as returned by the request API.
> > + *
> > + * Context: Holds mutex lock, not suitable from atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_enable_ts(struct hte_ts_desc *desc)
> > +{
> > +	return hte_ts_dis_en_common(desc, true);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_enable_ts);
> > +
> > +static int hte_simple_xlate(struct hte_chip *gc,
> > +			    const struct of_phandle_args *args,
> > +			    struct hte_ts_desc *desc,
> > +			    u32 *id)
> > +{
> > +	if (!id || !desc || !gc)
> > +		return -EINVAL;
> > +
> > +	/*
> > +	 * For the providers which do not have any internal mappings between
> > +	 * logically exposed ids and actual ids, will set both
> > +	 * the same.
> > +	 *
> > +	 * In case there is a internal mapping needed, providers will need to
> > +	 * provide its own xlate function where con_id will be sent as
> > +	 * args[0] and it will return xlated id. Later xlated id will be
> > +	 * used for any future exchanges between provider and subsystems.
> > +	 */
> > +
> > +	if (args) {
> > +		if (gc->of_hte_n_cells < 1)
> > +			return -EINVAL;
> > +
> > +		if (args->args_count != gc->of_hte_n_cells)
> > +			return -EINVAL;
> > +
> > +		*id = args->args[0];
> > +		desc->con_id = *id;
> > +	} else {
> > +		*id = desc->con_id;
> > +	}
> > +
> > +	if (desc->con_id > gc->nlines)
> > +		return -EINVAL;
> > +
> > +	desc->data_subsys = NULL;
> > +
> > +	return 0;
> > +}
> > +
> > +static struct hte_device *of_node_to_htedevice(struct device_node *np)
> > +{
> > +	struct hte_device *gdev;
> > +
> > +	spin_lock(&hte_lock);
> > +
> > +	list_for_each_entry(gdev, &hte_devices, list)
> > +		if (gdev->chip && gdev->chip->dev &&
> > +		    gdev->chip->dev->of_node == np) {
> > +			spin_unlock(&hte_lock);
> > +			return gdev;
> > +		}
> > +
> > +	spin_unlock(&hte_lock);
> > +
> > +	return ERR_PTR(-ENODEV);
> > +}
> > +
> > +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
> > +			 u32 xlated_id, void (*cb)(enum hte_notify n))
> > +{
> > +	struct hte_ts_info *ei;
> > +	struct hte_ts_buf *buf;
> > +	int ret;
> > +	u32 con_id = desc->con_id;
> > +
> > +	if (!try_module_get(gdev->owner))
> > +		return -ENODEV;
> > +
> > +	ei = &gdev->ei[xlated_id];
> > +	ei->xlated_id = xlated_id;
> > +
> > +	/*
> > +	 * There a chance that multiple consumers requesting same entity,
> > +	 * lock here.
> > +	 */
> > +	mutex_lock(&ei->mlock);
> > +
> > +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> > +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
> > +			xlated_id);
> > +		ret = -EUSERS;
> > +		goto unlock;
> > +	}
> > +
> > +	buf = hte_ts_buf_allocate();
> > +	if (IS_ERR(buf)) {
> > +		dev_err(gdev->chip->dev, "Buffer allocation failed");
> > +		ret = PTR_ERR(buf);
> > +		goto unlock;
> > +	}
> > +
> > +	/* Set default here, let consumer decide how much to set later */
> > +	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
> > +				      sizeof(struct hte_ts_data));
> > +  
> 
> It's good to keep to consistent style of no line break between a statement
> and it's error check.
> 
> > +	if (ret) {
> > +		dev_err(gdev->chip->dev, "Fifo set length failed");
> > +		goto buf_rel;
> > +	}
> > +
> > +	buf->access->reset(buf);
> > +	buf->valid = true;
> > +
> > +	ei->buf = buf;
> > +	ei->cb = cb;
> > +
> > +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
> > +	if (ret < 0) {
> > +		dev_err(gdev->chip->dev, "ts request failed\n");
> > +		goto buf_rel;
> > +	}
> > +
> > +	desc->data_subsys = ei;
> > +	ei->desc = desc;
> > +
> > +	atomic_inc(&gdev->ts_req);
> > +	set_bit(HTE_TS_REGISTERED, &ei->flags);
> > +	mutex_unlock(&ei->mlock);
> > +
> > +	if (!desc->name) {
> > +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
> > +		if (desc->name)
> > +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
> > +				  con_id);
> > +	}
> > +
> > +	hte_ts_dbgfs_init(desc->name, ei);
> > +
> > +	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
> > +		__func__, con_id, xlated_id);
> > +
> > +	return 0;
> > +
> > +buf_rel:
> > +	buf->access->release(buf);
> > +unlock:
> > +	module_put(gdev->owner);
> > +	mutex_unlock(&ei->mlock);
> > +
> > +	return ret;
> > +}
> > +
> > +static struct hte_device *of_hte_dev_get(struct device *dev,
> > +					 struct device_node *np,
> > +					 const char *label,
> > +					 struct of_phandle_args *args)
> > +{
> > +	struct hte_device *gdev = NULL;
> > +	int index = 0;
> > +	int err;
> > +
> > +	if (label) {
> > +		index = of_property_match_string(np, "hte-names", label);
> > +		if (index < 0)
> > +			return ERR_PTR(index);
> > +	}
> > +
> > +	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
> > +					 args);
> > +	if (err) {
> > +		pr_err("%s(): can't parse \"htes\" property\n", __func__);
> > +		return ERR_PTR(err);
> > +	}
> > +
> > +	gdev = of_node_to_htedevice(args->np);
> > +	if (IS_ERR(gdev)) {
> > +		pr_err("%s(): HTE chip not found\n", __func__);
> > +		of_node_put(args->np);
> > +		return gdev;
> > +	}
> > +
> > +	return gdev;
> > +}
> > +
> > +static struct hte_ts_desc *__hte_req_ts(struct device *dev,
> > +					struct device_node *np,
> > +					const char *label,
> > +					void (*cb)(enum hte_notify n))
> > +{
> > +	struct hte_device *gdev = NULL;
> > +	struct hte_ts_desc *desc;
> > +	struct of_phandle_args args;
> > +	int ret;
> > +	u32 xlated_id;
> > +
> > +	gdev = of_hte_dev_get(dev, np, label, &args);
> > +	if (IS_ERR(gdev))
> > +		return ERR_CAST(gdev);
> > +
> > +	if (!gdev->chip) {
> > +		pr_debug("requested id does not have provider\n");
> > +		return ERR_PTR(-ENODEV);
> > +	}
> > +
> > +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> > +	if (!desc)
> > +		return ERR_PTR(-ENOMEM);
> > +
> > +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
> > +	if (ret < 0)
> > +		goto put;
> > +
> > +	desc->name = NULL;
> > +	if (label)
> > +		desc->name = kstrdup(label, GFP_KERNEL);
> > +
> > +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> > +	if (ret < 0)
> > +		goto put;
> > +
> > +	return desc;
> > +
> > +put:
> > +	of_node_put(args.np);
> > +	kfree(desc);
> > +
> > +	return ERR_PTR(ret);
> > +}
> > +
> > +/**
> > + * of_hte_request_ts() - Consumer calls this API to request the HTE facility
> > + * on the specified entity, where entity is provider specific for example,
> > + * GPIO lines, signals, buses etc...
> > + *
> > + * @dev: Consumer device.
> > + * @label: Optional label.
> > + * @cb: Optional notify callback to consumer when data is pushed by the
> > + * provider.
> > + *
> > + * Context: Holds mutex lock, not suitable from atomic context.
> > + * Returns: Timestamp descriptor on success or error ptr on failure.
> > + */
> > +struct hte_ts_desc *of_hte_request_ts(struct device *dev,
> > +				      const char *label,
> > +				      void (*cb)(enum hte_notify n))
> > +{
> > +
> > +	if (dev && dev->of_node)
> > +		return __hte_req_ts(dev, dev->of_node, label, cb);
> > +	else
> > +		return ERR_PTR(-EOPNOTSUPP);
> > +}
> > +EXPORT_SYMBOL_GPL(of_hte_request_ts);
> > +
> > +static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)  
> 
> I'm not seeing what is devm about this.
> 
> > +{
> > +	struct hte_ts_desc **p = res;
> > +
> > +	if (WARN_ON(!p || !*p))
> > +		return 0;
> > +
> > +	return *p == data;
> > +}
> > +
> > +static void __devm_hte_release_ts(struct device *dev, void *res)
> > +{
> > +	hte_release_ts(*(struct hte_ts_desc **)res);
> > +}
> > +
> > +/**
> > + * devm_hte_release_ts() - Resource managed hte_release_ts().  
> 
> I'd not introduce this until you have a user.  It very rarely actually makes
> sense to call a devm release manually. Not having one makes people think harder
> about it.
> 
> > + * @dev: HTE consumer/client device.
> > + * @desc: HTE ts descriptor.
> > + *
> > + * Release timestamp functionality and its resources previously allocated using
> > + * of_hte_request_ts(). Calling this function is usually not needed because
> > + * devm-allocated resources are automatically released on driver detach.
> > + *
> > + * Context: Same as hte_release_ts() function.
> > + * Returns: 0 on success otherwise negative error code.
> > + */
> > +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
> > +{
> > +	return devres_release(dev, __devm_hte_release_ts,
> > +			      devm_hte_ts_match_desc, desc);
> > +}
> > +EXPORT_SYMBOL_GPL(devm_hte_release_ts);
> > +
> > +/**
> > + * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().  
> 
> If it's kernel-doc it needs to give no warnings when you point the kernel-doc
> scripts at it.  They insist on full parameter documentation.
> 
> > + */
> > +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> > +					   const char *label,
> > +					   void (*cb)(enum hte_notify n))
> > +{
> > +
> > +	struct hte_ts_desc **ptr, *desc;
> > +
> > +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);  
> 
> Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
> and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
> last kernel cycle.
> 
> 
> > +	if (!ptr)
> > +		return ERR_PTR(-ENOMEM);
> > +
> > +	desc = of_hte_request_ts(dev, label, cb);
> > +	if (!IS_ERR(desc)) {
> > +		*ptr = desc;
> > +		devres_add(dev, ptr);
> > +	} else {
> > +		devres_free(ptr);
> > +	}
> > +	return desc;
> > +}
> > +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
> > +
> > +static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
> > +					  size_t val)  
> 
> Not a good name or indeed combination of different things.
> hte_desc_to_info() and some separate check on val would be better.
> 
> > +{
> > +	struct hte_ts_info *ei;
> > +
> > +	if (!desc || !desc->data_subsys || !val) {
> > +		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
> > +		return NULL;
> > +	}
> > +
> > +	ei = desc->data_subsys;
> > +	if (!ei || !ei->buf) {
> > +		pr_debug("%s:%d\n", __func__, __LINE__);
> > +		return NULL;
> > +	}
> > +
> > +	return ei;
> > +}
> > +
> > +static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
> > +{
> > +	size_t el_avail;
> > +
> > +	el_avail = buffer->access->el_available(buffer);
> > +
> > +	return (el_avail >= to_read) ? false : true;  
> 
> return el_avail < to_read;
> 
> > +}
> > +
> > +static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> > +			       struct hte_ts_data *el, size_t n, bool block)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +	int ret;
> > +	size_t to_read, copied;
> > +
> > +	ei = hte_para_check(desc, n);
> > +	if (!ei)
> > +		return -EINVAL;
> > +
> > +	buffer = ei->buf;
> > +
> > +	to_read = min_t(size_t, n, buffer->watermark);  
> 
> Needs a comment as not obvious why you'd read the min of that requested or
> the watermark if there might be more available.
> 
> > +
> > +	do {
> > +		if (hte_ts_buf_wait(buffer, to_read)) {
> > +			if (!block) {
> > +				/* Possibly early here to retrieve, try again */
> > +				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
> > +					__func__, ret);
> > +				return -EAGAIN;
> > +			}
> > +			ret = wait_event_interruptible(buffer->pollq,
> > +					!hte_ts_buf_wait(buffer, to_read));
> > +			if (ret)
> > +				return ret;
> > +		}
> > +		ret = buffer->access->read(buffer, (void *)el,  
> 
> If you have to cast to a void * that usually means something is wrong in your definitions.
> Why is it needed here?  Looks like read has an inappropriate definition.
> 
> > +					   n * buffer->bytes_per_datum,
> > +					   &copied);
> > +		if (ret < 0)
> > +			return ret;
> > +
> > +		if (copied > 0)
> > +			return 0;
> > +		else if (copied == 0 && !block)
> > +			return -EAGAIN;
> > +	} while (copied == 0);
> > +
> > +	return 0;
> > +}
> > +
> > +/**
> > + * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
> > + * nano seconds i.e. el->tsc will be in ns.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + * @el: buffer to store the timestamp details.
> > + * @n: Number of struct hte_timestamp_el elements.
> > + *
> > + * Context: Can be called from the atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> > +		       struct hte_ts_data *el, size_t n)
> > +{
> > +	return _hte_retrieve_ts_ns(desc, el, n, false);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
> > +
> > +/**
> > + * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
> > + * @desc: ts descriptor, same as returned from request API.
> > + * @el: buffer to store the timestamp data.
> > + * @n: Number of struct hte_ts_data data.
> > + *
> > + * Context: Can not be called from the atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> > +			    struct hte_ts_data *el, size_t n)
> > +{
> > +	return _hte_retrieve_ts_ns(desc, el, n, true);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
> > +
> > +/**
> > + * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
> > + * depth.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + * @len: New length/depth.
> > + *
> > + * The correct sequence to set buffer length is as below:
> > + * 1) Disable timestamp by calling hte_disable_ts API.
> > + * 2) Optionally retrieve all the timestamps by calling non blocking
> > + *    hte_retrieve_ts_ns() API. This step only needed if you still care about
> > + *    the data.
> > + * 3) Call this API.
> > + * 4) Enable timestamp by calling hte_enable_ts API.
> > + *
> > + * This API destroys previously allocated buffer and creates new one, because
> > + * of that, it is mandatory to follow above sequence to make sure there is no
> > + * race between various other APIs in the subsystem.  
> 
> Good docs.  This is why I mentioned in review of docs patch that it is better
> to just have that refer to the kernel-doc in these files.  Keep all this good
> information in one place.
> 
> > + *
> > + * By default during the request API call, HTE subsystem allocates software
> > + * buffer with predefined length, this API gives flexibility to adjust the
> > + * length according to consumer's need.
> > + *
> > + * Context: Can not be called from atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +	int ret;
> > +
> > +	ei = hte_para_check(desc, len);
> > +	if (!ei)
> > +		return -EINVAL;
> > +
> > +	buffer = ei->buf;
> > +	ret = buffer->access->set_length(buffer, len,
> > +					 sizeof(struct hte_ts_data));
> > +	if (ret)
> > +		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);  
> 
> Not point in printing things line __func__ manually in dev_err() etc.
> Dynamic debug includes that and gives far more information + control of this.
> 
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_set_buf_len);
> > +
> > +/**
> > + * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
> > + * depth or length.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + *
> > + * Context: Any context.
> > + * Returns: Positive length on success or 0 on failure.
> > + */
> > +size_t hte_get_buf_len(const struct hte_ts_desc *desc)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +
> > +	ei = hte_para_check(desc, 1);
> > +	if (!ei)
> > +		return 0;
> > +
> > +	buffer = ei->buf;
> > +
> > +	return buffer->access->get_length(buffer);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_get_buf_len);
> > +
> > +/**
> > + * hte_available_ts() - Returns total available timestamps.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + *
> > + * The API helps consumers to pre-allocate its internal buffer required
> > + * during hte_retrieve_ts_ns call.
> > + *
> > + * Context: Any context.
> > + * Returns: Positive value if elements are available else 0. The value is
> > + * number of total available struct hte_timestamp_el elements available not
> > + * the size in bytes.
> > + */
> > +size_t hte_available_ts(const struct hte_ts_desc *desc)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +
> > +	ei = hte_para_check(desc, 1);
> > +	if (!ei)
> > +		return 0;
> > +
> > +	buffer = ei->buf;
> > +
> > +	return buffer->access->el_available(buffer);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_available_ts);
> > +
> > +/**
> > + * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
> > + * buffer watermark. The correct sequence to call this API is as below:
> > + * 1) Disable timestamp by calling hte_disable_ts API.
> > + * 2) Call this API.
> > + * 3) Enable timestamp by calling hte_enable_ts API.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + * @val: New watermark.
> > + *
> > + * By default during the request API call, HTE subsystem sets watermark as 1,
> > + * this API gives flexibility to adjust the watermark according to consumer's
> > + * need. The consumers will get notification through callback registered during
> > + * request API either when timestamp is dropped or watermark is reached or will
> > + * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
> > + * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
> > + *
> > + * Context: Any context.  
> 
> You have no way of knowing that as will depend on the driver - I'd definitely
> suggest not from atomic context, but then that would be crazy so you are better
>  off not documenting any specific requirement at all.
> 
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +	int ret;
> > +
> > +	ei = hte_para_check(desc, val);
> > +	if (!ei)
> > +		return -EINVAL;
> > +
> > +	buffer = ei->buf;
> > +	ret = buffer->access->set_watermark(buffer, val);
> > +	if (ret)
> > +		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
> > +
> > +/**
> > + * hte_get_buf_watermark() - Consumer calls this API to get software
> > + * buffer watermark.
> > + * @desc: ts descriptor, same as returned from request API.
> > + *
> > + * Context: Any context.
> > + * Returns: Positive current watermark on success or 0 on failure.
> > + */
> > +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
> > +{
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +
> > +	ei = hte_para_check(desc, 1);
> > +	if (!ei)
> > +		return 0;
> > +
> > +	buffer = ei->buf;
> > +
> > +	return buffer->access->get_watermark(buffer);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
> > +
> > +/**
> > + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
> > + * node directly, where meaning of the entity is provider specific, for example
> > + * lines, signals, GPIOs, buses etc...
> > + *
> > + * @of_node: HTE provider device node.
> > + * @id: entity id to monitor, this id belongs to HTE provider of_node.
> > + * @cb: Optional callback to notify.
> > + *
> > + * Context: Holds mutex lock, can not be called from atomic context.  
> 
> What mutex and why?  If it is one you can check is held even better.
> 
> > + * Returns: ts descriptor on success or error pointers.
> > + */
> > +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> > +					  unsigned int id,
> > +					  void (*cb)(enum hte_notify n))
> > +{
> > +	struct hte_device *gdev;
> > +	struct hte_ts_desc *desc;
> > +	int ret;
> > +	u32 xlated_id;
> > +
> > +	gdev = of_node_to_htedevice(of_node);
> > +	if (IS_ERR(gdev))
> > +		return ERR_PTR(-ENOTSUPP);
> > +
> > +	if (!gdev->chip || !gdev->chip->ops)
> > +		return ERR_PTR(-ENOTSUPP);
> > +
> > +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> > +	if (!desc) {
> > +		ret = -ENOMEM;
> > +		goto out_put_device;
> > +	}  
> 
> Pass a desc pointer into this function rather than allocating the structure
> in here.  That lets the caller embed that structure inside one of it's own
> structures if it wants to, resulting in fewer small allocations which is always good.
> 
> It's far from obvious that the caller needs to free desc.
> 
> > +
> > +	desc->con_id = id;
> > +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
> > +	if (ret < 0) {
> > +		dev_err(gdev->chip->dev,
> > +			"failed to xlate id: %d\n", id);
> > +		goto out_free_desc;
> > +	}
> > +
> > +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> > +	if (ret < 0) {
> > +		dev_err(gdev->chip->dev,
> > +			"failed to request id: %d\n", id);
> > +		goto out_free_desc;
> > +	}
> > +
> > +	return desc;
> > +
> > +out_free_desc:
> > +	kfree(desc);
> > +
> > +out_put_device:
> > +	return ERR_PTR(ret);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
> > +
> > +/**
> > + * hte_get_clk_src_info() - Consumer calls this API to query clock source
> > + * information of the desc.
> > + *
> > + * @desc: ts descriptor, same as returned from request API.
> > + *
> > + * Context: Any context.
> > + * Returns: 0 on success else negative error code on failure.
> > + */
> > +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> > +			 struct hte_clk_info *ci)
> > +{
> > +	struct hte_chip *chip;
> > +	struct hte_ts_info *ei;
> > +
> > +	if (!desc || !desc->data_subsys || !ci) {
> > +		pr_debug("%s:%d\n", __func__, __LINE__);
> > +		return -EINVAL;
> > +	}
> > +
> > +	ei = desc->data_subsys;
> > +	if (!ei || !ei->gdev || !ei->gdev->chip)
> > +		return -EINVAL;
> > +
> > +	chip = ei->gdev->chip;
> > +	if (!chip->ops->get_clk_src_info)
> > +		return -ENOTSUPP;
> > +
> > +	return chip->ops->get_clk_src_info(chip, ci);
> > +}
> > +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
> > +
> > +static inline void hte_add_to_device_list(struct hte_device *gdev)
> > +{
> > +	struct hte_device *prev;  
> 
> Needs to take an appropriate lock as you may have concurrent calls.
> 
> > +
> > +	if (list_empty(&hte_devices)) {
> > +		list_add_tail(&gdev->list, &hte_devices);  
> 
> Needs a comment. I've no idea why you might want to only add it if there were
> no other hte_devices already there.
> 
> > +		return;
> > +	}
> > +
> > +	prev = list_last_entry(&hte_devices, struct hte_device, list);  
> Why woud you do this?
> 
> > +	list_add_tail(&gdev->list, &hte_devices);
> > +}
> > +
> > +/**
> > + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
> > + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
> > + * using this API from its ISR or atomic context.
> > + *
> > + * @chip: The HTE chip, used during the registration.
> > + * @xlated_id: entity id understood by both subsystem and provider, usually this
> > + * is obtained from xlate callback during request API.
> > + * @data: timestamp data.
> > + * @n: Size of the data.
> > + *
> > + * Context: Atomic.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> > +			  struct hte_ts_data *data, size_t n)
> > +{
> > +	unsigned int ret;
> > +	bool notify;
> > +	size_t el_avail;
> > +	struct hte_ts_buf *buffer;
> > +	struct hte_ts_info *ei;
> > +
> > +	if (!chip || !data || !chip->gdev)
> > +		return -EINVAL;
> > +
> > +	if (xlated_id > chip->nlines)
> > +		return -EINVAL;
> > +
> > +	ei = &chip->gdev->ei[xlated_id];
> > +
> > +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
> > +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
> > +		dev_dbg(chip->dev, "Unknown timestamp push\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* timestamp sequence counter, start from 0 */
> > +	data->seq = ei->seq++;
> > +
> > +	buffer = ei->buf;
> > +	el_avail = buffer->access->el_available(buffer);  
> 
> > +	ret = buffer->access->store(buffer, data, n);  
> 
> If we are doing this from the hte core, why is buffer definition in the scope of the
> drivers rather than the core?  That seems backwards to me.
> 
> > +	if (ret != n) {
> > +		atomic_inc(&ei->dropped_ts);
> > +		if (ei->cb)
> > +			ei->cb(HTE_TS_DROPPED);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;  
> 
> You push n but only check on el_avail + 1 here.  
> Also, this is the same as
> 
> 	notify = ((el_avail + 1) >= buffer->watermark;
> 
> 
> > +
> > +	/*
> > +	 * If there is a callback, its consumer's job to retrieve the timestamp.
> > +	 * For the rest, wake up the process.
> > +	 */
> > +	if (notify && ei->cb) {
> > +		ei->cb(HTE_TS_AVAIL);
> > +		return 0;  
> 
> Given you return 0 anyway, might as well not have this line.
> 
> > +	} else if (notify) {
> > +		wake_up_interruptible(&buffer->pollq);
> > +	}
> > +
> > +	return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
> > +
> > +/**
> > + * hte_register_chip() - Used by provider to register a HTE chip.
> > + * @chip: the HTE chip to add to subsystem.
> > + *
> > + * Context: Can not be called from atomic context.  
> 
> Whilst true, I'd think that was common sense as it would be insane
> to register something like this from atomic context.  So I'd say no
> need to comment on it!  Keep those comments for things that
> might be used like that.
> 
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_register_chip(struct hte_chip *chip)
> > +{
> > +	struct hte_device *gdev;
> > +	int ret;
> > +	u32 i;
> > +
> > +	if (!chip || !chip->dev || !chip->dev->of_node)
> > +		return -EINVAL;
> > +
> > +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
> > +		dev_err(chip->dev, "Driver needs to provide ops\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
> > +	if (!gdev)
> > +		return -ENOMEM;
> > +
> > +	gdev->chip = chip;
> > +	chip->gdev = gdev;
> > +	gdev->nlines = chip->nlines;
> > +	gdev->sdev = chip->dev;
> > +
> > +	/*
> > +	 * Allocate all the supported entities here at once, this will have
> > +	 * following advantages:
> > +	 * When provider pushes timestamp, it can then just send the
> > +	 * xlated_id, subsystem will use it as an index which
> > +	 * gives us the constant time access; this is important as mostly
> > +	 * providers will be pushing the timestamps from their ISR.
> > +	 */
> > +	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
> > +			   GFP_KERNEL);  
> 
> I'd be tempted to do this as a 0 length element at the end of gdev
> then do the allocation in one go use struct_size() etc to work out
> how long it is.  Cuts down on allocations + error paths to deal with
> for no obvious disadvantage.
> 
> > +	if (!gdev->ei) {
> > +		ret = -ENOMEM;
> > +		goto err_free_gdev;
> > +	}
> > +
> > +	for (i = 0; i < chip->nlines; i++) {
> > +		gdev->ei[i].flags = 0;  
> 
> zero allocated, so don't bother setting things to 0 where it's a fairly obvious
> base state.  If you set something to 0 to act as some form of documentation then
> that's fine, but I don't think that's true here.
> 
> > +		gdev->ei[i].gdev = gdev;
> > +		gdev->ei[i].seq = 0;
> > +		mutex_init(&gdev->ei[i].mlock);
> > +	}
> > +
> > +	if (chip->dev->driver)
> > +		gdev->owner = chip->dev->driver->owner;
> > +	else
> > +		gdev->owner = THIS_MODULE;
> > +
> > +	if (!chip->xlate) {
> > +		chip->xlate = hte_simple_xlate;
> > +		/* Just a id number to monitor */
> > +		chip->of_hte_n_cells = 1;
> > +	}
> > +
> > +	of_node_get(chip->dev->of_node);
> > +
> > +	INIT_LIST_HEAD(&gdev->list);
> > +
> > +	spin_lock(&hte_lock);
> > +	hte_add_to_device_list(gdev);
> > +	spin_unlock(&hte_lock);
> > +
> > +	hte_chip_dbgfs_init(gdev);
> > +
> > +	dev_dbg(chip->dev, "Added hte chip\n");
> > +	return 0;
> > +
> > +err_free_gdev:
> > +	kfree(gdev);
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_register_chip);
> > +
> > +/**
> > + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
> > + * @chip: the HTE chip to remove.
> > + *
> > + * Context: Can not be called from atomic context.
> > + * Returns: 0 on success or a negative error code on failure.
> > + */
> > +int hte_unregister_chip(struct hte_chip *chip)
> > +{
> > +	struct hte_device *gdev = chip->gdev;
> > +
> > +	spin_lock(&hte_lock);
> > +	list_del(&gdev->list);
> > +	spin_unlock(&hte_lock);
> > +
> > +	gdev->chip = NULL;
> > +
> > +	of_node_put(chip->dev->of_node);
> > +	hte_dbgfs_deinit(gdev->dbg_root);
> > +	kfree(gdev->ei);
> > +	kfree(gdev);
> > +
> > +	dev_dbg(chip->dev, "Removed hte chip\n");
> > +	return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(hte_unregister_chip);
> > +
> > +/* Driver APIs ends */  
> 
> Don't bother with file layout type comments.  They don't add that much and tend
> to rot horribly over time as people move code around in files.
> 
> > diff --git a/include/linux/hte.h b/include/linux/hte.h
> > new file mode 100644
> > index 000000000000..e1737579d4c4
> > --- /dev/null
> > +++ b/include/linux/hte.h
> > @@ -0,0 +1,278 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2021 NVIDIA Corporation
> > + *
> > + * Author: Dipen Patel <dipenp@nvidia.com>
> > + */
> > +
> > +#ifndef __LINUX_HTE_H
> > +#define __LINUX_HTE_H
> > +
> > +struct hte_chip;
> > +struct hte_device;
> > +struct of_phandle_args;
> > +
> > +/**
> > + * Used by providers to indicate the direction of the timestamp.
> > + */
> > +#define HTE_EVENT_RISING_EDGE          0x1
> > +#define HTE_EVENT_FALLING_EDGE         0x2  
> 
> Use an enum rather than a define for this as it's a value that can take a
> set of distinct values.  Also, provide a name for 'I've no idea' which
> I'm guessing is 0 currently.
> 
> > +
> > +/**
> > + * struct hte_ts_data - HTE timestamp data.
> > + * The provider uses and fills timestamp related details during push_timestamp
> > + * API call. The consumer uses during retrieve_timestamp API call.
> > + *
> > + * @tsc: Timestamp value.
> > + * @seq: Sequence counter of the timestamps.
> > + * @dir: Direction of the event at the time of timestamp.
> > + */
> > +struct hte_ts_data {
> > +	u64 tsc;
> > +	u64 seq;
> > +	int dir;
> > +};
> > +
> > +/**
> > + * struct hte_clk_info - Clock source info that HTE provider uses.
> > + * The provider uses hardware clock as a source to timestamp real time. This
> > + * structure presents the clock information to consumers. 
> > + *
> > + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
> > + * @type: Clock type. CLOCK_* types.  
> 
> So this is something we got a it wrong in IIO. It's much better to define
> a subset of clocks that can be potentially used.  There are some that make
> absolutely no sense and consumers really don't want to have to deal with them.
>  
> > + */
> > +struct hte_clk_info {
> > +	u64 hz;
> > +	clockid_t type;
> > +};
> > +
> > +/**
> > + * HTE subsystem notifications for the consumers.
> > + *
> > + * @HTE_TS_AVAIL: Timestamps available notification.
> > + * @HTE_TS_DROPPED: Timestamps dropped notification.  
> 
> Something I've missed so far is whether drops are in a kfifo or a ring
> fashion.  I'm guess that's stated somewhere, but it might be useful to have
> it here.
> 
> > + */
> > +enum hte_notify {
> > +	HTE_TS_AVAIL = 1,
> > +	HTE_TS_DROPPED,
> > +	HTE_NUM_NOTIFIER,
> > +};
> > +
> > +/**
> > + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
> > + * communication token between consumers to subsystem and subsystem to
> > + * providers.
> > + *
> > + * @con_id: This is the same id sent in request APIs.
> > + * @name: Descriptive name of the entity that is being monitored for the
> > + * realtime timestamping.
> > + * @data_subsys: Subsystem's private data relate to requested con_id.
> > + */
> > +struct hte_ts_desc {
> > +	u32 con_id;
> > +	char *name;
> > +	void *data_subsys;
> > +};
> > +
> > +/**
> > + * struct hte_ops - HTE operations set by providers.
> > + *
> > + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
> > + * non-zero for failures.
> > + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
> > + * non-zero for failures.
> > + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
> > + * non-zero for failures.
> > + * @disable: Hook to disable specified timestamp. Returns 0 on success,
> > + * non-zero for failures.
> > + * @get_clk_src_info: Optional hook to get the clock information provider uses
> > + * to timestamp. Returns 0 for success and negative error code for failure. On
> > + * success HTE subsystem fills up provided struct hte_clk_info.  
> 
> Why optional?  Consumers will probably need that information.
> 
> > + *
> > + * xlated_id parameter is used to communicate between HTE subsystem and the
> > + * providers. It is the same id returned during xlate API call and translated
> > + * by the provider. This may be helpful as both subsystem and provider locate
> > + * the requested entity in constant time, where entity could be anything from
> > + * lines, signals, events, buses etc.. that providers support.
> > + */
> > +struct hte_ops {
> > +	int (*request)(struct hte_chip *chip, u32 xlated_id);
> > +	int (*release)(struct hte_chip *chip, u32 xlated_id);
> > +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
> > +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
> > +	int (*get_clk_src_info)(struct hte_chip *chip,
> > +				struct hte_clk_info *ci);
> > +};
> > +
> > +/**
> > + * struct hte_chip - Abstract HTE chip structure.
> > + * @name: functional name of the HTE IP block.
> > + * @dev: device providing the HTE.  
> 
> Unclear naming.  Is this the parent device, or one associated with the HTE itself?
> I'm guessing today you don't have one associated with the HTE, but it is plausible you
> might gain on in future to make it fit nicely in the device model as a function of another
> device.
> 
> > + * @ops: callbacks for this HTE.
> > + * @nlines: number of lines/signals supported by this chip.
> > + * @xlate: Callback which translates consumer supplied logical ids to
> > + * physical ids, return from 0 for the success and negative for the
> > + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
> > + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
> > + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
> > + * @data: chip specific private data.
> > + */
> > +struct hte_chip {
> > +	const char *name;
> > +	struct device *dev;
> > +	const struct hte_ops *ops;
> > +	u32 nlines;
> > +	int (*xlate)(struct hte_chip *gc,
> > +		     const struct of_phandle_args *args,
> > +		     struct hte_ts_desc *desc, u32 *xlated_id);
> > +	u8 of_hte_n_cells;
> > +
> > +	/* only used internally by the HTE framework */
> > +	struct hte_device *gdev;
> > +	void *data;
> > +};
> > +
> > +#if IS_ENABLED(CONFIG_HTE)
> > +/* HTE APIs for the providers */
> > +int hte_register_chip(struct hte_chip *chip);
> > +int hte_unregister_chip(struct hte_chip *chip);
> > +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> > +			  struct hte_ts_data *data, size_t n);
> > +
> > +/* HTE APIs for the consumers */
> > +
> > +int hte_release_ts(struct hte_ts_desc *desc);
> > +struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
> > +				      void (*cb)(enum hte_notify n));
> > +
> > +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> > +					   const char *label,
> > +					   void (*cb)(enum hte_notify n));
> > +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> > +					  unsigned int id,
> > +					  void (*cb)(enum hte_notify n));
> > +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
> > +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
> > +		       size_t n);
> > +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> > +			    struct hte_ts_data *el, size_t n);
> > +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
> > +size_t hte_get_buf_len(const struct hte_ts_desc *desc);
> > +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
> > +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
> > +size_t hte_available_ts(const struct hte_ts_desc *desc);
> > +int hte_enable_ts(struct hte_ts_desc *desc);
> > +int hte_disable_ts(struct hte_ts_desc *desc);
> > +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> > +			 struct hte_clk_info *ci);
> > +
> >  
> 


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
  2021-06-27 17:40   ` Randy Dunlap
  2021-07-04 20:15   ` Jonathan Cameron
@ 2021-07-05  7:30   ` Greg KH
  2021-07-28  0:34     ` Dipen Patel
  2 siblings, 1 reply; 75+ messages in thread
From: Greg KH @ 2021-07-05  7:30 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Fri, Jun 25, 2021 at 04:55:23PM -0700, Dipen Patel wrote:
> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> +{
> +	const struct hte_chip *chip = gdev->chip;
> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
> +
> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
> +	if (!gdev->dbg_root)
> +		return;

No need to check for this, if it fails, your other debugfs calls
will handle it just fine.


> +
> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
> +				&gdev->ts_req);
> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
> +			   &gdev->nlines);
> +}
> +
> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> +{
> +	if (!ei->gdev->dbg_root || !name)
> +		return;
> +
> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
> +	if (!ei->ts_dbg_root)
> +		return;

Again, no need to check.

> +
> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
> +			      &ei->buf->datum_len);
> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
> +			      &ei->buf->watermark);
> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
> +				&ei->dropped_ts);
> +}
> +
> +static inline void hte_dbgfs_deinit(struct dentry *root)
> +{
> +	if (!root)
> +		return;

No need to check this.

> +
> +	debugfs_remove_recursive(root);

Do not wrap a single call with another call :)


thanks,

greg k-h

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
  2021-07-01 14:21   ` Kent Gibson
  2021-07-04 20:27   ` Jonathan Cameron
@ 2021-07-08 23:33   ` Michał Mirosław
  2021-07-29  2:43     ` Dipen Patel
  2 siblings, 1 reply; 75+ messages in thread
From: Michał Mirosław @ 2021-07-08 23:33 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
> Tegra194 device has multiple HTE instances also known as GTE
> (Generic hardware Timestamping Engine) which can timestamp subset of
> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> and exposes timestamping ability on those lines to the consumers
> through HTE subsystem.
[...]
> +	ret = of_property_read_u32(dev->of_node, "slices", &slices);
> +	if (ret != 0) {
> +		dev_err(dev, "Could not read slices\n");
> +		return -EINVAL;
> +	}
> +
> +	hte_dev->sl = devm_kzalloc(dev, sizeof(struct hte_slices) * slices,
> +				   GFP_KERNEL);

Nit: There is devm_kcalloc() that will check for overflow in the
multiply in case @slices from DT is broken.

Best Regards
Michał Mirosław

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
                     ` (2 preceding siblings ...)
  2021-07-01 14:24   ` Kent Gibson
@ 2021-07-09  8:30   ` Jon Hunter
  2021-07-30  2:33     ` Dipen Patel
  3 siblings, 1 reply; 75+ messages in thread
From: Jon Hunter @ 2021-07-09  8:30 UTC (permalink / raw)
  To: Dipen Patel, thierry.reding, linux-kernel, linux-tegra,
	linux-gpio, linus.walleij, bgolaszewski, warthog618, devicetree,
	linux-doc, robh+dt


On 26/06/2021 00:55, Dipen Patel wrote:
> This patch adds new clock type for the GPIO controller which can
> timestamp gpio lines using hardware means. To expose such
> functionalities to the userspace, code has been added in this patch
> where during line create call, it checks for new clock type and if
> requested, calls hardware timestamp related API from gpiolib.c.
> During line change event, it retrieves timestamp in nano seconds by
> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
> it disables this functionality by calling gpiod_hw_timestamp_control.
> 
> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> ---
>  drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>  include/uapi/linux/gpio.h   |  1 +
>  2 files changed, 64 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> index 1631727bf0da..9f98c727e937 100644
> --- a/drivers/gpio/gpiolib-cdev.c
> +++ b/drivers/gpio/gpiolib-cdev.c
> @@ -518,6 +518,7 @@ struct linereq {
>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>  	 GPIO_V2_LINE_BIAS_FLAGS)
>  
>  static void linereq_put_event(struct linereq *lr,
> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>  
>  static u64 line_event_timestamp(struct line *line)
>  {
> +	bool block;
> +
>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>  		return ktime_get_real_ns();
>  
> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> +		if (irq_count())
> +			block = false;
> +		else
> +			block = true;
> +
> +		return gpiod_get_hw_timestamp(line->desc, block);
> +	}
> +
>  	return ktime_get_ns();
>  }


Looking at line_event_timestamp() and the callers of this function, it
appears that this should always return nanoseconds. Does
gpiod_get_hw_timestamp() return nanoseconds?

Jon

-- 
nvpublic

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 01/11] Documentation: Add HTE subsystem guide
  2021-07-04 18:55   ` Jonathan Cameron
@ 2021-07-27 23:44     ` Dipen Patel
  2021-08-01 15:24       ` Jonathan Cameron
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-27 23:44 UTC (permalink / raw)
  To: Jonathan Cameron
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt


On 7/4/21 11:55 AM, Jonathan Cameron wrote:
> On Fri, 25 Jun 2021 16:55:22 -0700
> Dipen Patel <dipenp@nvidia.com> wrote:
>
>> Adding hte document which can help understand various APIs implemented
>> in HTE framework for the HTE producers and the consumers.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Some editorial stuff inline. (I can't resist even on RFCs)
>
> Certainly interesting. I'm running a bit tight on time today, so not sure how
> much of the code I'll get a chance to look at.  Will try to get to it soon though.
>
> Jonathan
Thanks Jonathan for the review comment and time. My answers inline.
>
>> ---
>>  Documentation/hte/hte.rst | 198 ++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 198 insertions(+)
>>  create mode 100644 Documentation/hte/hte.rst
>>
>> diff --git a/Documentation/hte/hte.rst b/Documentation/hte/hte.rst
>> new file mode 100644
>> index 000000000000..11744dbc6d16
>> --- /dev/null
>> +++ b/Documentation/hte/hte.rst
>> @@ -0,0 +1,198 @@
>> +============================================
>> +The Linux Hardware Timestamping Engine (HTE)
>> +============================================
>> +
>> +:Author: Dipen Patel
>> +
>> +Introduction
>> +------------
>> +
>> +The certain devices have the built in hardware timestamping engine which can
> Certain devices have built in hardware timestamping engines which can
>
>> +monitor sets of system signals, lines, buses etc... in realtime for the state
> for state changes;
>
>> +change; upon detecting the change it can automatically store the timestamp at
> they can
Will add above in next RFC version2.
>
>> +the moment of occurrence. Such functionality may help achieve better accuracy
>> +in obtaining timestamp than using software counterparts i.e. ktime and friends.
>> +
>> +This document describes the API that can be used by hardware timestamping
>> +engine provider and consumer drivers that want to use the hardware timestamping
>> +engine (HTE) framework.
>> +
>> +The HTE framework APIs for the providers
>> +----------------------------------------
>> +Each driver must #include <linux/hte.h>. The ``linux/hte.h`` declares the
>> +following functions for the provider:
>> +
>> +.. c:function:: int hte_register_chip( struct hte_chip *chip )
>> +		int hte_unregister_chip( struct hte_chip *chip )
>> +
>> +	The provider uses these APIs to un/register itself with HTE framework.
>> +
>> +.. c:function:: int hte_push_ts_ns_atomic( const struct hte_chip *chip, u32 xlated_id, struct hte_ts_data *data, size_t n )
>> +
>> +	The provider pushes timestamp data in nano seconds unit using this API.
>> +
>> +The detail about parameters and API usage are described in each functions
>> +definitions in ``drivers/hte/hte.c`` file.
>> +
>> +The HTE framework APIs for the consumers
>> +----------------------------------------
>> +The consumers use following APIs to control the line for the timestamp:
>> +
> When documenting APIs you may well be better including a reference to the files
> themselves and using kernel doc there.  The documentation build can then pull that
> in when creating the html docs etc (and crucially you don't have to provide the
> same docs in two places.).   Having them here is very convenient for the RFC however :)
You mean to omit description here and put reference to file like ``drivers/hte/hte.c``?
>
>> +.. c:function:: int hte_release_ts( struct hte_ts_desc *desc )
>> +		int devm_hte_release_ts( struct device *dev, struct hte_ts_desc *desc )
>> +
>> +	The consumer uses API to release specified desc from timestamping.
>> +	The API frees resources associated with the desc and disables the
>> +	timestamping on it. The later is managed version of the same API.
>> +
>> +.. c:function:: struct hte_ts_desc *of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
>> +		struct hte_ts_desc *devm_of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
>> +
>> +	The consumers can use above request APIs to request real timestamp
>> +	capability on specified entity. The later is resource managed version
>> +	of the of_hte_request_ts API. Both the APIs expect consumer to follow
>> +	device tree bindings for the HTE consumer. The details about binding
>> +	is in ``Documentation/devicetree/bindings/hte/hte-consumer.yaml``.
>> +
>> +.. c:function:: struct hte_ts_desc *hte_req_ts_by_dt_node( struct device_node *of_node, unsigned int id, void (*cb)(enum hte_notify n) )
>> +
>> +	The consumer can request timestamping directly specifying provider
>> +	device tree node.
> When does this make sense?

This is needed when provider has dependencies on other IP within chip, for example

tegra chip GPIO HTE has to talk to GPIO controller to fully enable HTE functionality.

>
>> +
>> +.. c:function:: int hte_enable_ts( struct hte_ts_desc *desc )
>> +.. c:function:: int hte_disable_ts( struct hte_ts_desc *desc )
>> +
>> +	The consumer can enable/disable timestamping on given desc.
>> +
>> +.. c:function:: int hte_retrieve_ts_ns( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
>> +		int hte_retrieve_ts_ns_wait( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
>> +
>> +	The consumer uses above two API versions to get/retrieve timestamp data
>> +	for the given desc. The later is blocking version.
>> +
>> +.. c:function:: hte_get_clk_src_info(const struct hte_line_desc *desc, struct hte_clk_info *ci)
>> +
>> +	The consumer retrieves clock source information that provider uses to
>> +	timestamp entity in the structure hte_clk_info. This information
>> +	specifies clock rate in HZ and clock.
>> +
>> +The details on struct hte_clk_info
>> +-----------------------------------
>> +This structure presents detail of the hardware clock that provider uses for
>> +realtime timestamping purposes. The consumer can use hte_get_clk_src_info API
>> +to get the information in hte_clk_info structure. It has hz and type parameters
>> +where hz represents clock rate in HZ and type is clock type of clockid_t and
>> +of CLOCK_* family (for example, CLOCK_MONOTONIC).
>> +
>> +The consumers calling of_hte_request_ts or hte_req_ts_by_dt_node APIs with
>> +cb parameter set, usually will call hte_retrieve_ts (non blocking
>> +version) after being notified by the callbacks from HTE subsystem. The
>> +consumers calling those requests APIs with cb parameter NULL, usually will call
>> +hte_retrieve_ts_wait API.
>> +
>> +The HTE subsystem provides software buffer per requested id/entity to store
>> +timestamp data (struct hte_ts_data type). The consumers can manage the buffer.
>> +It also provides buffer watermark which can notify (if cb parameter is provided
>> +during request API call) consumer or unblock consumers calling
>> +hte_retrieve_ts_wait API. The following APIs are used to manipulate the
>> +software buffer:
> Have you come across any devices that have a hardware fifo for these timestamps?
> It's moderately common on sensor hubs to do so, and then you get into a fun question
> of how to manage the watermark.  You don't want to pull from the hardware too early,
> but conversely you can get out of sync between the software and hardware buffers if
> someone reasons less than 'watermark' samples from the software buffer.
>
> Anyhow, it can be entertaining.  So in those cases it can be simpler to explicitly provide
> control of two separate watermarks.

The provider I have dealt with had single hardware FIFO to store timestamps

indiscriminately. I am sure this will come up in future in which case we can

expand it to separate watermark.

>
>> +
>> +.. c:function:: int hte_set_buf_len( const struct hte_ts_desc *desc,unsigned int len )
>> +		int hte_get_buf_len( const struct hte_ts_desc *desc )
>> +
>> +	The consumer uses above APIs to set/get software buffer depth.
> What happens if there is content when it is resized?

I have described in the hte_set_buf_len API description. To summarize, you can

follow certain sequences to consume old data if you still care. Otherwise this

is a destructive API.

>
>> +
>> +.. c:function:: int hte_set_buf_watermark( const struct hte_ts_desc *desc, unsigned int val )
>> +		int hte_get_buf_watermark( const struct hte_ts_desc *desc )
>> +
>> +	The consumer uses above APIs to set/get software threshold, threshold
>> +	can be used to notity or unblock waiting consumer when data becomes
>> +	available equal or above to threshold value.
>> +
>> +.. c:function:: size_t hte_available_ts( const struct hte_ts_desc *desc )
>> +
>> +	The consumer uses above API to get available timestamp data stored
>> +	in the software buffer for the desc.
>> +
>> +The detail about parameters and API usage are described in each functions
>> +definitions in ``drivers/hte/hte.c`` file.
>> +
>> +The HTE timestamp element detail
>> +--------------------------------
>> +The struct hte_ts_data, declared at ``include/linux/hte.h``, is used to pass
>> +timestamp details between the consumers and the providers. It expresses
>> +timestamp data in nano second in u64 data type.
> I'd suggest s64 to match with kernel timestamp format.
Make sense, I will update in next revision.
>
>> For now all the HTE APIs
>> +using struct hte_ts_data requires tsc to be in nano seconds. The timestamp
>> +element structure stores below information along with timestamp data::
>> +
>> + struct hte_ts_data {
>> +	/*
>> +	 * Timestamp value
>> +	 */
>> +	u64 tsc;
>> +	/*
>> +	 * The sequence counter, keep track of the number of timestamps.
>> +	 * It can be used to check if data is dropped in between.
>> +	 */
> Is this a hardware feature?  A bit unusual to have this rather than simple
> overflow flag to indicate we dropped an unknown number of samples.
Its software feature. I Believe having seq helps consumer to backtrack.
>
>> +	u64 seq;
>> +	/* Direction of the event, i.e. falling or rising */
>> +	int dir;
> Given an even could do more than that potentially, or indeed not be able to
> tell if it was rising or falling, I would suggest an enum to which we can add
> more options as needed.
I have two defines in hte.h for now. I can convert them into enum type.
>
>> + };
>> +
>> +The typical hte_ts_data data life cycle::
>> +In this example the provider provides timestamp in nano seconds and for the
>> +GPIO line::
>> +
>> + - Monitors GPIO line change.
>> + - Detects the state change on GPIO line.
>> + - Converts timestamps in nano seconds and stores it in tsc.
>> + - Stores GPIO direction in dir variable if the provider has that hardware
>> + capability.
> We definitely want to know if it does or not.  How does an application query that?
Its stored in dir field of the hte_ts_data structure.
>
>> + - Pushes this hte_timestamp_el object to HTE subsystem.
>> + - HTE subsystem increments seq counter and stores it in software buffer
>> + dedicated to requested GPIO line.
> Ah. So that seq counter is only for software drops if the fifo fills up.
Yes.
>
>> + - Waiting consumer gets notified.
>> + - The consumer calls the retrieve timestamp API.
>> +
>> +HTE subsystem debugfs attributes
>> +--------------------------------
>> +HTE subsystem creates debugfs attributes at ``/sys/kernel/debug/hte/``.
>> +It also creates line/signal related debugfs attributes at
>> +``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
>> +
>> +`ts_requested`
>> +		The total number of entities requested from the given provider,
>> +		where entity is the provider specific and could represent
>> +		lines, GPIO, chip signals, buses etc...
>> +                The attribute will be availble at
>> +		``/sys/kernel/debug/hte/<provider>/``.
>> +
>> +		Read only value
>> +
>> +`total_ts`
>> +		The total number of entities supported by the provider.
>> +                The attribute will be availble at
>> +		``/sys/kernel/debug/hte/<provider>/``.
>> +
>> +		Read only value
>> +
>> +`ts_buffer_depth`
>> +		The software buffer lenth to store timestamp data.
>> +                The attribute will be availble at
>> +		``/sys/kernel/debug/hte/<provider>/<label or id>/``.
>> +
>> +		Read only value
>> +
>> +`ts_buffer_watermark`
>> +		The software buffer watermark or threshold.
>> +                The attribute will be availble at
>> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
>> +
>> +		Read only value
>> +
>> +`dropped_timestamps`
>> +		The dropped timestamps for a given line.
>> +                The attribute will be availble at
>> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
>> +
>> +		Read only value

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-05  7:30   ` Greg KH
@ 2021-07-28  0:34     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-28  0:34 UTC (permalink / raw)
  To: Greg KH
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

Thanks Greg for the review comments. I will address all in next RFC version2.

Best Regards,

Dipen Patel

On 7/5/21 12:30 AM, Greg KH wrote:
> On Fri, Jun 25, 2021 at 04:55:23PM -0700, Dipen Patel wrote:
>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>> +{
>> +	const struct hte_chip *chip = gdev->chip;
>> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
>> +
>> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
>> +	if (!gdev->dbg_root)
>> +		return;
> No need to check for this, if it fails, your other debugfs calls
> will handle it just fine.
>
>
>> +
>> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
>> +				&gdev->ts_req);
>> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
>> +			   &gdev->nlines);
>> +}
>> +
>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>> +{
>> +	if (!ei->gdev->dbg_root || !name)
>> +		return;
>> +
>> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
>> +	if (!ei->ts_dbg_root)
>> +		return;
> Again, no need to check.
>
>> +
>> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
>> +			      &ei->buf->datum_len);
>> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
>> +			      &ei->buf->watermark);
>> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
>> +				&ei->dropped_ts);
>> +}
>> +
>> +static inline void hte_dbgfs_deinit(struct dentry *root)
>> +{
>> +	if (!root)
>> +		return;
> No need to check this.
>
>> +
>> +	debugfs_remove_recursive(root);
> Do not wrap a single call with another call :)
>
>
> thanks,
>
> greg k-h

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-04 20:15   ` Jonathan Cameron
  2021-07-04 20:45     ` Jonathan Cameron
@ 2021-07-28  4:38     ` Dipen Patel
  2021-08-01 16:13       ` Jonathan Cameron
  1 sibling, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-28  4:38 UTC (permalink / raw)
  To: Jonathan Cameron
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt


On 7/4/21 1:15 PM, Jonathan Cameron wrote:
> On Fri, 25 Jun 2021 16:55:23 -0700
> Dipen Patel <dipenp@nvidia.com> wrote:
>
>> Some devices can timestamp system lines/signals/Buses in real-time
>> using the hardware counter or other hardware means which can give
>> finer granularity and help avoid jitter introduced by software means
>> of timestamping. To utilize such functionality there has to be
>> framework where such devices can register themselves as producers or
>> providers so that the consumers or clients devices can request specific
>> line from the providers. This patch introduces such subsystem as
>> hardware timestamping engine (HTE).
>>
>> It provides below APIs for the provider:
>> - hte_register_chip() -- To register the HTE chip.
>> - hte_unregister_chip() -- To unregister the HTE chip.
>> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
>>
>> It provides below APIs for the consumer:
>> - of_hte_request_ts() -- To request timestamp functionality.
>> - devm_of_hte_request_ts() -- Managed version of the above.
>> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
>> using HTE provider dt node.
>> - devm_hte_release_ts() -- The managed version to release timestamp
>> functionality and associated resources.
>> - hte_retrieve_ts_ns() -- To retrieve timestamps.
>> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
>> - hte_enable_ts() -- To disable timestamp functionality.
>> - hte_disable_ts() -- To enable timestamp functionality.
>> - hte_available_ts() -- To query available timestamp data.
>> - hte_release_ts() -- To release timestamp functionality and its
>> associated resources.
>> - hte_get_clk_src_info() -- To query clock source information from
>> the provider
>>
>> It provides centralized software buffer management per requested id to
>> store the timestamp data for the consumers as below:
>> - hte_set_buf_len() -- To set the buffer length.
>> - hte_get_buf_len() -- To get the buffer length.
>> - hte_set_buf_watermark() -- To set the software threshold/watermark.
>> - hte_get_buf_watermark() -- To get the software threshold/watermark.
>>
>> The detail about parameters and API usage are described in each
>> functions definitions in drivers/hte/hte.c file.
>>
>> The patch adds compilation support in Makefile and menu options in
>> Kconfig.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
> around what this is doing + it is an RFC :)
Thanks for the review comments. My responses inline.
>
>> ---
>>  drivers/Kconfig      |    2 +
>>  drivers/Makefile     |    1 +
>>  drivers/hte/Kconfig  |   22 +
>>  drivers/hte/Makefile |    1 +
>>  drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
>>  include/linux/hte.h  |  278 +++++++++
>>  6 files changed, 1672 insertions(+)
>>  create mode 100644 drivers/hte/Kconfig
>>  create mode 100644 drivers/hte/Makefile
>>  create mode 100644 drivers/hte/hte.c
>>  create mode 100644 include/linux/hte.h
>>
>> diff --git a/drivers/Kconfig b/drivers/Kconfig
>> index 47980c6b1945..9b078964974b 100644
>> --- a/drivers/Kconfig
>> +++ b/drivers/Kconfig
>> @@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
>>  source "drivers/counter/Kconfig"
>>  
>>  source "drivers/most/Kconfig"
>> +
>> +source "drivers/hte/Kconfig"
>>  endmenu
>> diff --git a/drivers/Makefile b/drivers/Makefile
>> index 5a6d613e868d..0a996a698e4c 100644
>> --- a/drivers/Makefile
>> +++ b/drivers/Makefile
>> @@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
>>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
>>  obj-$(CONFIG_COUNTER)		+= counter/
>>  obj-$(CONFIG_MOST)		+= most/
>> +obj-$(CONFIG_HTE)		+= hte/
>> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
>> new file mode 100644
>> index 000000000000..394e112f7dfb
>> --- /dev/null
>> +++ b/drivers/hte/Kconfig
>> @@ -0,0 +1,22 @@
>> +# SPDX-License-Identifier: GPL-2.0-only
>> +menuconfig HTE
>> +        bool "Hardware Timestamping Engine (HTE) Support"
>> +        help
>> +          Hardware Timestamping Engine (HTE) Support.
> Tidy this up, but think that's already been commented on.
Will do, not sure why my editor did not show this issue.
>
>> +
>> +          Some devices provide hardware timestamping engine which can timestamp
>> +	  certain device lines/signals in realtime. This way to provide
>> +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
>> +	  comes with benefit for the applications like autonomous machines
>> +	  needing accurate timestamping event with less jitter.
>> +
>> +	  This framework provides a generic interface to such HTE devices
>> +          within the Linux kernel. It provides an API to register and
>> +	  unregister a HTE provider chip, configurable sw buffer to
>> +	  store the timestamps, push the timestamp from the HTE providers and
>> +	  retrieve timestamps for the consumers. It also provides means for the
>> +	  consumers to request signals it wishes to hardware timestamp and
>> +	  release them if not required.
>> +
>> +          If unsure, say no.
>> +
>> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
>> new file mode 100644
>> index 000000000000..9899dbe516f7
>> --- /dev/null
>> +++ b/drivers/hte/Makefile
>> @@ -0,0 +1 @@
>> +obj-$(CONFIG_HTE)		+= hte.o
>> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
>> new file mode 100644
>> index 000000000000..c53260d1e250
>> --- /dev/null
>> +++ b/drivers/hte/hte.c
>> @@ -0,0 +1,1368 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2021 NVIDIA Corporation
>> + *
>> + * Author: Dipen Patel <dipenp@nvidia.com>
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/module.h>
>> +#include <linux/err.h>
>> +#include <linux/slab.h>
>> +#include <linux/of.h>
>> +#include <linux/of_device.h>
>> +#include <linux/kfifo.h>
>> +#include <linux/mutex.h>
>> +#include <linux/sched.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/hte.h>
>> +#include <linux/delay.h>
>> +#include <linux/debugfs.h>
>> +
>> +/* Global list of the HTE devices */
>> +static DEFINE_SPINLOCK(hte_lock);
>> +static LIST_HEAD(hte_devices);
>> +
>> +enum {
>> +	HTE_TS_REGISTERED,
>> +	HTE_TS_DISABLE,
>> +};
>> +
>> +/* Default FIFO depth */
>> +#define HTE_EV_FIFO_EL		32
>> +
>> +#define HTE_TS_NAME_LEN		10
>> +
>> +struct hte_ts_buf;
>> +
>> +/**
>> + * struct hte_ts_buf_acc_func - Software buffer management functions.
>> + * @store: Store timestamp from atomic context as providers most likely
>> + * be pushing timestamps from their interrupt handlers.
>> + * @read: Read timestamps from the buffer.
>> + * @el_available: Available timestamps to retrieve. The client can use this to
>> + * query available elements so that it can pre-allocate internal buffer to send
>> + * to during hte_retrieve_ts_ns API.
>> + * @set_length: Set length/depth of the buffer.
>> + * @get_length: Get length/depth of the buffer.
>> + * @set_watermark: Set software threshold of the buffer.
>> + * @get_watermark: Get software threshold of the buffer.
>> + * @release: Release/free buffer.
>> + * @reset: Reset the buffer.
>> + */
>> +struct hte_ts_buf_acc_func {
>> +	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
>> +	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
>> +		    size_t *copied);
>> +	size_t (*el_available)(struct hte_ts_buf *buf);
>> +	int (*set_length)(struct hte_ts_buf *buf,
>> +			  size_t length, size_t bpd);
>> +	size_t (*get_length)(struct hte_ts_buf *buf);
>> +	int (*set_watermark)(struct hte_ts_buf *buf,
>> +			     size_t val);
>> +	size_t (*get_watermark)(struct hte_ts_buf *buf);
>> +	void (*release)(struct hte_ts_buf *buf);
>> +	void (*reset)(struct hte_ts_buf *buf);
>> +};
>> +
>> +/**
>> + * struct hte_ts_buf - Software buffer per requested id or entity to store
>> + * timestamps.
>> + *
>> + * @datum_len: Buffer depth or number of elements.
>> + * @bytes_per_datum: Element size in bytes.
>> + * @watermark: Software threshold at which client will be notified.
>> + * @valid: Validity of the buffer.
>> + * @pollq: Waitqueue for the blocking clients.
>> + * @access: Various buffer management functions.
>> + */
>> +struct hte_ts_buf {
>> +	size_t datum_len;
>> +	size_t bytes_per_datum;
>> +	size_t watermark;
>> +	bool valid;
>> +	wait_queue_head_t pollq;
>> +	const struct hte_ts_buf_acc_func *access;
>> +};
>> +
>> +/**
>> + * struct hte_ts_info - Information related to requested timestamp.
>> + *
>> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
>> + * See xlate callback API.
>> + * @flags: Flags holding state informations.
>> + * @seq: Timestamp sequence counter.
>> + * @dropped_ts: Dropped timestamps.
>> + * @cb: Callback to notify clients.
>> + * @mlock: Lock during timestamp request/release APIs.
>> + * @ts_dbg_root: Root for the debug fs.
>> + * @gdev: HTE abstract device that this timestamp belongs to.
>> + * @buf: Per requested timestamp software buffer.
>> + * @desc: Timestamp descriptor understood between clients and HTE subsystem.
>> + */
>> +struct hte_ts_info {
>> +	u32 xlated_id;
>> +	unsigned long flags;
>> +	u64 seq;
>> +	atomic_t dropped_ts;
>> +	void (*cb)(enum hte_notify n);
>> +	struct mutex mlock;
>> +	struct dentry *ts_dbg_root;
>> +	struct hte_device *gdev;
>> +	struct hte_ts_buf *buf;
>> +	struct hte_ts_desc *desc;
>> +};
>> +
>> +/**
>> + * struct hte_device - HTE abstract device
>> + * @nlines: Number of entities this device supports.
>> + * @ts_req: Total number of entities requested.
>> + * @ei: Timestamp information.
>> + * @sdev: Device used at various debug prints.
>> + * @dbg_root: Root directory for debug fs.
>> + * @list: List node for internal use.
> Be more specific of what sort of internal use.
Sure..
>
>> + * @chip: HTE chip providing this HTE device.
>> + * @owner: helps prevent removal of modules when in use.
>> + */
>> +struct hte_device {
>> +	u32 nlines;
>> +	atomic_t ts_req;
>> +	struct hte_ts_info *ei;
>> +	struct device *sdev;
>> +	struct dentry *dbg_root;
>> +	struct list_head list;
>> +	struct hte_chip *chip;
>> +	struct module *owner;
>> +};
>> +
>> +/* Buffer management functions */
>> +
>> +/**
>> + * struct hte_kfifo - Software buffer wrapper.
>> + * @buffer: Abstract buffer device.
>> + * @gkf: Actual software buffer type, this case its FIFO.
>> + */
>> +struct hte_kfifo {
>> +	struct hte_ts_buf buffer;
>> +	struct kfifo gkf;
>> +};
>> +
>> +#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
>> +
>> +static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
>> +					size_t n)
>> +{
>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>> +
>> +	if (unlikely(!r->valid))
>> +		return 0;
>> +
>> +	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
>> +}
>> +
>> +static inline int hte_ts_buf_read(struct hte_ts_buf *r,
>> +				  unsigned char *buf, size_t n,
>> +				  size_t *copied)
>> +{
>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>> +
>> +	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
>> +		return -EINVAL;
>> +
>> +	*copied = kfifo_out(&kf->gkf, buf, n);
>> +
>> +	return 0;
>> +}
>> +
>> +static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
>> +{
>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>> +
>> +	if (!r->valid)
>> +		return 0;
>> +
>> +	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
>> +}
>> +
>> +static int hte_ts_buf_set_length(struct hte_ts_buf *r,
>> +				 size_t length, size_t bpd)
>> +{
>> +	int ret = 0;
>> +	struct hte_kfifo *buf;
>> +
>> +	if ((length == 0) || (bpd == 0) || !r)
>> +		return -EINVAL;
>> +
>> +	buf = buf_to_kfifo(r);
>> +
>> +	if (r->datum_len != length) {
>> +		if (r->valid)
>> +			kfifo_free(&buf->gkf);
>> +		r->valid = false;
>> +		r->datum_len = length;
>> +		r->bytes_per_datum = bpd;
>> +		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
>> +		if (!ret)
>> +			r->valid = true;
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
>> +{
>> +	if ((!r->valid) || !r->datum_len)
>> +		return 0;
>> +
>> +	return r->datum_len;
>> +}
>> +
>> +static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
>> +{
>> +	if ((!r->valid) || (val > r->datum_len))
>> +		return -EINVAL;
>> +
>> +	r->watermark = val;
>> +
>> +	return 0;
>> +}
>> +
>> +static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
>> +{
>> +	if (!r->valid)
>> +		return 0;
>> +
>> +	return r->watermark;
>> +}
>> +
>> +static inline void hte_ts_buf_release(struct hte_ts_buf *r)
>> +{
>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>> +
>> +	r->valid = false;
>> +	kfifo_free(&kf->gkf);
>> +	kfree(kf);
>> +}
>> +
>> +static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
>> +{
>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>> +
>> +	if (!r->valid)
>> +		return;
>> +
>> +	kfifo_reset(&kf->gkf);
>> +}
>> +
>> +static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
>> +	.store = &hte_ts_store_to_buf,
>> +	.read = &hte_ts_buf_read,
>> +	.el_available = &hte_ts_buf_el_available,
>> +	.set_length = &hte_ts_buf_set_length,
>> +	.get_length = &hte_ts_buf_get_length,
>> +	.set_watermark = &hte_ts_buf_set_watermark,
>> +	.get_watermark = &hte_ts_buf_get_watermark,
>> +	.release = &hte_ts_buf_release,
>> +	.reset = &hte_ts_buf_reset,
>> +};
>> +
>> +static struct hte_ts_buf *hte_ts_buf_allocate(void)
>> +{
>> +	struct hte_kfifo *kf;
>> +
>> +	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
>> +	if (!kf)
>> +		return ERR_PTR(-ENOMEM);
>> +
>> +	init_waitqueue_head(&kf->buffer.pollq);
>> +	kf->buffer.watermark = 1;
>> +	kf->buffer.datum_len = 0;
>> +	kf->buffer.valid = false;
>> +	kf->buffer.access = &kfifo_access_funcs;
>> +
>> +	return &kf->buffer;
>> +}
>> +/* End of buffer management */
>> +
>> +/* Debugfs management */
>> +
>> +#ifdef CONFIG_DEBUG_FS
>> +
>> +static struct dentry *hte_root;
>> +
>> +static void __init hte_subsys_dbgfs_init(void)
>> +{
>> +	/* creates /sys/kernel/debug/hte/ */
>> +	hte_root = debugfs_create_dir("hte", NULL);
>> +}
>> +subsys_initcall(hte_subsys_dbgfs_init);
>> +
>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>> +{
>> +	const struct hte_chip *chip = gdev->chip;
>> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
>> +
>> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
>> +	if (!gdev->dbg_root)
>> +		return;
>> +
>> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
>> +				&gdev->ts_req);
>> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
>> +			   &gdev->nlines);
>> +}
>> +
>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>> +{
>> +	if (!ei->gdev->dbg_root || !name)
>> +		return;
>> +
>> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
>> +	if (!ei->ts_dbg_root)
>> +		return;
>> +
>> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
>> +			      &ei->buf->datum_len);
>> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
>> +			      &ei->buf->watermark);
>> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
>> +				&ei->dropped_ts);
>> +}
>> +
>> +static inline void hte_dbgfs_deinit(struct dentry *root)
>> +{
>> +	if (!root)
>> +		return;
>> +
>> +	debugfs_remove_recursive(root);
>> +}
>> +
>> +#else
>> +
>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>> +{
>> +}
>> +
>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>> +{
>> +}
>> +
>> +static inline void hte_dbgfs_deinit(struct dentry *root)
>> +{
>> +}
>> +
>> +#endif
>> +/* end of debugfs management*/
>> +
>> +/* Driver APIs */
>> +
>> +/**
>> + * hte_release_ts() - Consumer calls this API to release the entity, where
>> + * entity could be anything providers support, like lines, signals, buses,
>> + * etc...
>> + *
>> + * The correct sequence to call this API is as below:
>> + * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
>> + * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
>> + * still care about the data.
>> + * 3) Call this API.
>> + * Above sequence makes sure that entity gets released race free.
>> + *
>> + * @desc: timestamp descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: hte_dbgfs_deinit() function call may use sleeping locks,
>> + *	    not suitable from atomic context in that case.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_release_ts(struct hte_ts_desc *desc)
>> +{
>> +	u32 id;
>> +	int ret = 0;
>> +	struct hte_device *gdev;
>> +	struct hte_ts_info *ei;
>> +	struct hte_ts_buf *buf;
>> +
>> +	if (!desc)
>> +		return -EINVAL;
>> +
>> +	ei = (struct hte_ts_info *)desc->data_subsys;
> As data_subsys is void * you don't need to explicitly cast it to another pointer type.
Sure..
>
>> +
>> +	if (!ei || !ei->gdev || !ei->buf)
>> +		return -EINVAL;
>> +
>> +	gdev = ei->gdev;
>> +	buf = ei->buf;
>> +	id = desc->con_id;
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_info(gdev->sdev, "id:%d is not registered", id);
>> +		return -EUSERS;
>> +	}
>> +
>> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
>> +	if (ret) {
>> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
>> +		goto out;
>> +	}
>> +
>> +	atomic_dec(&gdev->ts_req);
>> +	atomic_set(&ei->dropped_ts, 0);
>> +
>> +	kfree(desc->name);
>> +	kfree(desc);
>> +	ei->desc = NULL;
>> +	ei->seq = 0;
>> +	buf->access->release(buf);
>> +
>> +	hte_dbgfs_deinit(ei->ts_dbg_root);
>> +	module_put(gdev->owner);
>> +
>> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
>> +
>> +out:
>> +	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_release_ts);
>> +
>> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
>> +{
>> +	u32 ts_id;
>> +	struct hte_device *gdev;
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +
>> +	if (!desc)
>> +		return -EINVAL;
>> +
>> +	ei = (struct hte_ts_info *)desc->data_subsys;
> As above, no need to cast - though it rather implies the type of data_subsys
> should not be void *.

desc is public facing structure, I wanted to make subsystem related

information opaque that is why I had it void *.

>
>> +
>> +	if (!ei || !ei->gdev)
>> +		return -EINVAL;
>> +
>> +	gdev = ei->gdev;
>> +	ts_id = desc->con_id;
>> +
>> +	mutex_lock(&ei->mlock);
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
>> +		ret = -EUSERS;
>> +		goto out;
>> +	}
>> +
>> +	if (en) {
>> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +			ret = 0;
>> +			goto out;
>> +		}
>> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
>> +		if (ret) {
>> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
>> +				 ts_id);
>> +			goto out;
>> +		}
>> +
>> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
>> +		ret = 0;
> ret is already 0 so no point in setting it again.
Yes, will clean up.
>
>> +	} else {
>> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +			ret = 0;
>> +			goto out;
>> +		}
>> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
>> +		if (ret) {
>> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
>> +				 ts_id);
>> +			goto out;
>> +		}
>> +
>> +		set_bit(HTE_TS_DISABLE, &ei->flags);
>> +		ret = 0;
>> +	}
>> +
>> +out:
>> +	mutex_unlock(&ei->mlock);
>> +	return ret;
>> +}
>> +
>> +/**
>> + * hte_disable_ts() - Disable timestamp on given descriptor.
>> + *
>> + * @desc: ts descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: Holds mutex lock, not suitable from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_disable_ts(struct hte_ts_desc *desc)
>> +{
>> +	return hte_ts_dis_en_common(desc, false);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_disable_ts);
>> +
>> +/**
>> + * hte_enable_ts() - Enable timestamp on given descriptor.
>> + *
>> + * @desc: ts descriptor, this is the same as returned by the request API.
>> + *
>> + * Context: Holds mutex lock, not suitable from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_enable_ts(struct hte_ts_desc *desc)
>> +{
>> +	return hte_ts_dis_en_common(desc, true);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_enable_ts);
>> +
>> +static int hte_simple_xlate(struct hte_chip *gc,
>> +			    const struct of_phandle_args *args,
>> +			    struct hte_ts_desc *desc,
>> +			    u32 *id)
>> +{
>> +	if (!id || !desc || !gc)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * For the providers which do not have any internal mappings between
>> +	 * logically exposed ids and actual ids, will set both
>> +	 * the same.
>> +	 *
>> +	 * In case there is a internal mapping needed, providers will need to
>> +	 * provide its own xlate function where con_id will be sent as
>> +	 * args[0] and it will return xlated id. Later xlated id will be
>> +	 * used for any future exchanges between provider and subsystems.
>> +	 */
>> +
>> +	if (args) {
>> +		if (gc->of_hte_n_cells < 1)
>> +			return -EINVAL;
>> +
>> +		if (args->args_count != gc->of_hte_n_cells)
>> +			return -EINVAL;
>> +
>> +		*id = args->args[0];
>> +		desc->con_id = *id;
>> +	} else {
>> +		*id = desc->con_id;
>> +	}
>> +
>> +	if (desc->con_id > gc->nlines)
>> +		return -EINVAL;
>> +
>> +	desc->data_subsys = NULL;
>> +
>> +	return 0;
>> +}
>> +
>> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
>> +{
>> +	struct hte_device *gdev;
>> +
>> +	spin_lock(&hte_lock);
>> +
>> +	list_for_each_entry(gdev, &hte_devices, list)
>> +		if (gdev->chip && gdev->chip->dev &&
>> +		    gdev->chip->dev->of_node == np) {
>> +			spin_unlock(&hte_lock);
>> +			return gdev;
>> +		}
>> +
>> +	spin_unlock(&hte_lock);
>> +
>> +	return ERR_PTR(-ENODEV);
>> +}
>> +
>> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
>> +			 u32 xlated_id, void (*cb)(enum hte_notify n))
>> +{
>> +	struct hte_ts_info *ei;
>> +	struct hte_ts_buf *buf;
>> +	int ret;
>> +	u32 con_id = desc->con_id;
>> +
>> +	if (!try_module_get(gdev->owner))
>> +		return -ENODEV;
>> +
>> +	ei = &gdev->ei[xlated_id];
>> +	ei->xlated_id = xlated_id;
>> +
>> +	/*
>> +	 * There a chance that multiple consumers requesting same entity,
>> +	 * lock here.
>> +	 */
>> +	mutex_lock(&ei->mlock);
>> +
>> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
>> +			xlated_id);
>> +		ret = -EUSERS;
>> +		goto unlock;
>> +	}
>> +
>> +	buf = hte_ts_buf_allocate();
>> +	if (IS_ERR(buf)) {
>> +		dev_err(gdev->chip->dev, "Buffer allocation failed");
>> +		ret = PTR_ERR(buf);
>> +		goto unlock;
>> +	}
>> +
>> +	/* Set default here, let consumer decide how much to set later */
>> +	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
>> +				      sizeof(struct hte_ts_data));
>> +
> It's good to keep to consistent style of no line break between a statement
> and it's error check.
Sure...
>
>> +	if (ret) {
>> +		dev_err(gdev->chip->dev, "Fifo set length failed");
>> +		goto buf_rel;
>> +	}
>> +
>> +	buf->access->reset(buf);
>> +	buf->valid = true;
>> +
>> +	ei->buf = buf;
>> +	ei->cb = cb;
>> +
>> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev, "ts request failed\n");
>> +		goto buf_rel;
>> +	}
>> +
>> +	desc->data_subsys = ei;
>> +	ei->desc = desc;
>> +
>> +	atomic_inc(&gdev->ts_req);
>> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
>> +	mutex_unlock(&ei->mlock);
>> +
>> +	if (!desc->name) {
>> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
>> +		if (desc->name)
>> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
>> +				  con_id);
>> +	}
>> +
>> +	hte_ts_dbgfs_init(desc->name, ei);
>> +
>> +	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
>> +		__func__, con_id, xlated_id);
>> +
>> +	return 0;
>> +
>> +buf_rel:
>> +	buf->access->release(buf);
>> +unlock:
>> +	module_put(gdev->owner);
>> +	mutex_unlock(&ei->mlock);
>> +
>> +	return ret;
>> +}
>> +
>> +static struct hte_device *of_hte_dev_get(struct device *dev,
>> +					 struct device_node *np,
>> +					 const char *label,
>> +					 struct of_phandle_args *args)
>> +{
>> +	struct hte_device *gdev = NULL;
>> +	int index = 0;
>> +	int err;
>> +
>> +	if (label) {
>> +		index = of_property_match_string(np, "hte-names", label);
>> +		if (index < 0)
>> +			return ERR_PTR(index);
>> +	}
>> +
>> +	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
>> +					 args);
>> +	if (err) {
>> +		pr_err("%s(): can't parse \"htes\" property\n", __func__);
>> +		return ERR_PTR(err);
>> +	}
>> +
>> +	gdev = of_node_to_htedevice(args->np);
>> +	if (IS_ERR(gdev)) {
>> +		pr_err("%s(): HTE chip not found\n", __func__);
>> +		of_node_put(args->np);
>> +		return gdev;
>> +	}
>> +
>> +	return gdev;
>> +}
>> +
>> +static struct hte_ts_desc *__hte_req_ts(struct device *dev,
>> +					struct device_node *np,
>> +					const char *label,
>> +					void (*cb)(enum hte_notify n))
>> +{
>> +	struct hte_device *gdev = NULL;
>> +	struct hte_ts_desc *desc;
>> +	struct of_phandle_args args;
>> +	int ret;
>> +	u32 xlated_id;
>> +
>> +	gdev = of_hte_dev_get(dev, np, label, &args);
>> +	if (IS_ERR(gdev))
>> +		return ERR_CAST(gdev);
>> +
>> +	if (!gdev->chip) {
>> +		pr_debug("requested id does not have provider\n");
>> +		return ERR_PTR(-ENODEV);
>> +	}
>> +
>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
>> +	if (!desc)
>> +		return ERR_PTR(-ENOMEM);
>> +
>> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
>> +	if (ret < 0)
>> +		goto put;
>> +
>> +	desc->name = NULL;
>> +	if (label)
>> +		desc->name = kstrdup(label, GFP_KERNEL);
>> +
>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
>> +	if (ret < 0)
>> +		goto put;
>> +
>> +	return desc;
>> +
>> +put:
>> +	of_node_put(args.np);
>> +	kfree(desc);
>> +
>> +	return ERR_PTR(ret);
>> +}
>> +
>> +/**
>> + * of_hte_request_ts() - Consumer calls this API to request the HTE facility
>> + * on the specified entity, where entity is provider specific for example,
>> + * GPIO lines, signals, buses etc...
>> + *
>> + * @dev: Consumer device.
>> + * @label: Optional label.
>> + * @cb: Optional notify callback to consumer when data is pushed by the
>> + * provider.
>> + *
>> + * Context: Holds mutex lock, not suitable from atomic context.
>> + * Returns: Timestamp descriptor on success or error ptr on failure.
>> + */
>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev,
>> +				      const char *label,
>> +				      void (*cb)(enum hte_notify n))
>> +{
>> +
>> +	if (dev && dev->of_node)
>> +		return __hte_req_ts(dev, dev->of_node, label, cb);
>> +	else
>> +		return ERR_PTR(-EOPNOTSUPP);
>> +}
>> +EXPORT_SYMBOL_GPL(of_hte_request_ts);
>> +
>> +static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)
> I'm not seeing what is devm about this.

It was part of devm release. However I am planing to remove devm release

function, in turns this will be deleted as well.

>
>> +{
>> +	struct hte_ts_desc **p = res;
>> +
>> +	if (WARN_ON(!p || !*p))
>> +		return 0;
>> +
>> +	return *p == data;
>> +}
>> +
>> +static void __devm_hte_release_ts(struct device *dev, void *res)
>> +{
>> +	hte_release_ts(*(struct hte_ts_desc **)res);
>> +}
>> +
>> +/**
>> + * devm_hte_release_ts() - Resource managed hte_release_ts().
> I'd not introduce this until you have a user.  It very rarely actually makes
> sense to call a devm release manually. Not having one makes people think harder
> about it.
Will remove.
>
>> + * @dev: HTE consumer/client device.
>> + * @desc: HTE ts descriptor.
>> + *
>> + * Release timestamp functionality and its resources previously allocated using
>> + * of_hte_request_ts(). Calling this function is usually not needed because
>> + * devm-allocated resources are automatically released on driver detach.
>> + *
>> + * Context: Same as hte_release_ts() function.
>> + * Returns: 0 on success otherwise negative error code.
>> + */
>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
>> +{
>> +	return devres_release(dev, __devm_hte_release_ts,
>> +			      devm_hte_ts_match_desc, desc);
>> +}
>> +EXPORT_SYMBOL_GPL(devm_hte_release_ts);
>> +
>> +/**
>> + * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().
> If it's kernel-doc it needs to give no warnings when you point the kernel-doc
> scripts at it.  They insist on full parameter documentation.
>
>> + */
>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
>> +					   const char *label,
>> +					   void (*cb)(enum hte_notify n))
>> +{
>> +
>> +	struct hte_ts_desc **ptr, *desc;
>> +
>> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);
> Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
> and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
> last kernel cycle.
I based my patches from linux-next/master. Not sure if that has

dev_add_action_or_reset

>
>
>> +	if (!ptr)
>> +		return ERR_PTR(-ENOMEM);
>> +
>> +	desc = of_hte_request_ts(dev, label, cb);
>> +	if (!IS_ERR(desc)) {
>> +		*ptr = desc;
>> +		devres_add(dev, ptr);
>> +	} else {
>> +		devres_free(ptr);
>> +	}
>> +	return desc;
>> +}
>> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
>> +
>> +static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
>> +					  size_t val)
> Not a good name or indeed combination of different things.
> hte_desc_to_info() and some separate check on val would be better.
Good suggestion, will follow new name.
>
>> +{
>> +	struct hte_ts_info *ei;
>> +
>> +	if (!desc || !desc->data_subsys || !val) {
>> +		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
>> +		return NULL;
>> +	}
>> +
>> +	ei = desc->data_subsys;
>> +	if (!ei || !ei->buf) {
>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>> +		return NULL;
>> +	}
>> +
>> +	return ei;
>> +}
>> +
>> +static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
>> +{
>> +	size_t el_avail;
>> +
>> +	el_avail = buffer->access->el_available(buffer);
>> +
>> +	return (el_avail >= to_read) ? false : true;
> return el_avail < to_read;
Sure...
>
>> +}
>> +
>> +static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
>> +			       struct hte_ts_data *el, size_t n, bool block)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +	size_t to_read, copied;
>> +
>> +	ei = hte_para_check(desc, n);
>> +	if (!ei)
>> +		return -EINVAL;
>> +
>> +	buffer = ei->buf;
>> +
>> +	to_read = min_t(size_t, n, buffer->watermark);
> Needs a comment as not obvious why you'd read the min of that requested or
> the watermark if there might be more available.
I will add detailed comment in next version of the RFC.
>
>> +
>> +	do {
>> +		if (hte_ts_buf_wait(buffer, to_read)) {
>> +			if (!block) {
>> +				/* Possibly early here to retrieve, try again */
>> +				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
>> +					__func__, ret);
>> +				return -EAGAIN;
>> +			}
>> +			ret = wait_event_interruptible(buffer->pollq,
>> +					!hte_ts_buf_wait(buffer, to_read));
>> +			if (ret)
>> +				return ret;
>> +		}
>> +		ret = buffer->access->read(buffer, (void *)el,
> If you have to cast to a void * that usually means something is wrong in your definitions.
> Why is it needed here?  Looks like read has an inappropriate definition.
Will change to void* in read definitions to remove this cast.
>
>> +					   n * buffer->bytes_per_datum,
>> +					   &copied);
>> +		if (ret < 0)
>> +			return ret;
>> +
>> +		if (copied > 0)
>> +			return 0;
>> +		else if (copied == 0 && !block)
>> +			return -EAGAIN;
>> +	} while (copied == 0);
>> +
>> +	return 0;
>> +}
>> +
>> +/**
>> + * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
>> + * nano seconds i.e. el->tsc will be in ns.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + * @el: buffer to store the timestamp details.
>> + * @n: Number of struct hte_timestamp_el elements.
>> + *
>> + * Context: Can be called from the atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
>> +		       struct hte_ts_data *el, size_t n)
>> +{
>> +	return _hte_retrieve_ts_ns(desc, el, n, false);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
>> +
>> +/**
>> + * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
>> + * @desc: ts descriptor, same as returned from request API.
>> + * @el: buffer to store the timestamp data.
>> + * @n: Number of struct hte_ts_data data.
>> + *
>> + * Context: Can not be called from the atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
>> +			    struct hte_ts_data *el, size_t n)
>> +{
>> +	return _hte_retrieve_ts_ns(desc, el, n, true);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
>> +
>> +/**
>> + * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
>> + * depth.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + * @len: New length/depth.
>> + *
>> + * The correct sequence to set buffer length is as below:
>> + * 1) Disable timestamp by calling hte_disable_ts API.
>> + * 2) Optionally retrieve all the timestamps by calling non blocking
>> + *    hte_retrieve_ts_ns() API. This step only needed if you still care about
>> + *    the data.
>> + * 3) Call this API.
>> + * 4) Enable timestamp by calling hte_enable_ts API.
>> + *
>> + * This API destroys previously allocated buffer and creates new one, because
>> + * of that, it is mandatory to follow above sequence to make sure there is no
>> + * race between various other APIs in the subsystem.
> Good docs.  This is why I mentioned in review of docs patch that it is better
> to just have that refer to the kernel-doc in these files.  Keep all this good
> information in one place.
Agree..
>
>> + *
>> + * By default during the request API call, HTE subsystem allocates software
>> + * buffer with predefined length, this API gives flexibility to adjust the
>> + * length according to consumer's need.
>> + *
>> + * Context: Can not be called from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +
>> +	ei = hte_para_check(desc, len);
>> +	if (!ei)
>> +		return -EINVAL;
>> +
>> +	buffer = ei->buf;
>> +	ret = buffer->access->set_length(buffer, len,
>> +					 sizeof(struct hte_ts_data));
>> +	if (ret)
>> +		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
> Not point in printing things line __func__ manually in dev_err() etc.
> Dynamic debug includes that and gives far more information + control of this.
yes, will remove.
>
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_set_buf_len);
>> +
>> +/**
>> + * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
>> + * depth or length.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + *
>> + * Context: Any context.
>> + * Returns: Positive length on success or 0 on failure.
>> + */
>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +
>> +	ei = hte_para_check(desc, 1);
>> +	if (!ei)
>> +		return 0;
>> +
>> +	buffer = ei->buf;
>> +
>> +	return buffer->access->get_length(buffer);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_get_buf_len);
>> +
>> +/**
>> + * hte_available_ts() - Returns total available timestamps.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + *
>> + * The API helps consumers to pre-allocate its internal buffer required
>> + * during hte_retrieve_ts_ns call.
>> + *
>> + * Context: Any context.
>> + * Returns: Positive value if elements are available else 0. The value is
>> + * number of total available struct hte_timestamp_el elements available not
>> + * the size in bytes.
>> + */
>> +size_t hte_available_ts(const struct hte_ts_desc *desc)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +
>> +	ei = hte_para_check(desc, 1);
>> +	if (!ei)
>> +		return 0;
>> +
>> +	buffer = ei->buf;
>> +
>> +	return buffer->access->el_available(buffer);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_available_ts);
>> +
>> +/**
>> + * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
>> + * buffer watermark. The correct sequence to call this API is as below:
>> + * 1) Disable timestamp by calling hte_disable_ts API.
>> + * 2) Call this API.
>> + * 3) Enable timestamp by calling hte_enable_ts API.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + * @val: New watermark.
>> + *
>> + * By default during the request API call, HTE subsystem sets watermark as 1,
>> + * this API gives flexibility to adjust the watermark according to consumer's
>> + * need. The consumers will get notification through callback registered during
>> + * request API either when timestamp is dropped or watermark is reached or will
>> + * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
>> + * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
>> + *
>> + * Context: Any context.
> You have no way of knowing that as will depend on the driver - I'd definitely
> suggest not from atomic context, but then that would be crazy so you are better
>  off not documenting any specific requirement at all.

set watermark does not talk to driver, at least for now as it is totally software

managed buffer from the hte core. I will remove context.

>
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +	int ret;
>> +
>> +	ei = hte_para_check(desc, val);
>> +	if (!ei)
>> +		return -EINVAL;
>> +
>> +	buffer = ei->buf;
>> +	ret = buffer->access->set_watermark(buffer, val);
>> +	if (ret)
>> +		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
>> +
>> +/**
>> + * hte_get_buf_watermark() - Consumer calls this API to get software
>> + * buffer watermark.
>> + * @desc: ts descriptor, same as returned from request API.
>> + *
>> + * Context: Any context.
>> + * Returns: Positive current watermark on success or 0 on failure.
>> + */
>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
>> +{
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +
>> +	ei = hte_para_check(desc, 1);
>> +	if (!ei)
>> +		return 0;
>> +
>> +	buffer = ei->buf;
>> +
>> +	return buffer->access->get_watermark(buffer);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
>> +
>> +/**
>> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
>> + * node directly, where meaning of the entity is provider specific, for example
>> + * lines, signals, GPIOs, buses etc...
>> + *
>> + * @of_node: HTE provider device node.
>> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
>> + * @cb: Optional callback to notify.
>> + *
>> + * Context: Holds mutex lock, can not be called from atomic context.
> What mutex and why?  If it is one you can check is held even better.

___hte_req_ts holds the mutex lock to serialize multiple consumers

requesting same entity.

>
>> + * Returns: ts descriptor on success or error pointers.
>> + */
>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
>> +					  unsigned int id,
>> +					  void (*cb)(enum hte_notify n))
>> +{
>> +	struct hte_device *gdev;
>> +	struct hte_ts_desc *desc;
>> +	int ret;
>> +	u32 xlated_id;
>> +
>> +	gdev = of_node_to_htedevice(of_node);
>> +	if (IS_ERR(gdev))
>> +		return ERR_PTR(-ENOTSUPP);
>> +
>> +	if (!gdev->chip || !gdev->chip->ops)
>> +		return ERR_PTR(-ENOTSUPP);
>> +
>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
>> +	if (!desc) {
>> +		ret = -ENOMEM;
>> +		goto out_put_device;
>> +	}
> Pass a desc pointer into this function rather than allocating the structure
> in here.  That lets the caller embed that structure inside one of it's own
> structures if it wants to, resulting in fewer small allocations which is always good.
>
> It's far from obvious that the caller needs to free desc.

Are you suggesting to shift burden of allocation/deallocation (static or dynamic)

at client/consumer side?

>
>> +
>> +	desc->con_id = id;
>> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev,
>> +			"failed to xlate id: %d\n", id);
>> +		goto out_free_desc;
>> +	}
>> +
>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
>> +	if (ret < 0) {
>> +		dev_err(gdev->chip->dev,
>> +			"failed to request id: %d\n", id);
>> +		goto out_free_desc;
>> +	}
>> +
>> +	return desc;
>> +
>> +out_free_desc:
>> +	kfree(desc);
>> +
>> +out_put_device:
>> +	return ERR_PTR(ret);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
>> +
>> +/**
>> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
>> + * information of the desc.
>> + *
>> + * @desc: ts descriptor, same as returned from request API.
>> + *
>> + * Context: Any context.
>> + * Returns: 0 on success else negative error code on failure.
>> + */
>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>> +			 struct hte_clk_info *ci)
>> +{
>> +	struct hte_chip *chip;
>> +	struct hte_ts_info *ei;
>> +
>> +	if (!desc || !desc->data_subsys || !ci) {
>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>> +		return -EINVAL;
>> +	}
>> +
>> +	ei = desc->data_subsys;
>> +	if (!ei || !ei->gdev || !ei->gdev->chip)
>> +		return -EINVAL;
>> +
>> +	chip = ei->gdev->chip;
>> +	if (!chip->ops->get_clk_src_info)
>> +		return -ENOTSUPP;
>> +
>> +	return chip->ops->get_clk_src_info(chip, ci);
>> +}
>> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
>> +
>> +static inline void hte_add_to_device_list(struct hte_device *gdev)
>> +{
>> +	struct hte_device *prev;
> Needs to take an appropriate lock as you may have concurrent calls.

There is spin_lock held from register API from where this gets

called.

>
>> +
>> +	if (list_empty(&hte_devices)) {
>> +		list_add_tail(&gdev->list, &hte_devices);
> Needs a comment. I've no idea why you might want to only add it if there were
> no other hte_devices already there.
>
>> +		return;
>> +	}
>> +
>> +	prev = list_last_entry(&hte_devices, struct hte_device, list);
> Why woud you do this?

Thanks for pointing out. I definitely missed cleaning this up. Now, I will

remove this function in next RFC version as one line can be added directly

in register API.

>
>> +	list_add_tail(&gdev->list, &hte_devices);
>> +}
>> +
>> +/**
>> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
>> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
>> + * using this API from its ISR or atomic context.
>> + *
>> + * @chip: The HTE chip, used during the registration.
>> + * @xlated_id: entity id understood by both subsystem and provider, usually this
>> + * is obtained from xlate callback during request API.
>> + * @data: timestamp data.
>> + * @n: Size of the data.
>> + *
>> + * Context: Atomic.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
>> +			  struct hte_ts_data *data, size_t n)
>> +{
>> +	unsigned int ret;
>> +	bool notify;
>> +	size_t el_avail;
>> +	struct hte_ts_buf *buffer;
>> +	struct hte_ts_info *ei;
>> +
>> +	if (!chip || !data || !chip->gdev)
>> +		return -EINVAL;
>> +
>> +	if (xlated_id > chip->nlines)
>> +		return -EINVAL;
>> +
>> +	ei = &chip->gdev->ei[xlated_id];
>> +
>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
>> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
>> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	/* timestamp sequence counter, start from 0 */
>> +	data->seq = ei->seq++;
>> +
>> +	buffer = ei->buf;
>> +	el_avail = buffer->access->el_available(buffer);
>> +	ret = buffer->access->store(buffer, data, n);
> If we are doing this from the hte core, why is buffer definition in the scope of the
> drivers rather than the core?  That seems backwards to me.

I do not understand this comment. The buffer definition is in scope of hte core

as it is the only entity that manages it.

>
>> +	if (ret != n) {
>> +		atomic_inc(&ei->dropped_ts);
>> +		if (ei->cb)
>> +			ei->cb(HTE_TS_DROPPED);
>> +		return -ENOMEM;
>> +	}
>> +
>> +	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;
> You push n but only check on el_avail + 1 here.  
> Also, this is the same as
Good catch, will correct that.
>
> 	notify = ((el_avail + 1) >= buffer->watermark;
right, not sure why do I have obsession with ternary operators :)
>
>
>> +
>> +	/*
>> +	 * If there is a callback, its consumer's job to retrieve the timestamp.
>> +	 * For the rest, wake up the process.
>> +	 */
>> +	if (notify && ei->cb) {
>> +		ei->cb(HTE_TS_AVAIL);
>> +		return 0;
> Given you return 0 anyway, might as well not have this line.
True, will remove.
>
>> +	} else if (notify) {
>> +		wake_up_interruptible(&buffer->pollq);
>> +	}
>> +
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
>> +
>> +/**
>> + * hte_register_chip() - Used by provider to register a HTE chip.
>> + * @chip: the HTE chip to add to subsystem.
>> + *
>> + * Context: Can not be called from atomic context.
> Whilst true, I'd think that was common sense as it would be insane
> to register something like this from atomic context.  So I'd say no
> need to comment on it!  Keep those comments for things that
> might be used like that.
Will remove...
>
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_register_chip(struct hte_chip *chip)
>> +{
>> +	struct hte_device *gdev;
>> +	int ret;
>> +	u32 i;
>> +
>> +	if (!chip || !chip->dev || !chip->dev->of_node)
>> +		return -EINVAL;
>> +
>> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
>> +		dev_err(chip->dev, "Driver needs to provide ops\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
>> +	if (!gdev)
>> +		return -ENOMEM;
>> +
>> +	gdev->chip = chip;
>> +	chip->gdev = gdev;
>> +	gdev->nlines = chip->nlines;
>> +	gdev->sdev = chip->dev;
>> +
>> +	/*
>> +	 * Allocate all the supported entities here at once, this will have
>> +	 * following advantages:
>> +	 * When provider pushes timestamp, it can then just send the
>> +	 * xlated_id, subsystem will use it as an index which
>> +	 * gives us the constant time access; this is important as mostly
>> +	 * providers will be pushing the timestamps from their ISR.
>> +	 */
>> +	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
>> +			   GFP_KERNEL);
> I'd be tempted to do this as a 0 length element at the end of gdev
> then do the allocation in one go use struct_size() etc to work out
> how long it is.  Cuts down on allocations + error paths to deal with
> for no obvious disadvantage.
Sure...
>
>> +	if (!gdev->ei) {
>> +		ret = -ENOMEM;
>> +		goto err_free_gdev;
>> +	}
>> +
>> +	for (i = 0; i < chip->nlines; i++) {
>> +		gdev->ei[i].flags = 0;
> zero allocated, so don't bother setting things to 0 where it's a fairly obvious
> base state.  If you set something to 0 to act as some form of documentation then
> that's fine, but I don't think that's true here.
True, will remove.
>
>> +		gdev->ei[i].gdev = gdev;
>> +		gdev->ei[i].seq = 0;
>> +		mutex_init(&gdev->ei[i].mlock);
>> +	}
>> +
>> +	if (chip->dev->driver)
>> +		gdev->owner = chip->dev->driver->owner;
>> +	else
>> +		gdev->owner = THIS_MODULE;
>> +
>> +	if (!chip->xlate) {
>> +		chip->xlate = hte_simple_xlate;
>> +		/* Just a id number to monitor */
>> +		chip->of_hte_n_cells = 1;
>> +	}
>> +
>> +	of_node_get(chip->dev->of_node);
>> +
>> +	INIT_LIST_HEAD(&gdev->list);
>> +
>> +	spin_lock(&hte_lock);
>> +	hte_add_to_device_list(gdev);
>> +	spin_unlock(&hte_lock);
>> +
>> +	hte_chip_dbgfs_init(gdev);
>> +
>> +	dev_dbg(chip->dev, "Added hte chip\n");
>> +	return 0;
>> +
>> +err_free_gdev:
>> +	kfree(gdev);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_register_chip);
>> +
>> +/**
>> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
>> + * @chip: the HTE chip to remove.
>> + *
>> + * Context: Can not be called from atomic context.
>> + * Returns: 0 on success or a negative error code on failure.
>> + */
>> +int hte_unregister_chip(struct hte_chip *chip)
>> +{
>> +	struct hte_device *gdev = chip->gdev;
>> +
>> +	spin_lock(&hte_lock);
>> +	list_del(&gdev->list);
>> +	spin_unlock(&hte_lock);
>> +
>> +	gdev->chip = NULL;
>> +
>> +	of_node_put(chip->dev->of_node);
>> +	hte_dbgfs_deinit(gdev->dbg_root);
>> +	kfree(gdev->ei);
>> +	kfree(gdev);
>> +
>> +	dev_dbg(chip->dev, "Removed hte chip\n");
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(hte_unregister_chip);
>> +
>> +/* Driver APIs ends */
> Don't bother with file layout type comments.  They don't add that much and tend
> to rot horribly over time as people move code around in files.
Sure..
>
>> diff --git a/include/linux/hte.h b/include/linux/hte.h
>> new file mode 100644
>> index 000000000000..e1737579d4c4
>> --- /dev/null
>> +++ b/include/linux/hte.h
>> @@ -0,0 +1,278 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2021 NVIDIA Corporation
>> + *
>> + * Author: Dipen Patel <dipenp@nvidia.com>
>> + */
>> +
>> +#ifndef __LINUX_HTE_H
>> +#define __LINUX_HTE_H
>> +
>> +struct hte_chip;
>> +struct hte_device;
>> +struct of_phandle_args;
>> +
>> +/**
>> + * Used by providers to indicate the direction of the timestamp.
>> + */
>> +#define HTE_EVENT_RISING_EDGE          0x1
>> +#define HTE_EVENT_FALLING_EDGE         0x2
> Use an enum rather than a define for this as it's a value that can take a
> set of distinct values.  Also, provide a name for 'I've no idea' which
> I'm guessing is 0 currently.
I will change to enum.
>
>> +
>> +/**
>> + * struct hte_ts_data - HTE timestamp data.
>> + * The provider uses and fills timestamp related details during push_timestamp
>> + * API call. The consumer uses during retrieve_timestamp API call.
>> + *
>> + * @tsc: Timestamp value.
>> + * @seq: Sequence counter of the timestamps.
>> + * @dir: Direction of the event at the time of timestamp.
>> + */
>> +struct hte_ts_data {
>> +	u64 tsc;
>> +	u64 seq;
>> +	int dir;
>> +};
>> +
>> +/**
>> + * struct hte_clk_info - Clock source info that HTE provider uses.
>> + * The provider uses hardware clock as a source to timestamp real time. This
>> + * structure presents the clock information to consumers. 
>> + *
>> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
>> + * @type: Clock type. CLOCK_* types.
> So this is something we got a it wrong in IIO. It's much better to define
> a subset of clocks that can be potentially used.  There are some that make
> absolutely no sense and consumers really don't want to have to deal with them.
Is there anything I have to change here?
>  
>> + */
>> +struct hte_clk_info {
>> +	u64 hz;
>> +	clockid_t type;
>> +};
>> +
>> +/**
>> + * HTE subsystem notifications for the consumers.
>> + *
>> + * @HTE_TS_AVAIL: Timestamps available notification.
>> + * @HTE_TS_DROPPED: Timestamps dropped notification.
> Something I've missed so far is whether drops are in a kfifo or a ring
> fashion.  I'm guess that's stated somewhere, but it might be useful to have
> it here.
Dropped are from kfifo if kfifo does not have space.
>
>> + */
>> +enum hte_notify {
>> +	HTE_TS_AVAIL = 1,
>> +	HTE_TS_DROPPED,
>> +	HTE_NUM_NOTIFIER,
>> +};
>> +
>> +/**
>> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
>> + * communication token between consumers to subsystem and subsystem to
>> + * providers.
>> + *
>> + * @con_id: This is the same id sent in request APIs.
>> + * @name: Descriptive name of the entity that is being monitored for the
>> + * realtime timestamping.
>> + * @data_subsys: Subsystem's private data relate to requested con_id.
>> + */
>> +struct hte_ts_desc {
>> +	u32 con_id;
>> +	char *name;
>> +	void *data_subsys;
>> +};
>> +
>> +/**
>> + * struct hte_ops - HTE operations set by providers.
>> + *
>> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
>> + * non-zero for failures.
>> + * @get_clk_src_info: Optional hook to get the clock information provider uses
>> + * to timestamp. Returns 0 for success and negative error code for failure. On
>> + * success HTE subsystem fills up provided struct hte_clk_info.
> Why optional?  Consumers will probably need that information.

Sure, will remove optional. But you wrote "probably", that would make it

optional :).

>
>> + *
>> + * xlated_id parameter is used to communicate between HTE subsystem and the
>> + * providers. It is the same id returned during xlate API call and translated
>> + * by the provider. This may be helpful as both subsystem and provider locate
>> + * the requested entity in constant time, where entity could be anything from
>> + * lines, signals, events, buses etc.. that providers support.
>> + */
>> +struct hte_ops {
>> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
>> +	int (*get_clk_src_info)(struct hte_chip *chip,
>> +				struct hte_clk_info *ci);
>> +};
>> +
>> +/**
>> + * struct hte_chip - Abstract HTE chip structure.
>> + * @name: functional name of the HTE IP block.
>> + * @dev: device providing the HTE.
> Unclear naming.  Is this the parent device, or one associated with the HTE itself?
> I'm guessing today you don't have one associated with the HTE, but it is plausible you
> might gain on in future to make it fit nicely in the device model as a function of another
> device.

This is provider's device, could be &pdev->dev or any dev provider deems fit hence the

generic name.

>
>> + * @ops: callbacks for this HTE.
>> + * @nlines: number of lines/signals supported by this chip.
>> + * @xlate: Callback which translates consumer supplied logical ids to
>> + * physical ids, return from 0 for the success and negative for the
>> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
>> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
>> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
>> + * @data: chip specific private data.
>> + */
>> +struct hte_chip {
>> +	const char *name;
>> +	struct device *dev;
>> +	const struct hte_ops *ops;
>> +	u32 nlines;
>> +	int (*xlate)(struct hte_chip *gc,
>> +		     const struct of_phandle_args *args,
>> +		     struct hte_ts_desc *desc, u32 *xlated_id);
>> +	u8 of_hte_n_cells;
>> +
>> +	/* only used internally by the HTE framework */
>> +	struct hte_device *gdev;
>> +	void *data;
>> +};
>> +
>> +#if IS_ENABLED(CONFIG_HTE)
>> +/* HTE APIs for the providers */
>> +int hte_register_chip(struct hte_chip *chip);
>> +int hte_unregister_chip(struct hte_chip *chip);
>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
>> +			  struct hte_ts_data *data, size_t n);
>> +
>> +/* HTE APIs for the consumers */
>> +
>> +int hte_release_ts(struct hte_ts_desc *desc);
>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
>> +				      void (*cb)(enum hte_notify n));
>> +
>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
>> +					   const char *label,
>> +					   void (*cb)(enum hte_notify n));
>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
>> +					  unsigned int id,
>> +					  void (*cb)(enum hte_notify n));
>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
>> +		       size_t n);
>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
>> +			    struct hte_ts_data *el, size_t n);
>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc);
>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
>> +size_t hte_available_ts(const struct hte_ts_desc *desc);
>> +int hte_enable_ts(struct hte_ts_desc *desc);
>> +int hte_disable_ts(struct hte_ts_desc *desc);
>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>> +			 struct hte_clk_info *ci);
>> +
>>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-04 20:45     ` Jonathan Cameron
@ 2021-07-28  5:12       ` Dipen Patel
  2021-08-01 16:48         ` Jonathan Cameron
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-28  5:12 UTC (permalink / raw)
  To: Jonathan Cameron
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt, William Breathitt Gray


On 7/4/21 1:45 PM, Jonathan Cameron wrote:
> On Sun, 4 Jul 2021 21:15:25 +0100
> Jonathan Cameron <jic23@kernel.org> wrote:
>
>> On Fri, 25 Jun 2021 16:55:23 -0700
>> Dipen Patel <dipenp@nvidia.com> wrote:
>>
>>> Some devices can timestamp system lines/signals/Buses in real-time
>>> using the hardware counter or other hardware means which can give
>>> finer granularity and help avoid jitter introduced by software means
>>> of timestamping. To utilize such functionality there has to be
>>> framework where such devices can register themselves as producers or
>>> providers so that the consumers or clients devices can request specific
>>> line from the providers. This patch introduces such subsystem as
>>> hardware timestamping engine (HTE).
>>>
>>> It provides below APIs for the provider:
>>> - hte_register_chip() -- To register the HTE chip.
>>> - hte_unregister_chip() -- To unregister the HTE chip.
>>> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
>>>
>>> It provides below APIs for the consumer:
>>> - of_hte_request_ts() -- To request timestamp functionality.
>>> - devm_of_hte_request_ts() -- Managed version of the above.
>>> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
>>> using HTE provider dt node.
>>> - devm_hte_release_ts() -- The managed version to release timestamp
>>> functionality and associated resources.
>>> - hte_retrieve_ts_ns() -- To retrieve timestamps.
>>> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
>>> - hte_enable_ts() -- To disable timestamp functionality.
>>> - hte_disable_ts() -- To enable timestamp functionality.
>>> - hte_available_ts() -- To query available timestamp data.
>>> - hte_release_ts() -- To release timestamp functionality and its
>>> associated resources.
>>> - hte_get_clk_src_info() -- To query clock source information from
>>> the provider
>>>
>>> It provides centralized software buffer management per requested id to
>>> store the timestamp data for the consumers as below:
>>> - hte_set_buf_len() -- To set the buffer length.
>>> - hte_get_buf_len() -- To get the buffer length.
>>> - hte_set_buf_watermark() -- To set the software threshold/watermark.
>>> - hte_get_buf_watermark() -- To get the software threshold/watermark.
>>>
>>> The detail about parameters and API usage are described in each
>>> functions definitions in drivers/hte/hte.c file.
>>>
>>> The patch adds compilation support in Makefile and menu options in
>>> Kconfig.
>>>
>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>  
>> Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
>> around what this is doing + it is an RFC :)
> Having read on through the rest of the series, one of the biggest things that became
> clear is you have more layers of abstraction in here than make sense.  Squash things
> together so you have fewer allocations (and hence fewer error paths etc).
> Don't introduce ops functions unless you have more than one answer to what they
> are (and a clear justification for those).   It's easy to add layers of indirection
> later, but for now they just make your code harder to read.  Reality is that the buffer
> is a kfifo, make it so everywhere, rather than pretending otherwise.

I can remove buffer abstraction for sure. Do you find any other not needed

abstractions that I can rethink of?

>
> I thought for a while that you were allowing a different buffer implementation for
> each of the your hte chips and was very confused as to why that would make sense.
>
> Anyhow, it's interesting.  I'm not sure yet if the way it all fits together makes
> sense and you will almost certainly want to support hardware FIFOs and those tend
> to want to be partly exposed through to the consumer.

For now I am dealing with the providers which has hardware fifo but it is

not per event/entity i.e. it stores timestamps from all the enabled entities.

For such providers it is not feasible to expose hardware fifo. Can you point

to some references where it is exposed so that if needed I can accommodate

similar changes for any future providers.

>
> As to similar devices.  Lots of sensorhubs have timestamping facilities but it is
> tightly coupled to the data streams, so probably doesn't make sense to map to a
> generic subsystem like this.  You'll hit some of the same issues as those though
> when you try to align these timestamps with system ones etc.
What mechanism does sensorhub/iio now have to retrieve timestamp?
>
> otherwise, some of the counter devices are closer to this.  Perhaps the ti ecap?
> https://www.ti.com/lit/ug/spru807b/spru807b.pdf?ts=1625431159791&ref_url=https%253A%252F%252Fwww.google.com%252F
>
> That has a short hardware buffer and can be used for absolute timestamp grabbing
> on rising triggers etc.

I certainly have to read about this device. Initial target was to address in chip

HTEs and see (based on upstream comments) how can it be expanded to other

devices.

>
> Jonathan
>
>
>>> ---
>>>  drivers/Kconfig      |    2 +
>>>  drivers/Makefile     |    1 +
>>>  drivers/hte/Kconfig  |   22 +
>>>  drivers/hte/Makefile |    1 +
>>>  drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
>>>  include/linux/hte.h  |  278 +++++++++
>>>  6 files changed, 1672 insertions(+)
>>>  create mode 100644 drivers/hte/Kconfig
>>>  create mode 100644 drivers/hte/Makefile
>>>  create mode 100644 drivers/hte/hte.c
>>>  create mode 100644 include/linux/hte.h
>>>
>>> diff --git a/drivers/Kconfig b/drivers/Kconfig
>>> index 47980c6b1945..9b078964974b 100644
>>> --- a/drivers/Kconfig
>>> +++ b/drivers/Kconfig
>>> @@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
>>>  source "drivers/counter/Kconfig"
>>>  
>>>  source "drivers/most/Kconfig"
>>> +
>>> +source "drivers/hte/Kconfig"
>>>  endmenu
>>> diff --git a/drivers/Makefile b/drivers/Makefile
>>> index 5a6d613e868d..0a996a698e4c 100644
>>> --- a/drivers/Makefile
>>> +++ b/drivers/Makefile
>>> @@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
>>>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
>>>  obj-$(CONFIG_COUNTER)		+= counter/
>>>  obj-$(CONFIG_MOST)		+= most/
>>> +obj-$(CONFIG_HTE)		+= hte/
>>> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
>>> new file mode 100644
>>> index 000000000000..394e112f7dfb
>>> --- /dev/null
>>> +++ b/drivers/hte/Kconfig
>>> @@ -0,0 +1,22 @@
>>> +# SPDX-License-Identifier: GPL-2.0-only
>>> +menuconfig HTE
>>> +        bool "Hardware Timestamping Engine (HTE) Support"
>>> +        help
>>> +          Hardware Timestamping Engine (HTE) Support.  
>> Tidy this up, but think that's already been commented on.
>>
>>> +
>>> +          Some devices provide hardware timestamping engine which can timestamp
>>> +	  certain device lines/signals in realtime. This way to provide
>>> +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
>>> +	  comes with benefit for the applications like autonomous machines
>>> +	  needing accurate timestamping event with less jitter.
>>> +
>>> +	  This framework provides a generic interface to such HTE devices
>>> +          within the Linux kernel. It provides an API to register and
>>> +	  unregister a HTE provider chip, configurable sw buffer to
>>> +	  store the timestamps, push the timestamp from the HTE providers and
>>> +	  retrieve timestamps for the consumers. It also provides means for the
>>> +	  consumers to request signals it wishes to hardware timestamp and
>>> +	  release them if not required.
>>> +
>>> +          If unsure, say no.
>>> +
>>> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
>>> new file mode 100644
>>> index 000000000000..9899dbe516f7
>>> --- /dev/null
>>> +++ b/drivers/hte/Makefile
>>> @@ -0,0 +1 @@
>>> +obj-$(CONFIG_HTE)		+= hte.o
>>> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
>>> new file mode 100644
>>> index 000000000000..c53260d1e250
>>> --- /dev/null
>>> +++ b/drivers/hte/hte.c
>>> @@ -0,0 +1,1368 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * Copyright (c) 2021 NVIDIA Corporation
>>> + *
>>> + * Author: Dipen Patel <dipenp@nvidia.com>
>>> + */
>>> +
>>> +#include <linux/kernel.h>
>>> +#include <linux/module.h>
>>> +#include <linux/err.h>
>>> +#include <linux/slab.h>
>>> +#include <linux/of.h>
>>> +#include <linux/of_device.h>
>>> +#include <linux/kfifo.h>
>>> +#include <linux/mutex.h>
>>> +#include <linux/sched.h>
>>> +#include <linux/uaccess.h>
>>> +#include <linux/hte.h>
>>> +#include <linux/delay.h>
>>> +#include <linux/debugfs.h>
>>> +
>>> +/* Global list of the HTE devices */
>>> +static DEFINE_SPINLOCK(hte_lock);
>>> +static LIST_HEAD(hte_devices);
>>> +
>>> +enum {
>>> +	HTE_TS_REGISTERED,
>>> +	HTE_TS_DISABLE,
>>> +};
>>> +
>>> +/* Default FIFO depth */
>>> +#define HTE_EV_FIFO_EL		32
>>> +
>>> +#define HTE_TS_NAME_LEN		10
>>> +
>>> +struct hte_ts_buf;
>>> +
>>> +/**
>>> + * struct hte_ts_buf_acc_func - Software buffer management functions.
>>> + * @store: Store timestamp from atomic context as providers most likely
>>> + * be pushing timestamps from their interrupt handlers.
>>> + * @read: Read timestamps from the buffer.
>>> + * @el_available: Available timestamps to retrieve. The client can use this to
>>> + * query available elements so that it can pre-allocate internal buffer to send
>>> + * to during hte_retrieve_ts_ns API.
>>> + * @set_length: Set length/depth of the buffer.
>>> + * @get_length: Get length/depth of the buffer.
>>> + * @set_watermark: Set software threshold of the buffer.
>>> + * @get_watermark: Get software threshold of the buffer.
>>> + * @release: Release/free buffer.
>>> + * @reset: Reset the buffer.
>>> + */
>>> +struct hte_ts_buf_acc_func {
>>> +	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
>>> +	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
>>> +		    size_t *copied);
>>> +	size_t (*el_available)(struct hte_ts_buf *buf);
>>> +	int (*set_length)(struct hte_ts_buf *buf,
>>> +			  size_t length, size_t bpd);
>>> +	size_t (*get_length)(struct hte_ts_buf *buf);
>>> +	int (*set_watermark)(struct hte_ts_buf *buf,
>>> +			     size_t val);
>>> +	size_t (*get_watermark)(struct hte_ts_buf *buf);
>>> +	void (*release)(struct hte_ts_buf *buf);
>>> +	void (*reset)(struct hte_ts_buf *buf);
>>> +};
>>> +
>>> +/**
>>> + * struct hte_ts_buf - Software buffer per requested id or entity to store
>>> + * timestamps.
>>> + *
>>> + * @datum_len: Buffer depth or number of elements.
>>> + * @bytes_per_datum: Element size in bytes.
>>> + * @watermark: Software threshold at which client will be notified.
>>> + * @valid: Validity of the buffer.
>>> + * @pollq: Waitqueue for the blocking clients.
>>> + * @access: Various buffer management functions.
>>> + */
>>> +struct hte_ts_buf {
>>> +	size_t datum_len;
>>> +	size_t bytes_per_datum;
>>> +	size_t watermark;
>>> +	bool valid;
>>> +	wait_queue_head_t pollq;
>>> +	const struct hte_ts_buf_acc_func *access;
>>> +};
>>> +
>>> +/**
>>> + * struct hte_ts_info - Information related to requested timestamp.
>>> + *
>>> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
>>> + * See xlate callback API.
>>> + * @flags: Flags holding state informations.
>>> + * @seq: Timestamp sequence counter.
>>> + * @dropped_ts: Dropped timestamps.
>>> + * @cb: Callback to notify clients.
>>> + * @mlock: Lock during timestamp request/release APIs.
>>> + * @ts_dbg_root: Root for the debug fs.
>>> + * @gdev: HTE abstract device that this timestamp belongs to.
>>> + * @buf: Per requested timestamp software buffer.
>>> + * @desc: Timestamp descriptor understood between clients and HTE subsystem.
>>> + */
>>> +struct hte_ts_info {
>>> +	u32 xlated_id;
>>> +	unsigned long flags;
>>> +	u64 seq;
>>> +	atomic_t dropped_ts;
>>> +	void (*cb)(enum hte_notify n);
>>> +	struct mutex mlock;
>>> +	struct dentry *ts_dbg_root;
>>> +	struct hte_device *gdev;
>>> +	struct hte_ts_buf *buf;
> Where there is one instance, just embed it.  Lots of small allocations just make
> for less readable code.
Which specific instance you are alluding to desc and buf types?
>
>>> +	struct hte_ts_desc *desc;
>>> +};
>>> +
>>> +/**
>>> + * struct hte_device - HTE abstract device
>>> + * @nlines: Number of entities this device supports.
>>> + * @ts_req: Total number of entities requested.
>>> + * @ei: Timestamp information.
>>> + * @sdev: Device used at various debug prints.
>>> + * @dbg_root: Root directory for debug fs.
>>> + * @list: List node for internal use.  
>> Be more specific of what sort of internal use.
>>
>>> + * @chip: HTE chip providing this HTE device.
>>> + * @owner: helps prevent removal of modules when in use.
>>> + */
>>> +struct hte_device {
>>> +	u32 nlines;
>>> +	atomic_t ts_req;
>>> +	struct hte_ts_info *ei;
>>> +	struct device *sdev;
>>> +	struct dentry *dbg_root;
>>> +	struct list_head list;
>>> +	struct hte_chip *chip;
>>> +	struct module *owner;
>>> +};
>>> +
>>> +/* Buffer management functions */
>>> +
>>> +/**
>>> + * struct hte_kfifo - Software buffer wrapper.
>>> + * @buffer: Abstract buffer device.
>>> + * @gkf: Actual software buffer type, this case its FIFO.
>>> + */
>>> +struct hte_kfifo {
>>> +	struct hte_ts_buf buffer;
>>> +	struct kfifo gkf;
>>> +};
>>> +
>>> +#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
>>> +
>>> +static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
>>> +					size_t n)
>>> +{
>>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>>> +
>>> +	if (unlikely(!r->valid))
>>> +		return 0;
>>> +
>>> +	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
>>> +}
>>> +
>>> +static inline int hte_ts_buf_read(struct hte_ts_buf *r,
>>> +				  unsigned char *buf, size_t n,
>>> +				  size_t *copied)
>>> +{
>>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>>> +
>>> +	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
>>> +		return -EINVAL;
>>> +
>>> +	*copied = kfifo_out(&kf->gkf, buf, n);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
>>> +{
>>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>>> +
>>> +	if (!r->valid)
>>> +		return 0;
>>> +
>>> +	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
>>> +}
>>> +
>>> +static int hte_ts_buf_set_length(struct hte_ts_buf *r,
>>> +				 size_t length, size_t bpd)
>>> +{
>>> +	int ret = 0;
>>> +	struct hte_kfifo *buf;
>>> +
>>> +	if ((length == 0) || (bpd == 0) || !r)
>>> +		return -EINVAL;
>>> +
>>> +	buf = buf_to_kfifo(r);
>>> +
>>> +	if (r->datum_len != length) {
>>> +		if (r->valid)
>>> +			kfifo_free(&buf->gkf);
>>> +		r->valid = false;
>>> +		r->datum_len = length;
>>> +		r->bytes_per_datum = bpd;
>>> +		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
>>> +		if (!ret)
>>> +			r->valid = true;
>>> +	}
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
>>> +{
>>> +	if ((!r->valid) || !r->datum_len)
>>> +		return 0;
>>> +
>>> +	return r->datum_len;
>>> +}
>>> +
>>> +static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
>>> +{
>>> +	if ((!r->valid) || (val > r->datum_len))
>>> +		return -EINVAL;
>>> +
>>> +	r->watermark = val;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
>>> +{
>>> +	if (!r->valid)
>>> +		return 0;
>>> +
>>> +	return r->watermark;
>>> +}
>>> +
>>> +static inline void hte_ts_buf_release(struct hte_ts_buf *r)
>>> +{
>>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>>> +
>>> +	r->valid = false;
>>> +	kfifo_free(&kf->gkf);
>>> +	kfree(kf);
>>> +}
>>> +
>>> +static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
>>> +{
>>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
>>> +
>>> +	if (!r->valid)
>>> +		return;
>>> +
>>> +	kfifo_reset(&kf->gkf);
>>> +}
>>> +
>>> +static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
>>> +	.store = &hte_ts_store_to_buf,
>>> +	.read = &hte_ts_buf_read,
>>> +	.el_available = &hte_ts_buf_el_available,
>>> +	.set_length = &hte_ts_buf_set_length,
>>> +	.get_length = &hte_ts_buf_get_length,
>>> +	.set_watermark = &hte_ts_buf_set_watermark,
>>> +	.get_watermark = &hte_ts_buf_get_watermark,
>>> +	.release = &hte_ts_buf_release,
>>> +	.reset = &hte_ts_buf_reset,
>>> +};
>>> +
>>> +static struct hte_ts_buf *hte_ts_buf_allocate(void)
>>> +{
>>> +	struct hte_kfifo *kf;
>>> +
>>> +	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
>>> +	if (!kf)
>>> +		return ERR_PTR(-ENOMEM);
>>> +
>>> +	init_waitqueue_head(&kf->buffer.pollq);
>>> +	kf->buffer.watermark = 1;
>>> +	kf->buffer.datum_len = 0;
>>> +	kf->buffer.valid = false;
>>> +	kf->buffer.access = &kfifo_access_funcs;
> Why do you have this level abstraction?  I would suggest you flatten all this
> into direct calls until you have some clear need for multiple buffer types.
>
> A long long time ago (10 years or more) we had similar abstractions in IIO because
> we though it would be helpful to support ring buffers and kfifos.  It wasn't, we ended
> up ripping them all out because all they resulted in was a more complex code base
> when in reality everyone was happy with a kfifo.

I agree. Initially I thought future providers may need different type of buffers so had

abstraction layer. But I guess its ok to not worry about that for now. I will remove this

in next RFC version.

>
> Jonathan
>
>
>>> +
>>> +	return &kf->buffer;
>>> +}
>>> +/* End of buffer management */
>>> +
>>> +/* Debugfs management */
>>> +
>>> +#ifdef CONFIG_DEBUG_FS
>>> +
>>> +static struct dentry *hte_root;
>>> +
>>> +static void __init hte_subsys_dbgfs_init(void)
>>> +{
>>> +	/* creates /sys/kernel/debug/hte/ */
>>> +	hte_root = debugfs_create_dir("hte", NULL);
>>> +}
>>> +subsys_initcall(hte_subsys_dbgfs_init);
>>> +
>>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>>> +{
>>> +	const struct hte_chip *chip = gdev->chip;
>>> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
>>> +
>>> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
>>> +	if (!gdev->dbg_root)
>>> +		return;
>>> +
>>> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
>>> +				&gdev->ts_req);
>>> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
>>> +			   &gdev->nlines);
>>> +}
>>> +
>>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>>> +{
>>> +	if (!ei->gdev->dbg_root || !name)
>>> +		return;
>>> +
>>> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
>>> +	if (!ei->ts_dbg_root)
>>> +		return;
>>> +
>>> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
>>> +			      &ei->buf->datum_len);
>>> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
>>> +			      &ei->buf->watermark);
>>> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
>>> +				&ei->dropped_ts);
>>> +}
>>> +
>>> +static inline void hte_dbgfs_deinit(struct dentry *root)
>>> +{
>>> +	if (!root)
>>> +		return;
>>> +
>>> +	debugfs_remove_recursive(root);
>>> +}
>>> +
>>> +#else
>>> +
>>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
>>> +{
>>> +}
>>> +
>>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
>>> +{
>>> +}
>>> +
>>> +static inline void hte_dbgfs_deinit(struct dentry *root)
>>> +{
>>> +}
>>> +
>>> +#endif
>>> +/* end of debugfs management*/
>>> +
>>> +/* Driver APIs */
>>> +
>>> +/**
>>> + * hte_release_ts() - Consumer calls this API to release the entity, where
>>> + * entity could be anything providers support, like lines, signals, buses,
>>> + * etc...
>>> + *
>>> + * The correct sequence to call this API is as below:
>>> + * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
>>> + * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
>>> + * still care about the data.
>>> + * 3) Call this API.
>>> + * Above sequence makes sure that entity gets released race free.
>>> + *
>>> + * @desc: timestamp descriptor, this is the same as returned by the request API.
>>> + *
>>> + * Context: hte_dbgfs_deinit() function call may use sleeping locks,
>>> + *	    not suitable from atomic context in that case.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_release_ts(struct hte_ts_desc *desc)
>>> +{
>>> +	u32 id;
>>> +	int ret = 0;
>>> +	struct hte_device *gdev;
>>> +	struct hte_ts_info *ei;
>>> +	struct hte_ts_buf *buf;
>>> +
>>> +	if (!desc)
>>> +		return -EINVAL;
>>> +
>>> +	ei = (struct hte_ts_info *)desc->data_subsys;  
>> As data_subsys is void * you don't need to explicitly cast it to another pointer type.
>>
>>> +
>>> +	if (!ei || !ei->gdev || !ei->buf)
>>> +		return -EINVAL;
>>> +
>>> +	gdev = ei->gdev;
>>> +	buf = ei->buf;
>>> +	id = desc->con_id;
>>> +
>>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>>> +		dev_info(gdev->sdev, "id:%d is not registered", id);
>>> +		return -EUSERS;
>>> +	}
>>> +
>>> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
>>> +	if (ret) {
>>> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
>>> +		goto out;
>>> +	}
>>> +
>>> +	atomic_dec(&gdev->ts_req);
>>> +	atomic_set(&ei->dropped_ts, 0);
>>> +
>>> +	kfree(desc->name);
>>> +	kfree(desc);
>>> +	ei->desc = NULL;
>>> +	ei->seq = 0;
>>> +	buf->access->release(buf);
>>> +
>>> +	hte_dbgfs_deinit(ei->ts_dbg_root);
>>> +	module_put(gdev->owner);
>>> +
>>> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
>>> +
>>> +out:
>>> +	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
>>> +	return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_release_ts);
>>> +
>>> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
>>> +{
>>> +	u32 ts_id;
>>> +	struct hte_device *gdev;
>>> +	struct hte_ts_info *ei;
>>> +	int ret;
>>> +
>>> +	if (!desc)
>>> +		return -EINVAL;
>>> +
>>> +	ei = (struct hte_ts_info *)desc->data_subsys;  
>> As above, no need to cast - though it rather implies the type of data_subsys
>> should not be void *.
>>
>>> +
>>> +	if (!ei || !ei->gdev)
>>> +		return -EINVAL;
>>> +
>>> +	gdev = ei->gdev;
>>> +	ts_id = desc->con_id;
>>> +
>>> +	mutex_lock(&ei->mlock);
>>> +
>>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>>> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
>>> +		ret = -EUSERS;
>>> +		goto out;
>>> +	}
>>> +
>>> +	if (en) {
>>> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
>>> +			ret = 0;
>>> +			goto out;
>>> +		}
>>> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
>>> +		if (ret) {
>>> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
>>> +				 ts_id);
>>> +			goto out;
>>> +		}
>>> +
>>> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
>>> +		ret = 0;  
>> ret is already 0 so no point in setting it again.
>>
>>> +	} else {
>>> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
>>> +			ret = 0;
>>> +			goto out;
>>> +		}
>>> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
>>> +		if (ret) {
>>> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
>>> +				 ts_id);
>>> +			goto out;
>>> +		}
>>> +
>>> +		set_bit(HTE_TS_DISABLE, &ei->flags);
>>> +		ret = 0;
>>> +	}
>>> +
>>> +out:
>>> +	mutex_unlock(&ei->mlock);
>>> +	return ret;
>>> +}
>>> +
>>> +/**
>>> + * hte_disable_ts() - Disable timestamp on given descriptor.
>>> + *
>>> + * @desc: ts descriptor, this is the same as returned by the request API.
>>> + *
>>> + * Context: Holds mutex lock, not suitable from atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_disable_ts(struct hte_ts_desc *desc)
>>> +{
>>> +	return hte_ts_dis_en_common(desc, false);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_disable_ts);
>>> +
>>> +/**
>>> + * hte_enable_ts() - Enable timestamp on given descriptor.
>>> + *
>>> + * @desc: ts descriptor, this is the same as returned by the request API.
>>> + *
>>> + * Context: Holds mutex lock, not suitable from atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_enable_ts(struct hte_ts_desc *desc)
>>> +{
>>> +	return hte_ts_dis_en_common(desc, true);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_enable_ts);
>>> +
>>> +static int hte_simple_xlate(struct hte_chip *gc,
>>> +			    const struct of_phandle_args *args,
>>> +			    struct hte_ts_desc *desc,
>>> +			    u32 *id)
>>> +{
>>> +	if (!id || !desc || !gc)
>>> +		return -EINVAL;
>>> +
>>> +	/*
>>> +	 * For the providers which do not have any internal mappings between
>>> +	 * logically exposed ids and actual ids, will set both
>>> +	 * the same.
>>> +	 *
>>> +	 * In case there is a internal mapping needed, providers will need to
>>> +	 * provide its own xlate function where con_id will be sent as
>>> +	 * args[0] and it will return xlated id. Later xlated id will be
>>> +	 * used for any future exchanges between provider and subsystems.
>>> +	 */
>>> +
>>> +	if (args) {
>>> +		if (gc->of_hte_n_cells < 1)
>>> +			return -EINVAL;
>>> +
>>> +		if (args->args_count != gc->of_hte_n_cells)
>>> +			return -EINVAL;
>>> +
>>> +		*id = args->args[0];
>>> +		desc->con_id = *id;
>>> +	} else {
>>> +		*id = desc->con_id;
>>> +	}
>>> +
>>> +	if (desc->con_id > gc->nlines)
>>> +		return -EINVAL;
>>> +
>>> +	desc->data_subsys = NULL;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
>>> +{
>>> +	struct hte_device *gdev;
>>> +
>>> +	spin_lock(&hte_lock);
>>> +
>>> +	list_for_each_entry(gdev, &hte_devices, list)
>>> +		if (gdev->chip && gdev->chip->dev &&
>>> +		    gdev->chip->dev->of_node == np) {
>>> +			spin_unlock(&hte_lock);
>>> +			return gdev;
>>> +		}
>>> +
>>> +	spin_unlock(&hte_lock);
>>> +
>>> +	return ERR_PTR(-ENODEV);
>>> +}
>>> +
>>> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
>>> +			 u32 xlated_id, void (*cb)(enum hte_notify n))
>>> +{
>>> +	struct hte_ts_info *ei;
>>> +	struct hte_ts_buf *buf;
>>> +	int ret;
>>> +	u32 con_id = desc->con_id;
>>> +
>>> +	if (!try_module_get(gdev->owner))
>>> +		return -ENODEV;
>>> +
>>> +	ei = &gdev->ei[xlated_id];
>>> +	ei->xlated_id = xlated_id;
>>> +
>>> +	/*
>>> +	 * There a chance that multiple consumers requesting same entity,
>>> +	 * lock here.
>>> +	 */
>>> +	mutex_lock(&ei->mlock);
>>> +
>>> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
>>> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
>>> +			xlated_id);
>>> +		ret = -EUSERS;
>>> +		goto unlock;
>>> +	}
>>> +
>>> +	buf = hte_ts_buf_allocate();
>>> +	if (IS_ERR(buf)) {
>>> +		dev_err(gdev->chip->dev, "Buffer allocation failed");
>>> +		ret = PTR_ERR(buf);
>>> +		goto unlock;
>>> +	}
>>> +
>>> +	/* Set default here, let consumer decide how much to set later */
>>> +	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
>>> +				      sizeof(struct hte_ts_data));
>>> +  
>> It's good to keep to consistent style of no line break between a statement
>> and it's error check.
>>
>>> +	if (ret) {
>>> +		dev_err(gdev->chip->dev, "Fifo set length failed");
>>> +		goto buf_rel;
>>> +	}
>>> +
>>> +	buf->access->reset(buf);
>>> +	buf->valid = true;
>>> +
>>> +	ei->buf = buf;
>>> +	ei->cb = cb;
>>> +
>>> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
>>> +	if (ret < 0) {
>>> +		dev_err(gdev->chip->dev, "ts request failed\n");
>>> +		goto buf_rel;
>>> +	}
>>> +
>>> +	desc->data_subsys = ei;
>>> +	ei->desc = desc;
>>> +
>>> +	atomic_inc(&gdev->ts_req);
>>> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
>>> +	mutex_unlock(&ei->mlock);
>>> +
>>> +	if (!desc->name) {
>>> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
>>> +		if (desc->name)
>>> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
>>> +				  con_id);
>>> +	}
>>> +
>>> +	hte_ts_dbgfs_init(desc->name, ei);
>>> +
>>> +	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
>>> +		__func__, con_id, xlated_id);
>>> +
>>> +	return 0;
>>> +
>>> +buf_rel:
>>> +	buf->access->release(buf);
>>> +unlock:
>>> +	module_put(gdev->owner);
>>> +	mutex_unlock(&ei->mlock);
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static struct hte_device *of_hte_dev_get(struct device *dev,
>>> +					 struct device_node *np,
>>> +					 const char *label,
>>> +					 struct of_phandle_args *args)
>>> +{
>>> +	struct hte_device *gdev = NULL;
>>> +	int index = 0;
>>> +	int err;
>>> +
>>> +	if (label) {
>>> +		index = of_property_match_string(np, "hte-names", label);
>>> +		if (index < 0)
>>> +			return ERR_PTR(index);
>>> +	}
>>> +
>>> +	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
>>> +					 args);
>>> +	if (err) {
>>> +		pr_err("%s(): can't parse \"htes\" property\n", __func__);
>>> +		return ERR_PTR(err);
>>> +	}
>>> +
>>> +	gdev = of_node_to_htedevice(args->np);
>>> +	if (IS_ERR(gdev)) {
>>> +		pr_err("%s(): HTE chip not found\n", __func__);
>>> +		of_node_put(args->np);
>>> +		return gdev;
>>> +	}
>>> +
>>> +	return gdev;
>>> +}
>>> +
>>> +static struct hte_ts_desc *__hte_req_ts(struct device *dev,
>>> +					struct device_node *np,
>>> +					const char *label,
>>> +					void (*cb)(enum hte_notify n))
>>> +{
>>> +	struct hte_device *gdev = NULL;
>>> +	struct hte_ts_desc *desc;
>>> +	struct of_phandle_args args;
>>> +	int ret;
>>> +	u32 xlated_id;
>>> +
>>> +	gdev = of_hte_dev_get(dev, np, label, &args);
>>> +	if (IS_ERR(gdev))
>>> +		return ERR_CAST(gdev);
>>> +
>>> +	if (!gdev->chip) {
>>> +		pr_debug("requested id does not have provider\n");
>>> +		return ERR_PTR(-ENODEV);
>>> +	}
>>> +
>>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
>>> +	if (!desc)
>>> +		return ERR_PTR(-ENOMEM);
>>> +
>>> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
>>> +	if (ret < 0)
>>> +		goto put;
>>> +
>>> +	desc->name = NULL;
>>> +	if (label)
>>> +		desc->name = kstrdup(label, GFP_KERNEL);
>>> +
>>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
>>> +	if (ret < 0)
>>> +		goto put;
>>> +
>>> +	return desc;
>>> +
>>> +put:
>>> +	of_node_put(args.np);
>>> +	kfree(desc);
>>> +
>>> +	return ERR_PTR(ret);
>>> +}
>>> +
>>> +/**
>>> + * of_hte_request_ts() - Consumer calls this API to request the HTE facility
>>> + * on the specified entity, where entity is provider specific for example,
>>> + * GPIO lines, signals, buses etc...
>>> + *
>>> + * @dev: Consumer device.
>>> + * @label: Optional label.
>>> + * @cb: Optional notify callback to consumer when data is pushed by the
>>> + * provider.
>>> + *
>>> + * Context: Holds mutex lock, not suitable from atomic context.
>>> + * Returns: Timestamp descriptor on success or error ptr on failure.
>>> + */
>>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev,
>>> +				      const char *label,
>>> +				      void (*cb)(enum hte_notify n))
>>> +{
>>> +
>>> +	if (dev && dev->of_node)
>>> +		return __hte_req_ts(dev, dev->of_node, label, cb);
>>> +	else
>>> +		return ERR_PTR(-EOPNOTSUPP);
>>> +}
>>> +EXPORT_SYMBOL_GPL(of_hte_request_ts);
>>> +
>>> +static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)  
>> I'm not seeing what is devm about this.
>>
>>> +{
>>> +	struct hte_ts_desc **p = res;
>>> +
>>> +	if (WARN_ON(!p || !*p))
>>> +		return 0;
>>> +
>>> +	return *p == data;
>>> +}
>>> +
>>> +static void __devm_hte_release_ts(struct device *dev, void *res)
>>> +{
>>> +	hte_release_ts(*(struct hte_ts_desc **)res);
>>> +}
>>> +
>>> +/**
>>> + * devm_hte_release_ts() - Resource managed hte_release_ts().  
>> I'd not introduce this until you have a user.  It very rarely actually makes
>> sense to call a devm release manually. Not having one makes people think harder
>> about it.
>>
>>> + * @dev: HTE consumer/client device.
>>> + * @desc: HTE ts descriptor.
>>> + *
>>> + * Release timestamp functionality and its resources previously allocated using
>>> + * of_hte_request_ts(). Calling this function is usually not needed because
>>> + * devm-allocated resources are automatically released on driver detach.
>>> + *
>>> + * Context: Same as hte_release_ts() function.
>>> + * Returns: 0 on success otherwise negative error code.
>>> + */
>>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
>>> +{
>>> +	return devres_release(dev, __devm_hte_release_ts,
>>> +			      devm_hte_ts_match_desc, desc);
>>> +}
>>> +EXPORT_SYMBOL_GPL(devm_hte_release_ts);
>>> +
>>> +/**
>>> + * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().  
>> If it's kernel-doc it needs to give no warnings when you point the kernel-doc
>> scripts at it.  They insist on full parameter documentation.
>>
>>> + */
>>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
>>> +					   const char *label,
>>> +					   void (*cb)(enum hte_notify n))
>>> +{
>>> +
>>> +	struct hte_ts_desc **ptr, *desc;
>>> +
>>> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);  
>> Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
>> and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
>> last kernel cycle.
>>
>>
>>> +	if (!ptr)
>>> +		return ERR_PTR(-ENOMEM);
>>> +
>>> +	desc = of_hte_request_ts(dev, label, cb);
>>> +	if (!IS_ERR(desc)) {
>>> +		*ptr = desc;
>>> +		devres_add(dev, ptr);
>>> +	} else {
>>> +		devres_free(ptr);
>>> +	}
>>> +	return desc;
>>> +}
>>> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
>>> +
>>> +static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
>>> +					  size_t val)  
>> Not a good name or indeed combination of different things.
>> hte_desc_to_info() and some separate check on val would be better.
>>
>>> +{
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	if (!desc || !desc->data_subsys || !val) {
>>> +		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
>>> +		return NULL;
>>> +	}
>>> +
>>> +	ei = desc->data_subsys;
>>> +	if (!ei || !ei->buf) {
>>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>>> +		return NULL;
>>> +	}
>>> +
>>> +	return ei;
>>> +}
>>> +
>>> +static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
>>> +{
>>> +	size_t el_avail;
>>> +
>>> +	el_avail = buffer->access->el_available(buffer);
>>> +
>>> +	return (el_avail >= to_read) ? false : true;  
>> return el_avail < to_read;
>>
>>> +}
>>> +
>>> +static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
>>> +			       struct hte_ts_data *el, size_t n, bool block)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +	int ret;
>>> +	size_t to_read, copied;
>>> +
>>> +	ei = hte_para_check(desc, n);
>>> +	if (!ei)
>>> +		return -EINVAL;
>>> +
>>> +	buffer = ei->buf;
>>> +
>>> +	to_read = min_t(size_t, n, buffer->watermark);  
>> Needs a comment as not obvious why you'd read the min of that requested or
>> the watermark if there might be more available.
>>
>>> +
>>> +	do {
>>> +		if (hte_ts_buf_wait(buffer, to_read)) {
>>> +			if (!block) {
>>> +				/* Possibly early here to retrieve, try again */
>>> +				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
>>> +					__func__, ret);
>>> +				return -EAGAIN;
>>> +			}
>>> +			ret = wait_event_interruptible(buffer->pollq,
>>> +					!hte_ts_buf_wait(buffer, to_read));
>>> +			if (ret)
>>> +				return ret;
>>> +		}
>>> +		ret = buffer->access->read(buffer, (void *)el,  
>> If you have to cast to a void * that usually means something is wrong in your definitions.
>> Why is it needed here?  Looks like read has an inappropriate definition.
>>
>>> +					   n * buffer->bytes_per_datum,
>>> +					   &copied);
>>> +		if (ret < 0)
>>> +			return ret;
>>> +
>>> +		if (copied > 0)
>>> +			return 0;
>>> +		else if (copied == 0 && !block)
>>> +			return -EAGAIN;
>>> +	} while (copied == 0);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +/**
>>> + * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
>>> + * nano seconds i.e. el->tsc will be in ns.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + * @el: buffer to store the timestamp details.
>>> + * @n: Number of struct hte_timestamp_el elements.
>>> + *
>>> + * Context: Can be called from the atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
>>> +		       struct hte_ts_data *el, size_t n)
>>> +{
>>> +	return _hte_retrieve_ts_ns(desc, el, n, false);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
>>> +
>>> +/**
>>> + * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + * @el: buffer to store the timestamp data.
>>> + * @n: Number of struct hte_ts_data data.
>>> + *
>>> + * Context: Can not be called from the atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
>>> +			    struct hte_ts_data *el, size_t n)
>>> +{
>>> +	return _hte_retrieve_ts_ns(desc, el, n, true);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
>>> +
>>> +/**
>>> + * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
>>> + * depth.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + * @len: New length/depth.
>>> + *
>>> + * The correct sequence to set buffer length is as below:
>>> + * 1) Disable timestamp by calling hte_disable_ts API.
>>> + * 2) Optionally retrieve all the timestamps by calling non blocking
>>> + *    hte_retrieve_ts_ns() API. This step only needed if you still care about
>>> + *    the data.
>>> + * 3) Call this API.
>>> + * 4) Enable timestamp by calling hte_enable_ts API.
>>> + *
>>> + * This API destroys previously allocated buffer and creates new one, because
>>> + * of that, it is mandatory to follow above sequence to make sure there is no
>>> + * race between various other APIs in the subsystem.  
>> Good docs.  This is why I mentioned in review of docs patch that it is better
>> to just have that refer to the kernel-doc in these files.  Keep all this good
>> information in one place.
>>
>>> + *
>>> + * By default during the request API call, HTE subsystem allocates software
>>> + * buffer with predefined length, this API gives flexibility to adjust the
>>> + * length according to consumer's need.
>>> + *
>>> + * Context: Can not be called from atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +	int ret;
>>> +
>>> +	ei = hte_para_check(desc, len);
>>> +	if (!ei)
>>> +		return -EINVAL;
>>> +
>>> +	buffer = ei->buf;
>>> +	ret = buffer->access->set_length(buffer, len,
>>> +					 sizeof(struct hte_ts_data));
>>> +	if (ret)
>>> +		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);  
>> Not point in printing things line __func__ manually in dev_err() etc.
>> Dynamic debug includes that and gives far more information + control of this.
>>
>>> +
>>> +	return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_set_buf_len);
>>> +
>>> +/**
>>> + * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
>>> + * depth or length.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + *
>>> + * Context: Any context.
>>> + * Returns: Positive length on success or 0 on failure.
>>> + */
>>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	ei = hte_para_check(desc, 1);
>>> +	if (!ei)
>>> +		return 0;
>>> +
>>> +	buffer = ei->buf;
>>> +
>>> +	return buffer->access->get_length(buffer);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_get_buf_len);
>>> +
>>> +/**
>>> + * hte_available_ts() - Returns total available timestamps.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + *
>>> + * The API helps consumers to pre-allocate its internal buffer required
>>> + * during hte_retrieve_ts_ns call.
>>> + *
>>> + * Context: Any context.
>>> + * Returns: Positive value if elements are available else 0. The value is
>>> + * number of total available struct hte_timestamp_el elements available not
>>> + * the size in bytes.
>>> + */
>>> +size_t hte_available_ts(const struct hte_ts_desc *desc)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	ei = hte_para_check(desc, 1);
>>> +	if (!ei)
>>> +		return 0;
>>> +
>>> +	buffer = ei->buf;
>>> +
>>> +	return buffer->access->el_available(buffer);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_available_ts);
>>> +
>>> +/**
>>> + * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
>>> + * buffer watermark. The correct sequence to call this API is as below:
>>> + * 1) Disable timestamp by calling hte_disable_ts API.
>>> + * 2) Call this API.
>>> + * 3) Enable timestamp by calling hte_enable_ts API.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + * @val: New watermark.
>>> + *
>>> + * By default during the request API call, HTE subsystem sets watermark as 1,
>>> + * this API gives flexibility to adjust the watermark according to consumer's
>>> + * need. The consumers will get notification through callback registered during
>>> + * request API either when timestamp is dropped or watermark is reached or will
>>> + * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
>>> + * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
>>> + *
>>> + * Context: Any context.  
>> You have no way of knowing that as will depend on the driver - I'd definitely
>> suggest not from atomic context, but then that would be crazy so you are better
>>  off not documenting any specific requirement at all.
>>
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +	int ret;
>>> +
>>> +	ei = hte_para_check(desc, val);
>>> +	if (!ei)
>>> +		return -EINVAL;
>>> +
>>> +	buffer = ei->buf;
>>> +	ret = buffer->access->set_watermark(buffer, val);
>>> +	if (ret)
>>> +		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
>>> +
>>> +	return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
>>> +
>>> +/**
>>> + * hte_get_buf_watermark() - Consumer calls this API to get software
>>> + * buffer watermark.
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + *
>>> + * Context: Any context.
>>> + * Returns: Positive current watermark on success or 0 on failure.
>>> + */
>>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
>>> +{
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	ei = hte_para_check(desc, 1);
>>> +	if (!ei)
>>> +		return 0;
>>> +
>>> +	buffer = ei->buf;
>>> +
>>> +	return buffer->access->get_watermark(buffer);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
>>> +
>>> +/**
>>> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
>>> + * node directly, where meaning of the entity is provider specific, for example
>>> + * lines, signals, GPIOs, buses etc...
>>> + *
>>> + * @of_node: HTE provider device node.
>>> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
>>> + * @cb: Optional callback to notify.
>>> + *
>>> + * Context: Holds mutex lock, can not be called from atomic context.  
>> What mutex and why?  If it is one you can check is held even better.
>>
>>> + * Returns: ts descriptor on success or error pointers.
>>> + */
>>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
>>> +					  unsigned int id,
>>> +					  void (*cb)(enum hte_notify n))
>>> +{
>>> +	struct hte_device *gdev;
>>> +	struct hte_ts_desc *desc;
>>> +	int ret;
>>> +	u32 xlated_id;
>>> +
>>> +	gdev = of_node_to_htedevice(of_node);
>>> +	if (IS_ERR(gdev))
>>> +		return ERR_PTR(-ENOTSUPP);
>>> +
>>> +	if (!gdev->chip || !gdev->chip->ops)
>>> +		return ERR_PTR(-ENOTSUPP);
>>> +
>>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
>>> +	if (!desc) {
>>> +		ret = -ENOMEM;
>>> +		goto out_put_device;
>>> +	}  
>> Pass a desc pointer into this function rather than allocating the structure
>> in here.  That lets the caller embed that structure inside one of it's own
>> structures if it wants to, resulting in fewer small allocations which is always good.
>>
>> It's far from obvious that the caller needs to free desc.
>>
>>> +
>>> +	desc->con_id = id;
>>> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
>>> +	if (ret < 0) {
>>> +		dev_err(gdev->chip->dev,
>>> +			"failed to xlate id: %d\n", id);
>>> +		goto out_free_desc;
>>> +	}
>>> +
>>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
>>> +	if (ret < 0) {
>>> +		dev_err(gdev->chip->dev,
>>> +			"failed to request id: %d\n", id);
>>> +		goto out_free_desc;
>>> +	}
>>> +
>>> +	return desc;
>>> +
>>> +out_free_desc:
>>> +	kfree(desc);
>>> +
>>> +out_put_device:
>>> +	return ERR_PTR(ret);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
>>> +
>>> +/**
>>> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
>>> + * information of the desc.
>>> + *
>>> + * @desc: ts descriptor, same as returned from request API.
>>> + *
>>> + * Context: Any context.
>>> + * Returns: 0 on success else negative error code on failure.
>>> + */
>>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>>> +			 struct hte_clk_info *ci)
>>> +{
>>> +	struct hte_chip *chip;
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	if (!desc || !desc->data_subsys || !ci) {
>>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	ei = desc->data_subsys;
>>> +	if (!ei || !ei->gdev || !ei->gdev->chip)
>>> +		return -EINVAL;
>>> +
>>> +	chip = ei->gdev->chip;
>>> +	if (!chip->ops->get_clk_src_info)
>>> +		return -ENOTSUPP;
>>> +
>>> +	return chip->ops->get_clk_src_info(chip, ci);
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
>>> +
>>> +static inline void hte_add_to_device_list(struct hte_device *gdev)
>>> +{
>>> +	struct hte_device *prev;  
>> Needs to take an appropriate lock as you may have concurrent calls.
>>
>>> +
>>> +	if (list_empty(&hte_devices)) {
>>> +		list_add_tail(&gdev->list, &hte_devices);  
>> Needs a comment. I've no idea why you might want to only add it if there were
>> no other hte_devices already there.
>>
>>> +		return;
>>> +	}
>>> +
>>> +	prev = list_last_entry(&hte_devices, struct hte_device, list);  
>> Why woud you do this?
>>
>>> +	list_add_tail(&gdev->list, &hte_devices);
>>> +}
>>> +
>>> +/**
>>> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
>>> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
>>> + * using this API from its ISR or atomic context.
>>> + *
>>> + * @chip: The HTE chip, used during the registration.
>>> + * @xlated_id: entity id understood by both subsystem and provider, usually this
>>> + * is obtained from xlate callback during request API.
>>> + * @data: timestamp data.
>>> + * @n: Size of the data.
>>> + *
>>> + * Context: Atomic.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
>>> +			  struct hte_ts_data *data, size_t n)
>>> +{
>>> +	unsigned int ret;
>>> +	bool notify;
>>> +	size_t el_avail;
>>> +	struct hte_ts_buf *buffer;
>>> +	struct hte_ts_info *ei;
>>> +
>>> +	if (!chip || !data || !chip->gdev)
>>> +		return -EINVAL;
>>> +
>>> +	if (xlated_id > chip->nlines)
>>> +		return -EINVAL;
>>> +
>>> +	ei = &chip->gdev->ei[xlated_id];
>>> +
>>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
>>> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
>>> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* timestamp sequence counter, start from 0 */
>>> +	data->seq = ei->seq++;
>>> +
>>> +	buffer = ei->buf;
>>> +	el_avail = buffer->access->el_available(buffer);  
>>> +	ret = buffer->access->store(buffer, data, n);  
>> If we are doing this from the hte core, why is buffer definition in the scope of the
>> drivers rather than the core?  That seems backwards to me.
>>
>>> +	if (ret != n) {
>>> +		atomic_inc(&ei->dropped_ts);
>>> +		if (ei->cb)
>>> +			ei->cb(HTE_TS_DROPPED);
>>> +		return -ENOMEM;
>>> +	}
>>> +
>>> +	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;  
>> You push n but only check on el_avail + 1 here.  
>> Also, this is the same as
>>
>> 	notify = ((el_avail + 1) >= buffer->watermark;
>>
>>
>>> +
>>> +	/*
>>> +	 * If there is a callback, its consumer's job to retrieve the timestamp.
>>> +	 * For the rest, wake up the process.
>>> +	 */
>>> +	if (notify && ei->cb) {
>>> +		ei->cb(HTE_TS_AVAIL);
>>> +		return 0;  
>> Given you return 0 anyway, might as well not have this line.
>>
>>> +	} else if (notify) {
>>> +		wake_up_interruptible(&buffer->pollq);
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
>>> +
>>> +/**
>>> + * hte_register_chip() - Used by provider to register a HTE chip.
>>> + * @chip: the HTE chip to add to subsystem.
>>> + *
>>> + * Context: Can not be called from atomic context.  
>> Whilst true, I'd think that was common sense as it would be insane
>> to register something like this from atomic context.  So I'd say no
>> need to comment on it!  Keep those comments for things that
>> might be used like that.
>>
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_register_chip(struct hte_chip *chip)
>>> +{
>>> +	struct hte_device *gdev;
>>> +	int ret;
>>> +	u32 i;
>>> +
>>> +	if (!chip || !chip->dev || !chip->dev->of_node)
>>> +		return -EINVAL;
>>> +
>>> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
>>> +		dev_err(chip->dev, "Driver needs to provide ops\n");
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
>>> +	if (!gdev)
>>> +		return -ENOMEM;
>>> +
>>> +	gdev->chip = chip;
>>> +	chip->gdev = gdev;
>>> +	gdev->nlines = chip->nlines;
>>> +	gdev->sdev = chip->dev;
>>> +
>>> +	/*
>>> +	 * Allocate all the supported entities here at once, this will have
>>> +	 * following advantages:
>>> +	 * When provider pushes timestamp, it can then just send the
>>> +	 * xlated_id, subsystem will use it as an index which
>>> +	 * gives us the constant time access; this is important as mostly
>>> +	 * providers will be pushing the timestamps from their ISR.
>>> +	 */
>>> +	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
>>> +			   GFP_KERNEL);  
>> I'd be tempted to do this as a 0 length element at the end of gdev
>> then do the allocation in one go use struct_size() etc to work out
>> how long it is.  Cuts down on allocations + error paths to deal with
>> for no obvious disadvantage.
>>
>>> +	if (!gdev->ei) {
>>> +		ret = -ENOMEM;
>>> +		goto err_free_gdev;
>>> +	}
>>> +
>>> +	for (i = 0; i < chip->nlines; i++) {
>>> +		gdev->ei[i].flags = 0;  
>> zero allocated, so don't bother setting things to 0 where it's a fairly obvious
>> base state.  If you set something to 0 to act as some form of documentation then
>> that's fine, but I don't think that's true here.
>>
>>> +		gdev->ei[i].gdev = gdev;
>>> +		gdev->ei[i].seq = 0;
>>> +		mutex_init(&gdev->ei[i].mlock);
>>> +	}
>>> +
>>> +	if (chip->dev->driver)
>>> +		gdev->owner = chip->dev->driver->owner;
>>> +	else
>>> +		gdev->owner = THIS_MODULE;
>>> +
>>> +	if (!chip->xlate) {
>>> +		chip->xlate = hte_simple_xlate;
>>> +		/* Just a id number to monitor */
>>> +		chip->of_hte_n_cells = 1;
>>> +	}
>>> +
>>> +	of_node_get(chip->dev->of_node);
>>> +
>>> +	INIT_LIST_HEAD(&gdev->list);
>>> +
>>> +	spin_lock(&hte_lock);
>>> +	hte_add_to_device_list(gdev);
>>> +	spin_unlock(&hte_lock);
>>> +
>>> +	hte_chip_dbgfs_init(gdev);
>>> +
>>> +	dev_dbg(chip->dev, "Added hte chip\n");
>>> +	return 0;
>>> +
>>> +err_free_gdev:
>>> +	kfree(gdev);
>>> +
>>> +	return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_register_chip);
>>> +
>>> +/**
>>> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
>>> + * @chip: the HTE chip to remove.
>>> + *
>>> + * Context: Can not be called from atomic context.
>>> + * Returns: 0 on success or a negative error code on failure.
>>> + */
>>> +int hte_unregister_chip(struct hte_chip *chip)
>>> +{
>>> +	struct hte_device *gdev = chip->gdev;
>>> +
>>> +	spin_lock(&hte_lock);
>>> +	list_del(&gdev->list);
>>> +	spin_unlock(&hte_lock);
>>> +
>>> +	gdev->chip = NULL;
>>> +
>>> +	of_node_put(chip->dev->of_node);
>>> +	hte_dbgfs_deinit(gdev->dbg_root);
>>> +	kfree(gdev->ei);
>>> +	kfree(gdev);
>>> +
>>> +	dev_dbg(chip->dev, "Removed hte chip\n");
>>> +	return 0;
>>> +}
>>> +EXPORT_SYMBOL_GPL(hte_unregister_chip);
>>> +
>>> +/* Driver APIs ends */  
>> Don't bother with file layout type comments.  They don't add that much and tend
>> to rot horribly over time as people move code around in files.
>>
>>> diff --git a/include/linux/hte.h b/include/linux/hte.h
>>> new file mode 100644
>>> index 000000000000..e1737579d4c4
>>> --- /dev/null
>>> +++ b/include/linux/hte.h
>>> @@ -0,0 +1,278 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * Copyright (c) 2021 NVIDIA Corporation
>>> + *
>>> + * Author: Dipen Patel <dipenp@nvidia.com>
>>> + */
>>> +
>>> +#ifndef __LINUX_HTE_H
>>> +#define __LINUX_HTE_H
>>> +
>>> +struct hte_chip;
>>> +struct hte_device;
>>> +struct of_phandle_args;
>>> +
>>> +/**
>>> + * Used by providers to indicate the direction of the timestamp.
>>> + */
>>> +#define HTE_EVENT_RISING_EDGE          0x1
>>> +#define HTE_EVENT_FALLING_EDGE         0x2  
>> Use an enum rather than a define for this as it's a value that can take a
>> set of distinct values.  Also, provide a name for 'I've no idea' which
>> I'm guessing is 0 currently.
>>
>>> +
>>> +/**
>>> + * struct hte_ts_data - HTE timestamp data.
>>> + * The provider uses and fills timestamp related details during push_timestamp
>>> + * API call. The consumer uses during retrieve_timestamp API call.
>>> + *
>>> + * @tsc: Timestamp value.
>>> + * @seq: Sequence counter of the timestamps.
>>> + * @dir: Direction of the event at the time of timestamp.
>>> + */
>>> +struct hte_ts_data {
>>> +	u64 tsc;
>>> +	u64 seq;
>>> +	int dir;
>>> +};
>>> +
>>> +/**
>>> + * struct hte_clk_info - Clock source info that HTE provider uses.
>>> + * The provider uses hardware clock as a source to timestamp real time. This
>>> + * structure presents the clock information to consumers. 
>>> + *
>>> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
>>> + * @type: Clock type. CLOCK_* types.  
>> So this is something we got a it wrong in IIO. It's much better to define
>> a subset of clocks that can be potentially used.  There are some that make
>> absolutely no sense and consumers really don't want to have to deal with them.
>>  
>>> + */
>>> +struct hte_clk_info {
>>> +	u64 hz;
>>> +	clockid_t type;
>>> +};
>>> +
>>> +/**
>>> + * HTE subsystem notifications for the consumers.
>>> + *
>>> + * @HTE_TS_AVAIL: Timestamps available notification.
>>> + * @HTE_TS_DROPPED: Timestamps dropped notification.  
>> Something I've missed so far is whether drops are in a kfifo or a ring
>> fashion.  I'm guess that's stated somewhere, but it might be useful to have
>> it here.
>>
>>> + */
>>> +enum hte_notify {
>>> +	HTE_TS_AVAIL = 1,
>>> +	HTE_TS_DROPPED,
>>> +	HTE_NUM_NOTIFIER,
>>> +};
>>> +
>>> +/**
>>> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
>>> + * communication token between consumers to subsystem and subsystem to
>>> + * providers.
>>> + *
>>> + * @con_id: This is the same id sent in request APIs.
>>> + * @name: Descriptive name of the entity that is being monitored for the
>>> + * realtime timestamping.
>>> + * @data_subsys: Subsystem's private data relate to requested con_id.
>>> + */
>>> +struct hte_ts_desc {
>>> +	u32 con_id;
>>> +	char *name;
>>> +	void *data_subsys;
>>> +};
>>> +
>>> +/**
>>> + * struct hte_ops - HTE operations set by providers.
>>> + *
>>> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
>>> + * non-zero for failures.
>>> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
>>> + * non-zero for failures.
>>> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
>>> + * non-zero for failures.
>>> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
>>> + * non-zero for failures.
>>> + * @get_clk_src_info: Optional hook to get the clock information provider uses
>>> + * to timestamp. Returns 0 for success and negative error code for failure. On
>>> + * success HTE subsystem fills up provided struct hte_clk_info.  
>> Why optional?  Consumers will probably need that information.
>>
>>> + *
>>> + * xlated_id parameter is used to communicate between HTE subsystem and the
>>> + * providers. It is the same id returned during xlate API call and translated
>>> + * by the provider. This may be helpful as both subsystem and provider locate
>>> + * the requested entity in constant time, where entity could be anything from
>>> + * lines, signals, events, buses etc.. that providers support.
>>> + */
>>> +struct hte_ops {
>>> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
>>> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
>>> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
>>> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
>>> +	int (*get_clk_src_info)(struct hte_chip *chip,
>>> +				struct hte_clk_info *ci);
>>> +};
>>> +
>>> +/**
>>> + * struct hte_chip - Abstract HTE chip structure.
>>> + * @name: functional name of the HTE IP block.
>>> + * @dev: device providing the HTE.  
>> Unclear naming.  Is this the parent device, or one associated with the HTE itself?
>> I'm guessing today you don't have one associated with the HTE, but it is plausible you
>> might gain on in future to make it fit nicely in the device model as a function of another
>> device.
>>
>>> + * @ops: callbacks for this HTE.
>>> + * @nlines: number of lines/signals supported by this chip.
>>> + * @xlate: Callback which translates consumer supplied logical ids to
>>> + * physical ids, return from 0 for the success and negative for the
>>> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
>>> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
>>> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
>>> + * @data: chip specific private data.
>>> + */
>>> +struct hte_chip {
>>> +	const char *name;
>>> +	struct device *dev;
>>> +	const struct hte_ops *ops;
>>> +	u32 nlines;
>>> +	int (*xlate)(struct hte_chip *gc,
>>> +		     const struct of_phandle_args *args,
>>> +		     struct hte_ts_desc *desc, u32 *xlated_id);
>>> +	u8 of_hte_n_cells;
>>> +
>>> +	/* only used internally by the HTE framework */
>>> +	struct hte_device *gdev;
>>> +	void *data;
>>> +};
>>> +
>>> +#if IS_ENABLED(CONFIG_HTE)
>>> +/* HTE APIs for the providers */
>>> +int hte_register_chip(struct hte_chip *chip);
>>> +int hte_unregister_chip(struct hte_chip *chip);
>>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
>>> +			  struct hte_ts_data *data, size_t n);
>>> +
>>> +/* HTE APIs for the consumers */
>>> +
>>> +int hte_release_ts(struct hte_ts_desc *desc);
>>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
>>> +				      void (*cb)(enum hte_notify n));
>>> +
>>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
>>> +					   const char *label,
>>> +					   void (*cb)(enum hte_notify n));
>>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
>>> +					  unsigned int id,
>>> +					  void (*cb)(enum hte_notify n));
>>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
>>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
>>> +		       size_t n);
>>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
>>> +			    struct hte_ts_data *el, size_t n);
>>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
>>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc);
>>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
>>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
>>> +size_t hte_available_ts(const struct hte_ts_desc *desc);
>>> +int hte_enable_ts(struct hte_ts_desc *desc);
>>> +int hte_disable_ts(struct hte_ts_desc *desc);
>>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>>> +			 struct hte_clk_info *ci);
>>> +
>>>  

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-01 14:21   ` Kent Gibson
@ 2021-07-28 23:59     ` Dipen Patel
  2021-07-30  7:01       ` Dipen Patel
  2021-07-31 15:43       ` Kent Gibson
  0 siblings, 2 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-28 23:59 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

Thanks Kent for the review comment. My responses inline.

On 7/1/21 7:21 AM, Kent Gibson wrote:
> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>> Tegra194 device has multiple HTE instances also known as GTE
>> (Generic hardware Timestamping Engine) which can timestamp subset of
>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>> and exposes timestamping ability on those lines to the consumers
>> through HTE subsystem.
>>
>> Also, with this patch, added:
>> - documentation about this provider and its capabilities at
>> Documentation/hte.
>> - Compilation support in Makefile and Kconfig
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  Documentation/hte/index.rst        |  21 ++
>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>  Documentation/index.rst            |   1 +
>>  drivers/hte/Kconfig                |  12 +
>>  drivers/hte/Makefile               |   1 +
>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>  6 files changed, 654 insertions(+)
>>  create mode 100644 Documentation/hte/index.rst
>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>  create mode 100644 drivers/hte/hte-tegra194.c
>>
>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>> new file mode 100644
>> index 000000000000..f311ebec6b47
>> --- /dev/null
>> +++ b/Documentation/hte/index.rst
>> @@ -0,0 +1,21 @@
>> +.. SPDX-License-Identifier: GPL-2.0
>> +
>> +============================================
>> +The Linux Hardware Timestamping Engine (HTE)
>> +============================================
>> +
>> +The HTE Subsystem
>> +=================
>> +
>> +.. toctree::
>> +   :maxdepth: 1
>> +
>> +   hte
>> +
>> +HTE Tegra Provider
>> +==================
>> +
>> +.. toctree::
>> +   :maxdepth: 1
>> +
>> +   tegra194-hte
>> \ No newline at end of file
>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>> new file mode 100644
>> index 000000000000..c23eaafcf080
>> --- /dev/null
>> +++ b/Documentation/hte/tegra194-hte.rst
>> @@ -0,0 +1,65 @@
>> +HTE Kernel provider driver
>> +==========================
>> +
>> +Description
>> +-----------
>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>> +known as generic timestamping engine (GTE). This provider driver implements
>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>> +value.
>> +
>> +GPIO GTE
>> +--------
>> +
>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>> +controller as it requires very specific bits to be set in GPIO config register.
>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>> +subsystem and GPIO GTE for in kernel consumers.
>> +
>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>> +
>> +	To enable HTE on given GPIO line.
>> +
>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>> +
>> +	To retrieve hardwre timestamp in nano seconds.
>> +
>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>> +
>> +	To query if HTE is enabled on the given GPIO.
>> +
>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>> +in nano second.
>> +
> <snip>
>
>> +
>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>> +{
>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>> +	u64 tsc;
>> +	int dir;
>> +	struct hte_ts_data el;
>> +
>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>> +		tsc = (((u64)tsh << 32) | tsl);
>> +
>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>> +
>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>> +		acv = pv ^ cv;
>> +		while (acv) {
>> +			bit_index = __builtin_ctz(acv);
>> +			if ((pv >> bit_index) & BIT(0))
>> +				dir = HTE_EVENT_RISING_EDGE;
>> +			else
>> +				dir = HTE_EVENT_FALLING_EDGE;
>> +
>> +			line_id = bit_index + (slice << 5);
>> +			el.dir = dir;
>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>> +					      sizeof(el));
>> +			acv &= ~BIT(bit_index);
>> +		}
>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>> +	}
>> +}
> What happens when the hte_push_ts_ns_atomic() fails?
> The timestamp will be quietly dropped?
> What happens when the interrupt corresponding to that dropped timestamp
> asks for it?  The irq handler thread will block until it can get a
> timestamp from the subsequent interrupt?

Two things happen, 1) at the push, HTE core increments seq counter

2) If the consumer has provided callback, it will either call that callback

with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect

view of dropped ts. However, I see the problem with the consumers not

providing callback, in that case, push_ts* API just wakes up process without

indicating why (assuming notify variable is true or else there is a chance for

the thread to block forever). One easy approach I can think of for now is to

make callback mandatory (which is optional right now), I will have to rethink

that scenario and will push corrected version next RFC version.

Thanks for pointing out.

>
> Which brings me back to the concern I have with the approach used in
> the hte/gpiolib integration - how do you guarantee that the timestamp
> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
> being handled, particularly in the face of errors such as:
>  - overflows of the timestamp FIFO in the chip

I currently do not have any indication mechanism as the providers

I am dealing with right now does not have overflow hardware detection

support. If the chip supports, it should be easy to integrate that feature.

I will provide some hook function or change in push_* API to accommodate

this in next version of RFC.

>  - overflows of software FIFOs as here

HTE core records sequence counter as well it callsback the consumer with

HTE_TS_DROPPED.

>  - lost interupts (if the hw generates interrupts faster than the CPU
>    can service them)

For this, I have no idea unless hardware supports some sort of mechanism

to catch that. For the current providers, as soon as it detects changes on lines

it captures TS in its hw fifo. Its interrupt gets generated based on threshold

set in that hw fifo. This interrupt is different than the lines of actual device

that is why I said I have no idea how we can tackle that. Let me know if there

is any idea or reference of the codes which does tackle this.


Regarding HTE/GPIOLIB integration comment:

You are right, currently, I have only tsc field returned from struct hte_ts_data

to gpiolib. If I can extend that to return hte_ts_data structure which has seq

counter, which I believe can be used to track the overflow situation. The

dropped scenario can be easily tracked if gpiolib can be notified with above

mentioned DROP event through callback. If that is the case, is it ok to have

some sort of callback per gpio in gpiolib?


Any idea how I can integrate callback notification with gpiolib if you do not agree on

above callback suggestion?

> ?
>
> Cheers,
> Kent.
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-04 20:27   ` Jonathan Cameron
@ 2021-07-29  2:42     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-29  2:42 UTC (permalink / raw)
  To: Jonathan Cameron
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

Thanks Jonathan, I accept all your comments, will be reflected in next RFC patch series.

Thanks for taking time to review and comment.


Best Regards,

Dipen Patel

On 7/4/21 1:27 PM, Jonathan Cameron wrote:
> On Fri, 25 Jun 2021 16:55:24 -0700
> Dipen Patel <dipenp@nvidia.com> wrote:
>
>> Tegra194 device has multiple HTE instances also known as GTE
>> (Generic hardware Timestamping Engine) which can timestamp subset of
>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>> and exposes timestamping ability on those lines to the consumers
>> through HTE subsystem.
>>
>> Also, with this patch, added:
>> - documentation about this provider and its capabilities at
>> Documentation/hte.
>> - Compilation support in Makefile and Kconfig
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> A few comments inline,
>
> J
>> ---
>>  Documentation/hte/index.rst        |  21 ++
>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>  Documentation/index.rst            |   1 +
>>  drivers/hte/Kconfig                |  12 +
>>  drivers/hte/Makefile               |   1 +
>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>  6 files changed, 654 insertions(+)
>>  create mode 100644 Documentation/hte/index.rst
>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>  create mode 100644 drivers/hte/hte-tegra194.c
>>
>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>> new file mode 100644
>> index 000000000000..f311ebec6b47
>> --- /dev/null
>> +++ b/Documentation/hte/index.rst
>> @@ -0,0 +1,21 @@
>> +.. SPDX-License-Identifier: GPL-2.0
>> +
>> +============================================
>> +The Linux Hardware Timestamping Engine (HTE)
>> +============================================
>> +
>> +The HTE Subsystem
>> +=================
>> +
>> +.. toctree::
>> +   :maxdepth: 1
>> +
>> +   hte
>> +
>> +HTE Tegra Provider
>> +==================
>> +
>> +.. toctree::
>> +   :maxdepth: 1
>> +
>> +   tegra194-hte
>> \ No newline at end of file
>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>> new file mode 100644
>> index 000000000000..c23eaafcf080
>> --- /dev/null
>> +++ b/Documentation/hte/tegra194-hte.rst
>> @@ -0,0 +1,65 @@
>> +HTE Kernel provider driver
>> +==========================
>> +
>> +Description
>> +-----------
>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>> +known as generic timestamping engine (GTE). This provider driver implements
>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>> +value.
>> +
>> +GPIO GTE
>> +--------
>> +
>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>> +controller as it requires very specific bits to be set in GPIO config register.
>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>> +subsystem and GPIO GTE for in kernel consumers.
>> +
>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>> +
>> +	To enable HTE on given GPIO line.
>> +
>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>> +
>> +	To retrieve hardwre timestamp in nano seconds.
>> +
>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>> +
>> +	To query if HTE is enabled on the given GPIO.
>> +
>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>> +in nano second.
>> +
>> +IRQ GTE
>> +--------
>> +
>> +This GTE instance helps timestamp LIC IRQ lines in real time. There are 352 IRQ
>> +lines which this instance can help timestamp realtime. The hte devicetree
>> +binding described at ``Documentation/devicetree/bindings/hte/`` gives out
>> +example how consumer can request IRQ line, since it is one to one mapping,
>> +consumers can simply specify IRQ number that they are interested in. There is
>> +no userspace consumer support for this GTE instance. The sample test code
>> +hte-tegra194-irq-test.c, located in ``drivers/hte/`` directory,
>> +demonstrates how to use IRQ GTE instance. The below is sample device tree
>> +snippet code for the test driver::
>> +
>> + tegra_hte_irq_test {
>> +        compatible = "nvidia,tegra194-hte-irq-test";
>> +        htes = <&tegra_hte_lic 0x19>;
>> +        hte-names = "hte-lic";
>> + };
>> +
>> +The source code of the driver both IRQ and GPIO GTE is locate at
>> +``drivers/hte/hte-tegra194.c``.
>> \ No newline at end of file
>> diff --git a/Documentation/index.rst b/Documentation/index.rst
>> index 1b13c2445e87..b41118577fe6 100644
>> --- a/Documentation/index.rst
>> +++ b/Documentation/index.rst
>> @@ -138,6 +138,7 @@ needed).
>>     misc-devices/index
>>     scheduler/index
>>     mhi/index
>> +   hte/index
>>  
>>  Architecture-agnostic documentation
>>  -----------------------------------
>> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
>> index 394e112f7dfb..f7b01fcc7190 100644
>> --- a/drivers/hte/Kconfig
>> +++ b/drivers/hte/Kconfig
>> @@ -20,3 +20,15 @@ menuconfig HTE
>>  
>>            If unsure, say no.
>>  
>> +if HTE
>> +
>> +config HTE_TEGRA194
>> +	tristate "NVIDIA Tegra194 HTE Support"
>> +	depends on ARCH_TEGRA_194_SOC
>> +	help
>> +	  Enable this option for integrated hardware timestamping engine also
>> +	  known as generic timestamping engine (GTE) support on NVIDIA Tegra194
>> +	  systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
>> +	  lines for timestamping in realtime.
>> +
>> +endif
>> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
>> index 9899dbe516f7..52f978cfc913 100644
>> --- a/drivers/hte/Makefile
>> +++ b/drivers/hte/Makefile
>> @@ -1 +1,2 @@
>>  obj-$(CONFIG_HTE)		+= hte.o
>> +obj-$(CONFIG_HTE_TEGRA194)	+= hte-tegra194.o
>> \ No newline at end of file
> fix that
>
>> diff --git a/drivers/hte/hte-tegra194.c b/drivers/hte/hte-tegra194.c
>> new file mode 100644
>> index 000000000000..8ad10efd3641
>> --- /dev/null
>> +++ b/drivers/hte/hte-tegra194.c
>> @@ -0,0 +1,554 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2021 NVIDIA Corporation
>> + *
>> + * Author: Dipen Patel <dipenp@nvidia.com>
>> + */
>> +
>> +#include <linux/err.h>
>> +#include <linux/io.h>
>> +#include <linux/module.h>
>> +#include <linux/slab.h>
>> +#include <linux/stat.h>
>> +#include <linux/interrupt.h>
>> +#include <linux/of.h>
>> +#include <linux/of_device.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/hte.h>
>> +#include <linux/uaccess.h>
>> +
>> +#define HTE_SUSPEND	0
>> +
>> +/* HTE source clock TSC is 31.25MHz */
>> +#define HTE_TS_CLK_RATE_HZ	31250000ULL
>> +#define HTE_CLK_RATE_NS		32
>> +#define HTE_TS_NS_SHIFT	__builtin_ctz(HTE_CLK_RATE_NS)
>> +
>> +#define NV_AON_SLICE_INVALID	-1
>> +
>> +/* AON HTE line map For slice 1 */
>> +#define NV_AON_HTE_SLICE1_IRQ_GPIO_28	12
>> +#define NV_AON_HTE_SLICE1_IRQ_GPIO_29	13
>> +
>> +/* AON HTE line map For slice 2 */
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_0	0
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_1	1
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_2	2
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_3	3
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_4	4
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_5	5
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_6	6
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_7	7
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_8	8
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_9	9
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_10	10
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_11	11
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_12	12
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_13	13
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_14	14
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_15	15
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_16	16
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_17	17
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_18	18
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_19	19
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_20	20
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_21	21
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_22	22
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_23	23
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_24	24
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_25	25
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_26	26
>> +#define NV_AON_HTE_SLICE2_IRQ_GPIO_27	27
>> +
>> +/* AON GPIO port AA pins */
>> +#define NV_AON_GPIO_PORT_AA_0		0
>> +#define NV_AON_GPIO_PORT_AA_1		1
>> +#define NV_AON_GPIO_PORT_AA_2		2
>> +#define NV_AON_GPIO_PORT_AA_3		3
>> +#define NV_AON_GPIO_PORT_AA_4		4
>> +#define NV_AON_GPIO_PORT_AA_5		5
>> +#define NV_AON_GPIO_PORT_AA_6		6
>> +#define NV_AON_GPIO_PORT_AA_7		7
>> +
>> +/* AON GPIO port BB pins */
>> +#define NV_AON_GPIO_PORT_BB_0		8
>> +#define NV_AON_GPIO_PORT_BB_1		9
>> +#define NV_AON_GPIO_PORT_BB_2		10
>> +#define NV_AON_GPIO_PORT_BB_3		11
>> +
>> +/* AON GPIO port CC pins */
>> +#define NV_AON_GPIO_PORT_CC_0		16
>> +#define NV_AON_GPIO_PORT_CC_1		17
>> +#define NV_AON_GPIO_PORT_CC_2		18
>> +#define NV_AON_GPIO_PORT_CC_3		19
>> +#define NV_AON_GPIO_PORT_CC_4		20
>> +#define NV_AON_GPIO_PORT_CC_5		21
>> +#define NV_AON_GPIO_PORT_CC_6		22
>> +#define NV_AON_GPIO_PORT_CC_7		23
>> +
>> +/* AON GPIO port DD pins */
>> +#define NV_AON_GPIO_PORT_DD_0		24
>> +#define NV_AON_GPIO_PORT_DD_1		25
>> +#define NV_AON_GPIO_PORT_DD_2		26
>> +
>> +/* AON GPIO port EE pins */
>> +#define NV_AON_GPIO_PORT_EE_0		32
>> +#define NV_AON_GPIO_PORT_EE_1		33
>> +#define NV_AON_GPIO_PORT_EE_2		34
>> +#define NV_AON_GPIO_PORT_EE_3		35
>> +#define NV_AON_GPIO_PORT_EE_4		36
>> +#define NV_AON_GPIO_PORT_EE_5		37
>> +#define NV_AON_GPIO_PORT_EE_6		38
>> +
>> +
>> +#define HTE_TECTRL		0x0
>> +#define HTE_TETSCH		0x4
>> +#define HTE_TETSCL		0x8
>> +#define HTE_TESRC		0xC
>> +#define HTE_TECCV		0x10
>> +#define HTE_TEPCV		0x14
>> +#define HTE_TECMD		0x1C
>> +#define HTE_TESTATUS		0x20
>> +#define HTE_SLICE0_TETEN	0x40
>> +#define HTE_SLICE1_TETEN	0x60
>> +
>> +#define HTE_SLICE_SIZE		(HTE_SLICE1_TETEN - HTE_SLICE0_TETEN)
>> +
>> +#define HTE_TECTRL_ENABLE_ENABLE	0x1
>> +
>> +#define HTE_TECTRL_OCCU_SHIFT		0x8
>> +#define HTE_TECTRL_INTR_SHIFT		0x1
>> +#define HTE_TECTRL_INTR_ENABLE		0x1
>> +
>> +#define HTE_TESRC_SLICE_SHIFT		16
>> +#define HTE_TESRC_SLICE_DEFAULT_MASK	0xFF
>> +
>> +#define HTE_TECMD_CMD_POP		0x1
>> +
>> +#define HTE_TESTATUS_OCCUPANCY_SHIFT	8
>> +#define HTE_TESTATUS_OCCUPANCY_MASK	0xFF
>> +
>> +struct hte_slices {
>> +	u32 r_val;
>> +	unsigned long flags;
>> +	/* to prevent lines mapped to same slice updating its register */
>> +	spinlock_t s_lock;
>> +};
>> +
>> +struct tegra_hte_line_mapped {
>> +	int slice;
>> +	u32 bit_index;
>> +};
>> +
>> +struct tegra_hte_line_table {
>> +	int map_sz;
>> +	const struct tegra_hte_line_mapped *map;
>> +};
>> +
>> +struct tegra_hte_soc {
>> +	int hte_irq;
>> +	u32 itr_thrshld;
>> +	u32 conf_rval;
>> +	struct hte_slices *sl;
>> +	const struct tegra_hte_line_table *line_map;
>> +	struct hte_chip *chip;
>> +	void __iomem *regs;
>> +};
>> +
>> +static const struct tegra_hte_line_mapped tegra194_aon_gpio_map[] = {
>> +	/* gpio, slice, bit_index */
>> +	[NV_AON_GPIO_PORT_AA_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_11},
>> +	[NV_AON_GPIO_PORT_AA_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_10},
>> +	[NV_AON_GPIO_PORT_AA_2]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_9},
>> +	[NV_AON_GPIO_PORT_AA_3]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_8},
>> +	[NV_AON_GPIO_PORT_AA_4]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_7},
>> +	[NV_AON_GPIO_PORT_AA_5]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_6},
>> +	[NV_AON_GPIO_PORT_AA_6]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_5},
>> +	[NV_AON_GPIO_PORT_AA_7]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_4},
>> +	[NV_AON_GPIO_PORT_BB_0]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_3},
>> +	[NV_AON_GPIO_PORT_BB_1]  = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_2},
>> +	[NV_AON_GPIO_PORT_BB_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_1},
>> +	[NV_AON_GPIO_PORT_BB_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_0},
>> +	[12] = {NV_AON_SLICE_INVALID, 0},
>> +	[13] = {NV_AON_SLICE_INVALID, 0},
>> +	[14] = {NV_AON_SLICE_INVALID, 0},
>> +	[15] = {NV_AON_SLICE_INVALID, 0},
>> +	[NV_AON_GPIO_PORT_CC_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_22},
>> +	[NV_AON_GPIO_PORT_CC_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_21},
>> +	[NV_AON_GPIO_PORT_CC_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_20},
>> +	[NV_AON_GPIO_PORT_CC_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_19},
>> +	[NV_AON_GPIO_PORT_CC_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_18},
>> +	[NV_AON_GPIO_PORT_CC_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_17},
>> +	[NV_AON_GPIO_PORT_CC_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_16},
>> +	[NV_AON_GPIO_PORT_CC_7] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_15},
>> +	[NV_AON_GPIO_PORT_DD_0] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_14},
>> +	[NV_AON_GPIO_PORT_DD_1] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_13},
>> +	[NV_AON_GPIO_PORT_DD_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_12},
>> +	[27] = {NV_AON_SLICE_INVALID, 0},
>> +	[28] = {NV_AON_SLICE_INVALID, 0},
>> +	[29] = {NV_AON_SLICE_INVALID, 0},
>> +	[30] = {NV_AON_SLICE_INVALID, 0},
>> +	[31] = {NV_AON_SLICE_INVALID, 0},
>> +	[NV_AON_GPIO_PORT_EE_0] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_29},
>> +	[NV_AON_GPIO_PORT_EE_1] = {1, NV_AON_HTE_SLICE1_IRQ_GPIO_28},
>> +	[NV_AON_GPIO_PORT_EE_2] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_27},
>> +	[NV_AON_GPIO_PORT_EE_3] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_26},
>> +	[NV_AON_GPIO_PORT_EE_4] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_25},
>> +	[NV_AON_GPIO_PORT_EE_5] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_24},
>> +	[NV_AON_GPIO_PORT_EE_6] = {2, NV_AON_HTE_SLICE2_IRQ_GPIO_23},
>> +};
>> +
>> +static const struct tegra_hte_line_table aon_hte_map = {
>> +	.map_sz = ARRAY_SIZE(tegra194_aon_gpio_map),
>> +	.map = tegra194_aon_gpio_map,
>> +};
>> +
>> +static inline u32 tegra_hte_readl(struct tegra_hte_soc *hte, u32 reg)
>> +{
>> +	return readl(hte->regs + reg);
>> +}
>> +
>> +static inline void tegra_hte_writel(struct tegra_hte_soc *hte, u32 reg,
>> +				    u32 val)
>> +{
>> +	writel(val, hte->regs + reg);
>> +}
>> +
>> +static inline int tegra_hte_map_to_line_id(u32 eid, struct tegra_hte_soc *gs,
>> +					  u32 *mapped)
>> +{
>> +	const struct tegra_hte_line_mapped *m;
>> +
>> +	if (gs->line_map) {
>> +		m = gs->line_map->map;
>> +		if (eid > gs->line_map->map_sz)
>> +			return -EINVAL;
>> +		if (m[eid].slice == NV_AON_SLICE_INVALID)
>> +			return -EINVAL;
>> +
>> +		*mapped = (m[eid].slice << 5) + m[eid].bit_index;
>> +	} else {
>> +		*mapped = eid;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int tegra_hte_line_xlate(struct hte_chip *gc,
>> +				 const struct of_phandle_args *args,
>> +				 struct hte_ts_desc *desc, u32 *xlated_id)
>> +{
>> +	int ret = 0;
>> +
>> +	if (!gc || !desc || !xlated_id)
>> +		return -EINVAL;
>> +
>> +	if (args) {
>> +		if (gc->of_hte_n_cells < 1)
>> +			return -EINVAL;
>> +
>> +		if (args->args_count != gc->of_hte_n_cells)
>> +			return -EINVAL;
>> +
>> +		desc->con_id = args->args[0];
>> +	}
>> +
>> +	ret = tegra_hte_map_to_line_id(desc->con_id, gc->data,
>> +				       xlated_id);
>> +	if (ret < 0) {
>> +		dev_dbg(gc->dev, "con_id:%u mapping failed\n",
>> +			desc->con_id);
>> +		return ret;
>> +	}
>> +
>> +	if (*xlated_id > gc->nlines)
>> +		return -EINVAL;
>> +
>> +	dev_dbg(gc->dev, "requested id:%u, xlated id:%u\n",
>> +		desc->con_id, *xlated_id);
>> +
>> +	return 0;
>> +}
>> +
>> +static int tegra_hte_en_dis_common(struct hte_chip *chip, u32 line_id, bool en)
>> +{
>> +	u32 slice, sl_bit_shift, line_bit, val, reg;
>> +	struct tegra_hte_soc *gs;
>> +
>> +	sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
>> +
>> +	if (!chip)
>> +		return -EINVAL;
>> +
>> +	gs = (struct tegra_hte_soc *)chip->data;
>> +
>> +	if (line_id > chip->nlines) {
>> +		dev_err(chip->dev,
>> +			"line id: %u is not supported by this controller\n",
>> +			line_id);
>> +		return -EINVAL;
>> +	}
>> +
>> +	slice = line_id >> sl_bit_shift;
>> +	line_bit = line_id & (HTE_SLICE_SIZE - 1);
>> +	reg = (slice << sl_bit_shift) + HTE_SLICE0_TETEN;
>> +
>> +	spin_lock(&gs->sl[slice].s_lock);
>> +
>> +	if (test_bit(HTE_SUSPEND, &gs->sl[slice].flags)) {
>> +		spin_unlock(&gs->sl[slice].s_lock);
>> +		dev_dbg(chip->dev, "device suspended");
>> +		return -EBUSY;
>> +	}
>> +
>> +	val = tegra_hte_readl(gs, reg);
>> +	if (en)
>> +		val = val | (1 << line_bit);
>> +	else
>> +		val = val & (~(1 << line_bit));
>> +	tegra_hte_writel(gs, reg, val);
>> +
>> +	spin_unlock(&gs->sl[slice].s_lock);
>> +
>> +	dev_dbg(chip->dev, "line: %u, slice %u, line_bit %u, reg:0x%x\n",
>> +		line_id, slice, line_bit, reg);
>> +
>> +	return 0;
>> +}
>> +
>> +static int tegra_hte_request(struct hte_chip *chip, u32 line_id)
>> +{
>> +	return tegra_hte_en_dis_common(chip, line_id, true);
>> +}
>> +
>> +static int tegra_hte_release(struct hte_chip *chip, u32 line_id)
>> +{
>> +	return tegra_hte_en_dis_common(chip, line_id, false);
>> +}
>> +
>> +static int tegra_hte_clk_src_info(struct hte_chip *chip,
>> +				  struct hte_clk_info *ci)
>> +{
>> +	(void)chip;
>> +
>> +	ci->hz = HTE_TS_CLK_RATE_HZ;
>> +	ci->type = CLOCK_MONOTONIC;
>> +
>> +	return 0;
>> +}
>> +
>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>> +{
>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>> +	u64 tsc;
>> +	int dir;
>> +	struct hte_ts_data el;
>> +
>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>> +		tsc = (((u64)tsh << 32) | tsl);
>> +
>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>> +
>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>> +		acv = pv ^ cv;
>> +		while (acv) {
>> +			bit_index = __builtin_ctz(acv);
>> +			if ((pv >> bit_index) & BIT(0))
>> +				dir = HTE_EVENT_RISING_EDGE;
>> +			else
>> +				dir = HTE_EVENT_FALLING_EDGE;
>> +
>> +			line_id = bit_index + (slice << 5);
>> +			el.dir = dir;
>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>> +					      sizeof(el));
>> +			acv &= ~BIT(bit_index);
>> +		}
>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>> +	}
>> +}
>> +
>> +static irqreturn_t tegra_hte_isr(int irq, void *dev_id)
>> +{
>> +	struct tegra_hte_soc *gs = dev_id;
>> +
>> +	tegra_hte_read_fifo(gs);
>> +
>> +	return IRQ_HANDLED;
>> +}
>> +
>> +static const struct of_device_id tegra_hte_of_match[] = {
>> +	{ .compatible = "nvidia,tegra194-gte-lic"},
>> +	{ .compatible = "nvidia,tegra194-gte-aon", .data = &aon_hte_map},
>> +	{ }
>> +};
>> +MODULE_DEVICE_TABLE(of, tegra_hte_of_match);
>> +
>> +static const struct hte_ops g_ops = {
>> +	.request = tegra_hte_request,
>> +	.release = tegra_hte_release,
>> +	.enable = tegra_hte_request,
>> +	.disable = tegra_hte_release,
>> +	.get_clk_src_info = tegra_hte_clk_src_info,
>> +};
>> +
>> +static int tegra_hte_probe(struct platform_device *pdev)
>> +{
>> +	int ret;
>> +	u32 i, slices, val = 0;
>> +	struct device *dev;
>> +	struct tegra_hte_soc *hte_dev;
>> +	struct hte_chip *gc;
>> +
>> +	dev = &pdev->dev;
>> +
>> +	hte_dev = devm_kzalloc(dev, sizeof(*hte_dev), GFP_KERNEL);
>> +	if (!hte_dev)
>> +		return -ENOMEM;
>> +
>> +	gc = devm_kzalloc(dev, sizeof(*gc), GFP_KERNEL);
>> +	if (!gc)
>> +		return -ENOMEM;
>> +
>> +	dev_set_drvdata(&pdev->dev, hte_dev);
>> +	hte_dev->line_map = of_device_get_match_data(&pdev->dev);
>> +
>> +	hte_dev->regs = devm_platform_ioremap_resource(pdev, 0);
>> +	if (IS_ERR(hte_dev->regs))
>> +		return PTR_ERR(hte_dev->regs);
>> +
>> +	ret = of_property_read_u32(dev->of_node, "int-threshold",
>> +				   &hte_dev->itr_thrshld);
>> +	if (ret != 0)
>> +		hte_dev->itr_thrshld = 1;
>> +
>> +	ret = of_property_read_u32(dev->of_node, "slices", &slices);
>> +	if (ret != 0) {
>> +		dev_err(dev, "Could not read slices\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	hte_dev->sl = devm_kzalloc(dev, sizeof(struct hte_slices) * slices,
> Preference for sizeof(*hte_dev->sl) as it saves me checking the size is correct
> for the type.
>
>> +				   GFP_KERNEL);
>> +	if (!hte_dev->sl)
>> +		return -ENOMEM;
>> +
>> +	ret = platform_get_irq(pdev, 0);
>> +	if (ret < 0) {
>> +		dev_err(dev, "get irq failed.\n");
> dev_err_probe() probably so you don't print the message if deferred probing is
> going on.
>
>> +		return ret;
>> +	}
>> +	hte_dev->hte_irq = ret;
>> +	ret = devm_request_irq(dev, hte_dev->hte_irq, tegra_hte_isr, 0,
>> +			       dev_name(dev), hte_dev);
>> +	if (ret < 0) {
>> +		dev_err(dev, "request irq failed.\n");
>> +		return ret;
>> +	}
>> +
>> +	gc->nlines = slices << 5;
>> +	gc->ops = &g_ops;
>> +	gc->dev = dev;
>> +	hte_dev->chip = gc;
>> +	gc->data = (void *)hte_dev;
> Don't case to void * - cast to the actual type if necessary.
> If it is a void * then most likely it shouldn't be if we always put something
> in particular it in it.
>
>> +	gc->xlate = tegra_hte_line_xlate;
>> +	gc->of_hte_n_cells = 1;
>> +
>> +	ret = hte_register_chip(hte_dev->chip);
>> +
> No blank line before error handler.  Under the circumstances is that not
> fatal?
>
>> +	if (ret)
>> +		dev_err(gc->dev, "hte chip register failed");
>> +
>> +	for (i = 0; i < slices; i++) {
>> +		hte_dev->sl[i].flags = 0;
>> +		spin_lock_init(&hte_dev->sl[i].s_lock);
>> +	}
>> +
>> +	val = HTE_TECTRL_ENABLE_ENABLE |
>> +	      (HTE_TECTRL_INTR_ENABLE << HTE_TECTRL_INTR_SHIFT) |
>> +	      (hte_dev->itr_thrshld << HTE_TECTRL_OCCU_SHIFT);
>> +	tegra_hte_writel(hte_dev, HTE_TECTRL, val);
> You could use a devm_add_action_or_reset() to deal with unwinding this
> plus add a devm_hte_register_chip() and then you can get rid of remove
> entirely which is always nice.
>
>> +
>> +	dev_dbg(gc->dev, "lines: %d, slices:%d", gc->nlines, slices);
>> +	return 0;
>> +}
>> +
>> +static int tegra_hte_remove(struct platform_device *pdev)
>> +{
>> +	struct tegra_hte_soc *gs = dev_get_drvdata(&pdev->dev);
>> +
>> +	tegra_hte_writel(gs, HTE_TECTRL, 0);
>> +
>> +	return hte_unregister_chip(gs->chip);
>> +}
>> +
>> +#ifdef CONFIG_PM_SLEEP
> Personally I prefer the approach of marking PM functions
> __maybe_unused and dropping the ifdef protections.
> There have been a lot of subtle issues in the build system in the
> past around those and it's much easier to just let the compiler
> drop them if they are unused.
>
>> +static int tegra_hte_resume_early(struct device *dev)
>> +{
>> +	u32 i;
>> +	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
>> +	u32 slices = gs->chip->nlines >> 5;
> Whilst it's the same thing, I'd prefer a divide there by however lines there are in a slice.
>
>> +	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
>> +
>> +	tegra_hte_writel(gs, HTE_TECTRL, gs->conf_rval);
>> +
>> +	for (i = 0; i < slices; i++) {
>> +		spin_lock(&gs->sl[i].s_lock);
>> +		tegra_hte_writel(gs,
>> +				 ((i << sl_bit_shift) + HTE_SLICE0_TETEN),
>> +				 gs->sl[i].r_val);
>> +		clear_bit(HTE_SUSPEND, &gs->sl[i].flags);
>> +		spin_unlock(&gs->sl[i].s_lock);
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int tegra_hte_suspend_late(struct device *dev)
>> +{
>> +	u32 i;
>> +	struct tegra_hte_soc *gs = dev_get_drvdata(dev);
>> +	u32 slices = gs->chip->nlines >> 5;
>> +	u32 sl_bit_shift = __builtin_ctz(HTE_SLICE_SIZE);
>> +
>> +	gs->conf_rval = tegra_hte_readl(gs, HTE_TECTRL);
>> +	for (i = 0; i < slices; i++) {
>> +		spin_lock(&gs->sl[i].s_lock);
>> +		gs->sl[i].r_val = tegra_hte_readl(gs,
>> +				((i << sl_bit_shift) + HTE_SLICE0_TETEN));
>> +		set_bit(HTE_SUSPEND, &gs->sl[i].flags);
>> +		spin_unlock(&gs->sl[i].s_lock);
>> +	}
>> +
>> +	return 0;
>> +}
>> +#endif
>> +
>> +static const struct dev_pm_ops tegra_hte_pm = {
>> +	SET_LATE_SYSTEM_SLEEP_PM_OPS(tegra_hte_suspend_late,
>> +				     tegra_hte_resume_early)
>> +};
>> +
>> +static struct platform_driver tegra_hte_driver = {
>> +	.probe = tegra_hte_probe,
>> +	.remove = tegra_hte_remove,
>> +	.driver = {
>> +		.name = "tegra_hte",
>> +		.pm = &tegra_hte_pm,
>> +		.of_match_table = tegra_hte_of_match,
>> +	},
>> +};
>> +
>> +module_platform_driver(tegra_hte_driver);
>> +
>> +MODULE_AUTHOR("Dipen Patel <dipenp@nvidia.com>");
>> +MODULE_DESCRIPTION("NVIDIA Tegra HTE (Hardware Timestamping Engine) driver");
>> +MODULE_LICENSE("GPL v2");

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-08 23:33   ` Michał Mirosław
@ 2021-07-29  2:43     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-29  2:43 UTC (permalink / raw)
  To: Michał Mirosław
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

Thanks for the suggestion, will change it in next RFC series.

On 7/8/21 4:33 PM, Michał Mirosław wrote:
> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>> Tegra194 device has multiple HTE instances also known as GTE
>> (Generic hardware Timestamping Engine) which can timestamp subset of
>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>> and exposes timestamping ability on those lines to the consumers
>> through HTE subsystem.
> [...]
>> +	ret = of_property_read_u32(dev->of_node, "slices", &slices);
>> +	if (ret != 0) {
>> +		dev_err(dev, "Could not read slices\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	hte_dev->sl = devm_kzalloc(dev, sizeof(struct hte_slices) * slices,
>> +				   GFP_KERNEL);
> Nit: There is devm_kcalloc() that will check for overflow in the
> multiply in case @slices from DT is broken.
>
> Best Regards
> Michał Mirosław

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-06-27 10:56   ` Linus Walleij
@ 2021-07-30  1:32     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  1:32 UTC (permalink / raw)
  To: Linus Walleij
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring


On 6/27/21 3:56 AM, Linus Walleij wrote:
> Hi Dipen,
>
> thanks a lot for this very interesting patch set!
>
> I'm gonna try to review properly, just pointing out some conceptual
> things to begin with. Bindings is a good place to start.
>
> On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
>
>> +description: |
>> +  HTE properties should be named "htes". The exact meaning of each htes
>> +  property must be documented in the device tree binding for each device.
>> +  An optional property "hte-names" may contain a list of strings to label
>> +  each of the HTE devices listed in the "htes" property.
> I think this is a bit over-abbreviated. IIO has:
> io-channels =...
> io-channel-names =...
>
> Given DT:s infatuation with using english plural I would opt for:
> hardware-timestamps = ..
> hardware-timestamp-names = ...
I can change it to suggested names in next RFC series.
>
> The "engine" part is a bit of an nVidia:ism I think and a too generic
> term. Could as well be "processor" or "automata" but nVidia just
> happened to name it an engine. (DMA engine would be a precedent
> though, so no hard preference from my side.)
>
> When reading this it is pretty intuitively evident what is going on.
>
> Other than that it looks really good!
>
>> +++ b/Documentation/devicetree/bindings/hte/hte.yaml
> I would name this hardware-timestamp-common.yamp or so.

Sure, but do I have to change hte-consumer and other hte named

yaml as well in this directory? If yes, I am referring HTE everywhere  in the

code (framework is named as hte itself), I hope that is fine and does not

create any confusion.

>
>> +title: HTE providers
> Spell this out: Hardware timestamp providers
Can I do hardware timestamp engine provider instead?
>
>> +properties:
>> +  $nodename:
>> +    pattern: "^hte(@.*|-[0-9a-f])*$"
> Likewise:
> hardware-timestamp@ ...
>
> I think this is good because it is very unambiguous.
>
>> +examples:
>> +  - |
>> +    tegra_hte_aon: hte@c1e0000 {
>> +              compatible = "nvidia,tegra194-gte-aon";
>> +              reg = <0xc1e0000 0x10000>;
>> +              interrupts = <0 13 0x4>;
>> +              int-threshold = <1>;
>> +              slices = <3>;
>> +              #hte-cells = <1>;
>> +    };
> The examples can be kept to the tegra194 bindings I think, this
> generic binding doesn't need an example as such.
Ok, will remove it.
>
>> +$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#
> This one should be named like this, that is great.
>
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Tegra194 on chip generic hardware timestamping engine (HTE)
> This is clear and nice.
>
>> +  int-threshold:
>> +    description:
>> +      HTE device generates its interrupt based on this u32 FIFO threshold
>> +      value. The recommended value is 1.
>> +    minimum: 1
>> +    maximum: 256
> Does this mean a single timestamp in the FIFO will generate an IRQ?
> Then spell that out so it is clear.
In the description I said that.
>
>> +  slices:
>> +   description:
>> +    HTE lines are arranged in 32 bit slice where each bit represents different
>> +    line/signal that it can enable/configure for the timestamp. It is u32
>> +    property and depends on the HTE instance in the chip.
>> +   oneOf:
>> +    - items:
>> +        - const: 3
>> +    - items:
>> +        - const: 11
> Can't you just use
> enum: [3, 11]
> ?
Sure, will change it.
>
>> +  '#hte-cells':
>> +    const: 1
> So IMO this would be something like
> #hardware-timestamp-cells
Sure.
>
> Other than this it overall looks very nice to me!
>
> Yours,
> Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-07-01 14:02   ` Rob Herring
@ 2021-07-30  1:56     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  1:56 UTC (permalink / raw)
  To: Rob Herring
  Cc: devicetree, linux-tegra, linux-gpio, linux-kernel, robh+dt,
	linus.walleij, thierry.reding, bgolaszewski, linux-doc,
	warthog618, jonathanh

Thanks Rob, I will correct those warning.

On 7/1/21 7:02 AM, Rob Herring wrote:
> On Fri, 25 Jun 2021 16:55:25 -0700, Dipen Patel wrote:
>> Introduces HTE devicetree binding details for the HTE subsystem. It
>> includes examples for the consumers, binding details for the providers
>> and specific binding details for the Tegra194 based HTE providers.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  .../devicetree/bindings/hte/hte-consumer.yaml | 47 +++++++++++
>>  .../devicetree/bindings/hte/hte.yaml          | 34 ++++++++
>>  .../bindings/hte/nvidia,tegra194-hte.yaml     | 83 +++++++++++++++++++
>>  3 files changed, 164 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>>  create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
>>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>>
> My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
> on your patch (DT_CHECKER_FLAGS is new in v5.13):
>
> yamllint warnings/errors:
> ./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:40:4: [warning] wrong indentation: expected 4 but found 3 (indentation)
> ./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:41:5: [warning] wrong indentation: expected 5 but found 4 (indentation)
> ./Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml:45:5: [warning] wrong indentation: expected 5 but found 4 (indentation)
> ./Documentation/devicetree/bindings/hte/hte.yaml:34:7: [error] no new line character at the end of file (new-line-at-end-of-file)
>
> dtschema/dtc warnings/errors:
> \ndoc reference errors (make refcheckdocs):
>
> See https://patchwork.ozlabs.org/patch/1497480
>
> This check can fail if there are any dependencies. The base for a patch
> series is generally the most recent rc1.
>
> If you already ran 'make dt_binding_check' and didn't see the above
> error(s), then make sure 'yamllint' is installed and dt-schema is up to
> date:
>
> pip3 install dtschema --upgrade
>
> Please check and re-submit.
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 04/11] dt-bindings: Add HTE bindings
  2021-07-01 15:54   ` Rob Herring
@ 2021-07-30  1:58     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  1:58 UTC (permalink / raw)
  To: Rob Herring
  Cc: Thierry Reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Linus Walleij, Bartosz Golaszewski,
	warthog618, devicetree, Linux Doc Mailing List


On 7/1/21 8:54 AM, Rob Herring wrote:
> On Fri, Jun 25, 2021 at 5:48 PM Dipen Patel <dipenp@nvidia.com> wrote:
>> Introduces HTE devicetree binding details for the HTE subsystem. It
>> includes examples for the consumers, binding details for the providers
>> and specific binding details for the Tegra194 based HTE providers.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  .../devicetree/bindings/hte/hte-consumer.yaml | 47 +++++++++++
>>  .../devicetree/bindings/hte/hte.yaml          | 34 ++++++++
>>  .../bindings/hte/nvidia,tegra194-hte.yaml     | 83 +++++++++++++++++++
>>  3 files changed, 164 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/hte/hte-consumer.yaml
>>  create mode 100644 Documentation/devicetree/bindings/hte/hte.yaml
>>  create mode 100644 Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>>
>> diff --git a/Documentation/devicetree/bindings/hte/hte-consumer.yaml b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
>> new file mode 100644
>> index 000000000000..79ae1f7d5185
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/hte/hte-consumer.yaml
>> @@ -0,0 +1,47 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/hte/hte-consumer.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: HTE Consumer Device Tree Bindings
>> +
>> +maintainers:
>> +  - Dipen Patel <dipenp@nvidia.com>
>> +
>> +description: |
>> +  HTE properties should be named "htes". The exact meaning of each htes
>> +  property must be documented in the device tree binding for each device.
>> +  An optional property "hte-names" may contain a list of strings to label
>> +  each of the HTE devices listed in the "htes" property.
>> +
>> +  The "hte-names" property if specified is used to map the name of the HTE
>> +  device requested by the devm_of_hte_request_ts() or of_hte_request_ts
>> +  call to an index into the list given by the "htes" property.
>> +
>> +properties:
>> +  htes:
>> +    $ref: /schemas/types.yaml#/definitions/phandle-array
>> +    description:
>> +      The list of HTE provider phandle. The provider must document the number
>> +      of cell that must be passed in this property along with phandle.
>> +
>> +  hte-names:
>> +    $ref: /schemas/types.yaml#/definitions/string-array
>> +    description:
>> +      An optional string property.
>> +
>> +required:
>> +  - "htes"
>> +
>> +dependencies:
>> +  hte-names: [ htes ]
>> +
>> +additionalProperties: true
>> +
>> +examples:
>> +  - |
>> +    hte_irq_consumer {
>> +              htes = <&tegra_hte_lic 0x19>;
>> +              hte-names = "hte-irq";
>> +    };
>> diff --git a/Documentation/devicetree/bindings/hte/hte.yaml b/Documentation/devicetree/bindings/hte/hte.yaml
>> new file mode 100644
>> index 000000000000..e285c38f1a05
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/hte/hte.yaml
>> @@ -0,0 +1,34 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/hte/hte.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: HTE providers
>> +
>> +maintainers:
>> +  - Dipen Patel <dipenp@nvidia.com>
>> +
>> +properties:
>> +  $nodename:
>> +    pattern: "^hte(@.*|-[0-9a-f])*$"
>> +
>> +  "#hte-cells":
>> +    description:
>> +      Number of cells in a HTE specifier.
>> +
>> +required:
>> +  - "#hte-cells"
>> +
>> +additionalProperties: true
>> +
>> +examples:
>> +  - |
>> +    tegra_hte_aon: hte@c1e0000 {
>> +              compatible = "nvidia,tegra194-gte-aon";
>> +              reg = <0xc1e0000 0x10000>;
>> +              interrupts = <0 13 0x4>;
>> +              int-threshold = <1>;
>> +              slices = <3>;
>> +              #hte-cells = <1>;
>> +    };
>> \ No newline at end of file
>> diff --git a/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>> new file mode 100644
>> index 000000000000..bb76cc1971f0
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/hte/nvidia,tegra194-hte.yaml
>> @@ -0,0 +1,83 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/hte/nvidia,tegra194-hte.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Tegra194 on chip generic hardware timestamping engine (HTE)
> I had to read until here to know what HTE is.
Do you think I should have added more description in commit message of hte.yaml for that matter?
>
> Is there another example of this type of h/w that this should be a
> generic binding?
For now, I have only this hardware.
>
> Rob

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 06/11] gpiolib: Add HTE support
  2021-07-01 14:24   ` Kent Gibson
@ 2021-07-30  2:25     ` Dipen Patel
  2021-07-31  5:13       ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  2:25 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 7/1/21 7:24 AM, Kent Gibson wrote:
> On Fri, Jun 25, 2021 at 04:55:27PM -0700, Dipen Patel wrote:
>> Some GPIO chip can provide hardware timestamp support on its GPIO lines
>> , in order to support that additional functions needs to be added which
>> can talk to both GPIO chip and HTE (hardware timestamping engine)
>> subsystem. This patch introduces functions which gpio consumer can use
>> to request hardware assisted timestamping. Below is the list of the APIs
>> that are added in gpiolib subsystem.
>>
>> - gpiod_hw_timestamp_control - to enable/disable HTE on specified GPIO
>> line. This API will return HTE specific descriptor for the specified
>> GPIO line during the enable call, it will be stored as pointer in the
>> gpio_desc structure as hw_ts_data.
>> - gpiod_is_hw_timestamp_enabled - to query if HTE is enabled on
>> specified GPIO line.
>> - gpiod_get_hw_timestamp - to retrieve hardware timestamps.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  drivers/gpio/gpiolib.c        | 92 +++++++++++++++++++++++++++++++++++
>>  drivers/gpio/gpiolib.h        | 11 +++++
>>  include/linux/gpio/consumer.h | 21 +++++++-
>>  include/linux/gpio/driver.h   | 13 +++++
>>  4 files changed, 135 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
>> index 220a9d8dd4e3..335eaddfde98 100644
>> --- a/drivers/gpio/gpiolib.c
>> +++ b/drivers/gpio/gpiolib.c
>> @@ -2361,6 +2361,98 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
>>  }
>>  EXPORT_SYMBOL_GPL(gpiod_direction_output);
>>  
>> +/**
>> + * gpiod_hw_timestamp_control - set the hardware assisted timestamp control.
>> + * @desc:	GPIO to set
>> + * @enable:	Set true to enable the hardware timestamp, false otherwise.
>> + *
>> + * Certain GPIO chip can rely on hardware assisted timestamp engines which can
>> + * record timestamp at the occurance of the configured events on selected GPIO
>> + * lines. This is helper API to control such engine.
>> + *
>> + * Return 0 in case of success, else an error code.
>> + */
>> +int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable)
>> +{
>> +	struct gpio_chip	*gc;
>> +	int			ret = 0;
>> +
>> +	VALIDATE_DESC(desc);
>> +	gc = desc->gdev->chip;
>> +
>> +	if (!gc->timestamp_control) {
>> +		gpiod_warn(desc,
>> +			   "%s: Hardware assisted ts not supported\n",
>> +			   __func__);
>> +		return -ENOTSUPP;
>> +	}
>> +
>> +	ret = gc->timestamp_control(gc, gpio_chip_hwgpio(desc),
>> +				    &desc->hdesc, enable);
>> +
>> +	if (ret) {
>> +		gpiod_warn(desc,
>> +			   "%s: ts control operation failed\n", __func__);
>> +		return ret;
>> +	}
>> +
>> +	if (!enable)
>> +		desc->hdesc = NULL;
>> +
>> +	return ret;
>> +}
> Last I checked, pointer accesses are not guaranteed atomic, so how is
> hdesc protected from concurrent access?
> Here is it modified unprotected.
> Below it is read unprotected.

The assumption I made here was, gpiod_hw_timestamp_control will be

called after client has made at least gpdio_request call. With that assumption,

how two or more client/consumers call gpiod_hw_timestamp_control API

with the same gpio_desc? I believe its not allowed as gpiod_request call will

fail for the looser if there is a race and hence there will not be any race here

in this API. Let me know your thoughts.

>
>> +EXPORT_SYMBOL_GPL(gpiod_hw_timestamp_control);
>> +
>> +/**
>> + * gpiod_is_hw_timestamp_enabled - check if hardware assisted timestamp is
>> + * enabled.
>> + * @desc:	GPIO to check
>> + *
>> + * Return true in case of success, false otherwise.
>> + */
>> +bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
>> +{
>> +	if (!desc)
>> +		return false;
>> +
>> +	return (desc->hdesc) ? true : false;
>> +}
>> +EXPORT_SYMBOL_GPL(gpiod_is_hw_timestamp_enabled);
>> +
>> +/**
>> + * gpiod_get_hw_timestamp - Get hardware timestamp in nano seconds.
>> + * @desc:	GPIO to get the timestamp.
>> + * @block:	Set true to block until data is available.
>> + *
>> + * Return non-zero on success, else 0.
>> + */
>> +u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
>> +{
>> +	struct gpio_chip	*gc;
>> +	int			ret = 0;
>> +	u64 ts;
>> +
>> +	VALIDATE_DESC(desc);
>> +	gc = desc->gdev->chip;
>> +
>> +	if (!gc->get_hw_timestamp) {
>> +		gpiod_warn(desc,
>> +			   "%s: Hardware assisted ts not supported\n",
>> +			   __func__);
>> +		return -ENOTSUPP;
>> +	}
>> +
> Can't return an error code here.  Return value is u64, so this will look
> like a valid ts.
>
> Just return 0 on error, as you do immediately below...
yes, good catch. I forgot to clean that up.
>
>> +	ret = gc->get_hw_timestamp(gc, block, desc->hdesc, &ts);
>> +	if (ret) {
>> +		gpiod_warn(desc,
>> +			   "%s: get timestamp operation failed\n", __func__);
>> +		return 0;
>> +	}
>> +
>> +	return ts;
>> +}
>> +EXPORT_SYMBOL_GPL(gpiod_get_hw_timestamp);
>> +
>>  /**
>>   * gpiod_set_config - sets @config for a GPIO
>>   * @desc: descriptor of the GPIO for which to set the configuration
>> diff --git a/drivers/gpio/gpiolib.h b/drivers/gpio/gpiolib.h
>> index 30bc3f80f83e..5393e1d90f61 100644
>> --- a/drivers/gpio/gpiolib.h
>> +++ b/drivers/gpio/gpiolib.h
>> @@ -15,6 +15,7 @@
>>  #include <linux/device.h>
>>  #include <linux/module.h>
>>  #include <linux/cdev.h>
>> +#include <linux/hte.h>
>>  
>>  #define GPIOCHIP_NAME	"gpiochip"
>>  
>> @@ -117,6 +118,7 @@ struct gpio_desc {
>>  #define FLAG_EDGE_RISING     16	/* GPIO CDEV detects rising edge events */
>>  #define FLAG_EDGE_FALLING    17	/* GPIO CDEV detects falling edge events */
>>  #define FLAG_EVENT_CLOCK_REALTIME	18 /* GPIO CDEV reports REALTIME timestamps in events */
>> +#define FLAG_EVENT_CLOCK_HARDWARE	19 /* GPIO CDEV reports hardware timestamps in events */
>>  
>>  	/* Connection label */
>>  	const char		*label;
>> @@ -129,6 +131,15 @@ struct gpio_desc {
>>  	/* debounce period in microseconds */
>>  	unsigned int		debounce_period_us;
>>  #endif
>> +	/*
>> +	 * Hardware timestamp engine related internal data structure.
>> +	 * This gets set when the consumer calls gpiod_hw_timestamp_control to enable
>> +	 * hardware timestamping on the specified GPIO line. The API calls into HTE
>> +	 * subsystem, in turns HTE subsystem return the HTE descriptor for the GPIO
>> +	 * line. The hdesc will be later used with gpiod_is_hw_timestamp_enabled
>> +	 * and gpiod_get_hw_timestamp API calls.
>> +	 */
>> +	struct hte_ts_desc *hdesc;
>>  };
>>  
>>  #define gpiod_not_found(desc)		(IS_ERR(desc) && PTR_ERR(desc) == -ENOENT)
>> diff --git a/include/linux/gpio/consumer.h b/include/linux/gpio/consumer.h
>> index c73b25bc9213..476ee04de7d0 100644
>> --- a/include/linux/gpio/consumer.h
>> +++ b/include/linux/gpio/consumer.h
>> @@ -112,6 +112,9 @@ int gpiod_get_direction(struct gpio_desc *desc);
>>  int gpiod_direction_input(struct gpio_desc *desc);
>>  int gpiod_direction_output(struct gpio_desc *desc, int value);
>>  int gpiod_direction_output_raw(struct gpio_desc *desc, int value);
>> +int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable);
>> +bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc);
>> +u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block);
>>  
>>  /* Value get/set from non-sleeping context */
>>  int gpiod_get_value(const struct gpio_desc *desc);
>> @@ -353,8 +356,22 @@ static inline int gpiod_direction_output_raw(struct gpio_desc *desc, int value)
>>  	WARN_ON(desc);
>>  	return -ENOSYS;
>>  }
>> -
>> -
>> +static inline int gpiod_hw_timestamp_control(struct gpio_desc *desc,
>> +					     bool enable)
>> +{
>> +	WARN_ON(desc);
>> +	return -ENOSYS;
>> +}
>> +static inline bool gpiod_is_hw_timestamp_enabled(const struct gpio_desc *desc)
>> +{
>> +	WARN_ON(desc);
>> +	return false;
>> +}
>> +static inline u64 gpiod_get_hw_timestamp(struct gpio_desc *desc, bool block)
>> +{
>> +	WARN_ON(desc);
>> +	return 0;
>> +}
>>  static inline int gpiod_get_value(const struct gpio_desc *desc)
>>  {
>>  	/* GPIO can never have been requested */
>> diff --git a/include/linux/gpio/driver.h b/include/linux/gpio/driver.h
>> index 3a268781fcec..f343e8f54b08 100644
>> --- a/include/linux/gpio/driver.h
>> +++ b/include/linux/gpio/driver.h
>> @@ -10,6 +10,7 @@
>>  #include <linux/lockdep.h>
>>  #include <linux/pinctrl/pinctrl.h>
>>  #include <linux/pinctrl/pinconf-generic.h>
>> +#include <linux/hte.h> /* For hardware timestamping */
>>  
>>  struct gpio_desc;
>>  struct of_phandle_args;
>> @@ -304,6 +305,10 @@ struct gpio_irq_chip {
>>   * @add_pin_ranges: optional routine to initialize pin ranges, to be used when
>>   *	requires special mapping of the pins that provides GPIO functionality.
>>   *	It is called after adding GPIO chip and before adding IRQ chip.
>> + * @timestamp_control: Dependent on GPIO chip, an optional routine to
>> + * 	enable/disable hardware assisted timestamp.
>> + * @get_hw_timestamp: Retrieves hardware timestamp. The consumer can specify
>> + * 	block parameter if it wishes to block till timestamp is available. 
>>   * @base: identifies the first GPIO number handled by this chip;
>>   *	or, if negative during registration, requests dynamic ID allocation.
>>   *	DEPRECATION: providing anything non-negative and nailing the base
>> @@ -396,6 +401,14 @@ struct gpio_chip {
>>  
>>  	int			(*add_pin_ranges)(struct gpio_chip *gc);
>>  
>> +	int			(*timestamp_control)(struct gpio_chip *gc,
>> +						     unsigned int offset,
>> +						     struct hte_ts_desc **hdesc,
>> +						     bool enable);
>> +	int			(*get_hw_timestamp)(struct gpio_chip *gc,
>> +						    bool block,
>> +						    struct hte_ts_desc *hdesc,
>> +						    u64 *ts);
>>  	int			base;
>>  	u16			ngpio;
>>  	const char		*const *names;
>> -- 
>> 2.17.1
>>
> Cheers,
> Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-07-09  8:30   ` Jon Hunter
@ 2021-07-30  2:33     ` Dipen Patel
  2021-08-03 16:42       ` Jon Hunter
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  2:33 UTC (permalink / raw)
  To: Jon Hunter, thierry.reding, linux-kernel, linux-tegra,
	linux-gpio, linus.walleij, bgolaszewski, warthog618, devicetree,
	linux-doc, robh+dt


On 7/9/21 1:30 AM, Jon Hunter wrote:
> On 26/06/2021 00:55, Dipen Patel wrote:
>> This patch adds new clock type for the GPIO controller which can
>> timestamp gpio lines using hardware means. To expose such
>> functionalities to the userspace, code has been added in this patch
>> where during line create call, it checks for new clock type and if
>> requested, calls hardware timestamp related API from gpiolib.c.
>> During line change event, it retrieves timestamp in nano seconds by
>> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
>> it disables this functionality by calling gpiod_hw_timestamp_control.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>>  include/uapi/linux/gpio.h   |  1 +
>>  2 files changed, 64 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>> index 1631727bf0da..9f98c727e937 100644
>> --- a/drivers/gpio/gpiolib-cdev.c
>> +++ b/drivers/gpio/gpiolib-cdev.c
>> @@ -518,6 +518,7 @@ struct linereq {
>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>  
>>  static void linereq_put_event(struct linereq *lr,
>> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>>  
>>  static u64 line_event_timestamp(struct line *line)
>>  {
>> +	bool block;
>> +
>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>>  		return ktime_get_real_ns();
>>  
>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>> +		if (irq_count())
>> +			block = false;
>> +		else
>> +			block = true;
>> +
>> +		return gpiod_get_hw_timestamp(line->desc, block);
>> +	}
>> +
>>  	return ktime_get_ns();
>>  }
>
> Looking at line_event_timestamp() and the callers of this function, it
> appears that this should always return nanoseconds. Does
> gpiod_get_hw_timestamp() return nanoseconds?
Yes, it returns in ns to align with line_event_timestamp.
>
> Jon
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-07-01 14:24   ` Kent Gibson
@ 2021-07-30  3:07     ` Dipen Patel
  2021-07-31  6:05       ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  3:07 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 7/1/21 7:24 AM, Kent Gibson wrote:
> On Fri, Jun 25, 2021 at 04:55:29PM -0700, Dipen Patel wrote:
>> This patch adds new clock type for the GPIO controller which can
>> timestamp gpio lines using hardware means. To expose such
>> functionalities to the userspace, code has been added in this patch
>> where during line create call, it checks for new clock type and if
>> requested, calls hardware timestamp related API from gpiolib.c.
>> During line change event, it retrieves timestamp in nano seconds by
>> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
>> it disables this functionality by calling gpiod_hw_timestamp_control.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>> ---
>>  drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>>  include/uapi/linux/gpio.h   |  1 +
>>  2 files changed, 64 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>> index 1631727bf0da..9f98c727e937 100644
>> --- a/drivers/gpio/gpiolib-cdev.c
>> +++ b/drivers/gpio/gpiolib-cdev.c
>> @@ -518,6 +518,7 @@ struct linereq {
>>  	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>  	 GPIO_V2_LINE_EDGE_FLAGS | \
>>  	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>  	 GPIO_V2_LINE_BIAS_FLAGS)
>>  
>>  static void linereq_put_event(struct linereq *lr,
>> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>>  
>>  static u64 line_event_timestamp(struct line *line)
>>  {
>> +	bool block;
>> +
>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>>  		return ktime_get_real_ns();
>>  
>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>> +		if (irq_count())
>> +			block = false;
>> +		else
>> +			block = true;
>> +
>> +		return gpiod_get_hw_timestamp(line->desc, block);
>> +	}
>> +
> Use in_task() instead of block?
yes, will change to in_task.
>
>>  	return ktime_get_ns();
>>  }
>>  
>> @@ -828,6 +840,7 @@ static int edge_detector_setup(struct line *line,
>>  		return ret;
>>  
>>  	line->irq = irq;
>> +
>>  	return 0;
>>  }
>>  
> Remove gratuitous whitespace changes.
> If you dislike the formatting then suggest it in a separate patch.
I will remove this space.
>
>> @@ -891,7 +904,6 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>  	/* Return an error if an unknown flag is set */
>>  	if (flags & ~GPIO_V2_LINE_VALID_FLAGS)
>>  		return -EINVAL;
>> -
>>  	/*
>>  	 * Do not allow both INPUT and OUTPUT flags to be set as they are
>>  	 * contradictory.
>> @@ -900,6 +912,14 @@ static int gpio_v2_line_flags_validate(u64 flags)
>>  	    (flags & GPIO_V2_LINE_FLAG_OUTPUT))
>>  		return -EINVAL;
>>  
> Same here.
>
>> +	/*
>> +	 * Do not mix with any other clocks if hardware assisted timestamp is
>> +	 * asked.
>> +	 */
>> +	if ((flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME) &&
>> +	    (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE))
>> +		return -EINVAL;
>> +
> The comment is very hw timestamp centric. It should just be something
> along the lines of "only allow one event clock source".
Sure, will change it.
>
>>  	/* Edge detection requires explicit input. */
>>  	if ((flags & GPIO_V2_LINE_EDGE_FLAGS) &&
>>  	    !(flags & GPIO_V2_LINE_FLAG_INPUT))
>> @@ -992,6 +1012,8 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
>>  
>>  	assign_bit(FLAG_EVENT_CLOCK_REALTIME, flagsp,
>>  		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME);
>> +	assign_bit(FLAG_EVENT_CLOCK_HARDWARE, flagsp,
>> +		   flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE);
>>  }
>>  
>>  static long linereq_get_values(struct linereq *lr, void __user *ip)
>> @@ -1139,6 +1161,18 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>  			int val = gpio_v2_line_config_output_value(lc, i);
>>  
>>  			edge_detector_stop(&lr->lines[i]);
>> +
>> +			/*
>> +			 * Assuming line was input before and hardware
>> +			 * assisted timestamp only timestamps the input
>> +			 * lines.
>> +			 */
>> +			if (gpiod_is_hw_timestamp_enabled(desc)) {
>> +				ret = gpiod_hw_timestamp_control(desc, false);
>> +				if (ret)
>> +					return ret;
>> +			}
>> +
> So if you fail to disable the hw timestamp then you fail the set_config?
> Does that make sense?
> It should be impossible to fail, as per the preceding edge_detector_stop(),
> or any failure in this context is irrelevant and so can be ignored.

I am planning to remove is_hw_timestamp* API as it is not needed.

I will also remove ret check from timestamp_control API as it is not needed.

>
>>  			ret = gpiod_direction_output(desc, val);
>>  			if (ret)
>>  				return ret;
>> @@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>  					polarity_change);
>>  			if (ret)
>>  				return ret;
>> +
>> +			/* Check if new config sets hardware assisted clock */
>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>> +				ret = gpiod_hw_timestamp_control(desc, true);
>> +				if (ret)
>> +					return ret;
>> +			}
>>  		}
>>  
> The error code here can come from the pinctrl timestamp_control(), so it
> should be sanitised before being returned to userspace.

I do not understand what do you mean by sanitise. I just followed what

gpiod_direction_output did just above which also returns ret from gpio

driver code similar to timestamp_control API.

>
>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>> @@ -1281,8 +1322,12 @@ static void linereq_free(struct linereq *lr)
>>  
>>  	for (i = 0; i < lr->num_lines; i++) {
>>  		edge_detector_stop(&lr->lines[i]);
>> -		if (lr->lines[i].desc)
>> +		if (lr->lines[i].desc) {
>> +			if (gpiod_is_hw_timestamp_enabled(lr->lines[i].desc))
>> +				gpiod_hw_timestamp_control(lr->lines[i].desc,
>> +							   false);
>>  			gpiod_free(lr->lines[i].desc);
>> +		}
> Potential race on gpiod_is_hw_timestamp_enabled() and the call to
> gpiod_hw_timestamp_control()?
> Why not put the gpiod_is_hw_timestamp_enabled() check inside
> gpiod_hw_timestamp_control()?
>
> And the gpiod_hw_timestamp_control() call should be moved inside
> gpiod_free(), or more correctly gpiod_free_commit().
> i.e. whenever you free the gpio you release any associated hw timestamp.

I am planning to remove is_hw_timestamp* API, that should take care

of race condition. For gpiod_free comment, I had thought about it before

but then ruled out as it would mean that for all the clients who did not

register with HTE, during their gpiod_free call, it has to make unncessary

call into HTE, however HTE release_ts has necessary checks which will return

without doing anything. Let me know if you still think to move it in gpiod_free.

>
>>  	}
>>  	kfifo_free(&lr->events);
>>  	kfree(lr->label);
>> @@ -1409,6 +1454,15 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip)
>>  					flags & GPIO_V2_LINE_EDGE_FLAGS);
>>  			if (ret)
>>  				goto out_free_linereq;
>> +
>> +			/*
>> +			 * Check if hardware assisted timestamp is requested
>> +			 */
>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>> +				ret = gpiod_hw_timestamp_control(desc, true);
>> +				if (ret)
>> +					goto out_free_linereq;
>> +			}
>>  		}
>>  
> Comment can fit on one line, and probably isn't even necessary - the
> code is clear enough.
I will remove comment.
>
>>  		blocking_notifier_call_chain(&desc->gdev->notifier,
>> @@ -1956,8 +2010,15 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
>>  	if (test_bit(FLAG_EDGE_FALLING, &desc->flags))
>>  		info->flags |= GPIO_V2_LINE_FLAG_EDGE_FALLING;
>>  
>> +	/*
>> +	 * Practically it is possible that user will want both the real time
>> +	 * and hardware timestamps on GPIO events, for now however lets just
>> +	 * work with either clocks
>> +	 */
>>  	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &desc->flags))
>>  		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>> +	else if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &desc->flags))
>> +		info->flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>>
> If there is any need or intent to support multiple clock sources then
> avoid creeping API changes and add it now.
> Either way, drop the comment.
I will remove comment in next RFC.
>
>>  	debounce_period_us = READ_ONCE(desc->debounce_period_us);
>>  	if (debounce_period_us) {
>> diff --git a/include/uapi/linux/gpio.h b/include/uapi/linux/gpio.h
>> index eaaea3d8e6b4..d360545b4c21 100644
>> --- a/include/uapi/linux/gpio.h
>> +++ b/include/uapi/linux/gpio.h
>> @@ -80,6 +80,7 @@ enum gpio_v2_line_flag {
>>  	GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN	= _BITULL(9),
>>  	GPIO_V2_LINE_FLAG_BIAS_DISABLED		= _BITULL(10),
>>  	GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME	= _BITULL(11),
>> +	GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE	= _BITULL(12),
>>  };
>>  
>>  /**
>> -- 
>> 2.17.1
>>
> Cheers,
> Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-06-27 11:49   ` Linus Walleij
@ 2021-07-30  3:16     ` Dipen Patel
  0 siblings, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  3:16 UTC (permalink / raw)
  To: Linus Walleij
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring


On 6/27/21 4:49 AM, Linus Walleij wrote:
> On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
>
> Just a quick question about this:
>
>> +        GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
> Is the usage intended to be such that since hardware timestamp
> can not be guaranteed we need to ask for it and fail and if that
> fails maybe the software wants to fall back to the realtime or
> common timestamp?
>
> I'm thinking from the view of libgpiod or similar apps that abstract
> this and they will be "I want to use hardware timestamps if and
> only if it is available, otherwise I want to use this other timestamp"
> or is that use case uncommon, such that either you know exactly
> what you want or you should not be messing with hardware
> timestamps?


The way currently is implemented, if you have requested

FLAG_EVENT_CLOCK_HARDWARE and it fails, control will return

to userspace with an error. There is no fallback.

>
> Yours,
> Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 09/11] tools: gpio: Add new hardware clock type
  2021-06-27 11:36   ` Linus Walleij
@ 2021-07-30  3:17     ` Dipen Patel
  2021-07-31  6:16       ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  3:17 UTC (permalink / raw)
  To: Linus Walleij
  Cc: thierry.reding, Jon Hunter, linux-kernel, linux-tegra,
	open list:GPIO SUBSYSTEM, Bartosz Golaszewski, Kent Gibson,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring


On 6/27/21 4:36 AM, Linus Walleij wrote:
> On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
>
>> gpiolib-cdev is extended to support hardware clock type, this
>> patch reflects that fact.
>>
>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> (...)
>>                 case 'w':
>>                         config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
>>                         break;
>> +               case 't':
>> +                       config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
>> +                       break;
> After the checking of the command line options we need a small sanity
> check so we don't try to enable both realtime and hardware clock
> at the same time, we will only be able to request one of them.

This will any way fail at gpiolib-cdev layer. Do we want to add it here

as well?

>
> Yours,
> Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-28 23:59     ` Dipen Patel
@ 2021-07-30  7:01       ` Dipen Patel
  2021-07-31 15:43       ` Kent Gibson
  1 sibling, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-07-30  7:01 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 7/28/21 4:59 PM, Dipen Patel wrote:
> Thanks Kent for the review comment. My responses inline.
>
> On 7/1/21 7:21 AM, Kent Gibson wrote:
>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>>> Tegra194 device has multiple HTE instances also known as GTE
>>> (Generic hardware Timestamping Engine) which can timestamp subset of
>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>>> and exposes timestamping ability on those lines to the consumers
>>> through HTE subsystem.
>>>
>>> Also, with this patch, added:
>>> - documentation about this provider and its capabilities at
>>> Documentation/hte.
>>> - Compilation support in Makefile and Kconfig
>>>
>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>> ---
>>>  Documentation/hte/index.rst        |  21 ++
>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>>  Documentation/index.rst            |   1 +
>>>  drivers/hte/Kconfig                |  12 +
>>>  drivers/hte/Makefile               |   1 +
>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>>  6 files changed, 654 insertions(+)
>>>  create mode 100644 Documentation/hte/index.rst
>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>>  create mode 100644 drivers/hte/hte-tegra194.c
>>>
>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>>> new file mode 100644
>>> index 000000000000..f311ebec6b47
>>> --- /dev/null
>>> +++ b/Documentation/hte/index.rst
>>> @@ -0,0 +1,21 @@
>>> +.. SPDX-License-Identifier: GPL-2.0
>>> +
>>> +============================================
>>> +The Linux Hardware Timestamping Engine (HTE)
>>> +============================================
>>> +
>>> +The HTE Subsystem
>>> +=================
>>> +
>>> +.. toctree::
>>> +   :maxdepth: 1
>>> +
>>> +   hte
>>> +
>>> +HTE Tegra Provider
>>> +==================
>>> +
>>> +.. toctree::
>>> +   :maxdepth: 1
>>> +
>>> +   tegra194-hte
>>> \ No newline at end of file
>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>>> new file mode 100644
>>> index 000000000000..c23eaafcf080
>>> --- /dev/null
>>> +++ b/Documentation/hte/tegra194-hte.rst
>>> @@ -0,0 +1,65 @@
>>> +HTE Kernel provider driver
>>> +==========================
>>> +
>>> +Description
>>> +-----------
>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>>> +known as generic timestamping engine (GTE). This provider driver implements
>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>>> +value.
>>> +
>>> +GPIO GTE
>>> +--------
>>> +
>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>>> +controller as it requires very specific bits to be set in GPIO config register.
>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>>> +subsystem and GPIO GTE for in kernel consumers.
>>> +
>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>>> +
>>> +	To enable HTE on given GPIO line.
>>> +
>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>>> +
>>> +	To retrieve hardwre timestamp in nano seconds.
>>> +
>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>>> +
>>> +	To query if HTE is enabled on the given GPIO.
>>> +
>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>>> +in nano second.
>>> +
>> <snip>
>>
>>> +
>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>>> +{
>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>>> +	u64 tsc;
>>> +	int dir;
>>> +	struct hte_ts_data el;
>>> +
>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>>> +		tsc = (((u64)tsh << 32) | tsl);
>>> +
>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>>> +
>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>>> +		acv = pv ^ cv;
>>> +		while (acv) {
>>> +			bit_index = __builtin_ctz(acv);
>>> +			if ((pv >> bit_index) & BIT(0))
>>> +				dir = HTE_EVENT_RISING_EDGE;
>>> +			else
>>> +				dir = HTE_EVENT_FALLING_EDGE;
>>> +
>>> +			line_id = bit_index + (slice << 5);
>>> +			el.dir = dir;
>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>>> +					      sizeof(el));
>>> +			acv &= ~BIT(bit_index);
>>> +		}
>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>>> +	}
>>> +}
>> What happens when the hte_push_ts_ns_atomic() fails?
>> The timestamp will be quietly dropped?
>> What happens when the interrupt corresponding to that dropped timestamp
>> asks for it?  The irq handler thread will block until it can get a
>> timestamp from the subsequent interrupt?
> Two things happen, 1) at the push, HTE core increments seq counter
>
> 2) If the consumer has provided callback, it will either call that callback
>
> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
>
> view of dropped ts. However, I see the problem with the consumers not
>
> providing callback, in that case, push_ts* API just wakes up process without
>
> indicating why (assuming notify variable is true or else there is a chance for
>
> the thread to block forever). One easy approach I can think of for now is to
>
> make callback mandatory (which is optional right now), I will have to rethink
>
> that scenario and will push corrected version next RFC version.
>
> Thanks for pointing out.
>
>> Which brings me back to the concern I have with the approach used in
>> the hte/gpiolib integration - how do you guarantee that the timestamp
>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
>> being handled, particularly in the face of errors such as:
>>  - overflows of the timestamp FIFO in the chip
> I currently do not have any indication mechanism as the providers
>
> I am dealing with right now does not have overflow hardware detection
>
> support. If the chip supports, it should be easy to integrate that feature.
>
> I will provide some hook function or change in push_* API to accommodate
>
> this in next version of RFC.
>
>>  - overflows of software FIFOs as here
> HTE core records sequence counter as well it callsback the consumer with
>
> HTE_TS_DROPPED.
>
>>  - lost interupts (if the hw generates interrupts faster than the CPU
>>    can service them)
> For this, I have no idea unless hardware supports some sort of mechanism
>
> to catch that. For the current providers, as soon as it detects changes on lines
>
> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
>
> set in that hw fifo. This interrupt is different than the lines of actual device
>
> that is why I said I have no idea how we can tackle that. Let me know if there
>
> is any idea or reference of the codes which does tackle this.
>
>
> Regarding HTE/GPIOLIB integration comment:
>
> You are right, currently, I have only tsc field returned from struct hte_ts_data
>
> to gpiolib. If I can extend that to return hte_ts_data structure which has seq
>
> counter, which I believe can be used to track the overflow situation. The

The reason I only return timestamp and not other details like its seq

counter, is because to comply with line_event_timestamp since it returns

only u64. Not sure which is the best way to extend and bring out its seq.

>
> dropped scenario can be easily tracked if gpiolib can be notified with above
>
> mentioned DROP event through callback. If that is the case, is it ok to have
>
> some sort of callback per gpio in gpiolib?
>
>
> Any idea how I can integrate callback notification with gpiolib if you do not agree on
>
> above callback suggestion?
>
>> ?
>>
>> Cheers,
>> Kent.
>>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 06/11] gpiolib: Add HTE support
  2021-07-30  2:25     ` Dipen Patel
@ 2021-07-31  5:13       ` Kent Gibson
  0 siblings, 0 replies; 75+ messages in thread
From: Kent Gibson @ 2021-07-31  5:13 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Thu, Jul 29, 2021 at 07:25:36PM -0700, Dipen Patel wrote:
> 
> On 7/1/21 7:24 AM, Kent Gibson wrote:
> > On Fri, Jun 25, 2021 at 04:55:27PM -0700, Dipen Patel wrote:
> >> Some GPIO chip can provide hardware timestamp support on its GPIO lines
> >> , in order to support that additional functions needs to be added which
> >> can talk to both GPIO chip and HTE (hardware timestamping engine)
> >> subsystem. This patch introduces functions which gpio consumer can use
> >> to request hardware assisted timestamping. Below is the list of the APIs
> >> that are added in gpiolib subsystem.
> >>
> >> - gpiod_hw_timestamp_control - to enable/disable HTE on specified GPIO
> >> line. This API will return HTE specific descriptor for the specified
> >> GPIO line during the enable call, it will be stored as pointer in the
> >> gpio_desc structure as hw_ts_data.
> >> - gpiod_is_hw_timestamp_enabled - to query if HTE is enabled on
> >> specified GPIO line.
> >> - gpiod_get_hw_timestamp - to retrieve hardware timestamps.
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >> ---
> >>  drivers/gpio/gpiolib.c        | 92 +++++++++++++++++++++++++++++++++++
> >>  drivers/gpio/gpiolib.h        | 11 +++++
> >>  include/linux/gpio/consumer.h | 21 +++++++-
> >>  include/linux/gpio/driver.h   | 13 +++++
> >>  4 files changed, 135 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
> >> index 220a9d8dd4e3..335eaddfde98 100644
> >> --- a/drivers/gpio/gpiolib.c
> >> +++ b/drivers/gpio/gpiolib.c
> >> @@ -2361,6 +2361,98 @@ int gpiod_direction_output(struct gpio_desc *desc, int value)
> >>  }
> >>  EXPORT_SYMBOL_GPL(gpiod_direction_output);
> >>  
> >> +/**
> >> + * gpiod_hw_timestamp_control - set the hardware assisted timestamp control.
> >> + * @desc:	GPIO to set
> >> + * @enable:	Set true to enable the hardware timestamp, false otherwise.
> >> + *
> >> + * Certain GPIO chip can rely on hardware assisted timestamp engines which can
> >> + * record timestamp at the occurance of the configured events on selected GPIO
> >> + * lines. This is helper API to control such engine.
> >> + *
> >> + * Return 0 in case of success, else an error code.
> >> + */
> >> +int gpiod_hw_timestamp_control(struct gpio_desc *desc, bool enable)
> >> +{
> >> +	struct gpio_chip	*gc;
> >> +	int			ret = 0;
> >> +
> >> +	VALIDATE_DESC(desc);
> >> +	gc = desc->gdev->chip;
> >> +
> >> +	if (!gc->timestamp_control) {
> >> +		gpiod_warn(desc,
> >> +			   "%s: Hardware assisted ts not supported\n",
> >> +			   __func__);
> >> +		return -ENOTSUPP;
> >> +	}
> >> +
> >> +	ret = gc->timestamp_control(gc, gpio_chip_hwgpio(desc),
> >> +				    &desc->hdesc, enable);
> >> +
> >> +	if (ret) {
> >> +		gpiod_warn(desc,
> >> +			   "%s: ts control operation failed\n", __func__);
> >> +		return ret;
> >> +	}
> >> +
> >> +	if (!enable)
> >> +		desc->hdesc = NULL;
> >> +
> >> +	return ret;
> >> +}
> > Last I checked, pointer accesses are not guaranteed atomic, so how is
> > hdesc protected from concurrent access?
> > Here is it modified unprotected.
> > Below it is read unprotected.
> 
> The assumption I made here was, gpiod_hw_timestamp_control will be
> 
> called after client has made at least gpdio_request call. With that assumption,
> 
> how two or more client/consumers call gpiod_hw_timestamp_control API
> 
> with the same gpio_desc? I believe its not allowed as gpiod_request call will
> 
> fail for the looser if there is a race and hence there will not be any race here
> 
> in this API. Let me know your thoughts.
> 

My assumptions are that the userspace client is multi-threaded and that
there is nothing preventing concurrent uAPI calls, including closing the
line request fd.

The specific case I had in mind is one thread closing the req fd while
another is using set_config to switch to the hardware event clock.
In that race, the close be calling linereq_free() at the same time the
linereq_set_config_unlocked() is being called.  Both of those functions
make calls to the functions here that read and write the hdesc.

There may be others, e.g. line_event_timestamp() running in the
irq_thread at the same time a set_config call switches the event clock
away from the hardware clock.

So assume concurrent access unless you can prove otherwise.

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-07-30  3:07     ` Dipen Patel
@ 2021-07-31  6:05       ` Kent Gibson
  2021-08-03 22:41         ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-07-31  6:05 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt,
	andriy.shevchenko

On Thu, Jul 29, 2021 at 08:07:15PM -0700, Dipen Patel wrote:
> 
> On 7/1/21 7:24 AM, Kent Gibson wrote:

<snip>
> >
> >>  			ret = gpiod_direction_output(desc, val);
> >>  			if (ret)
> >>  				return ret;
> >> @@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
> >>  					polarity_change);
> >>  			if (ret)
> >>  				return ret;
> >> +
> >> +			/* Check if new config sets hardware assisted clock */
> >> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >> +				ret = gpiod_hw_timestamp_control(desc, true);
> >> +				if (ret)
> >> +					return ret;
> >> +			}
> >>  		}
> >>  
> > The error code here can come from the pinctrl timestamp_control(), so it
> > should be sanitised before being returned to userspace.
> 
> I do not understand what do you mean by sanitise. I just followed what
> 
> gpiod_direction_output did just above which also returns ret from gpio
> 
> driver code similar to timestamp_control API.
> 

In this context, sanitise means convert any kernel internal error codes
to their userspace equivalent before returning them to userspace.

Fair enough with the gpiod_direction_output() comparison.  I was thinking
of a patch Andy recently submitted[1] to sanitise gpiod_request(), which
can sometimes return EPROBE_DEFER.  But I guess we can wait until we find
a case of a driver returning an internal error code and add a sanitiser
then.

Cheers,
Kent.

[1] https://www.spinics.net/lists/linux-gpio/msg60998.html


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 09/11] tools: gpio: Add new hardware clock type
  2021-07-30  3:17     ` Dipen Patel
@ 2021-07-31  6:16       ` Kent Gibson
  2021-08-11  9:11         ` Linus Walleij
  0 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-07-31  6:16 UTC (permalink / raw)
  To: Dipen Patel
  Cc: Linus Walleij, thierry.reding, Jon Hunter, linux-kernel,
	linux-tegra, open list:GPIO SUBSYSTEM, Bartosz Golaszewski,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Thu, Jul 29, 2021 at 08:17:22PM -0700, Dipen Patel wrote:
> 
> On 6/27/21 4:36 AM, Linus Walleij wrote:
> > On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
> >
> >> gpiolib-cdev is extended to support hardware clock type, this
> >> patch reflects that fact.
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> > (...)
> >>                 case 'w':
> >>                         config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> >>                         break;
> >> +               case 't':
> >> +                       config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
> >> +                       break;
> > After the checking of the command line options we need a small sanity
> > check so we don't try to enable both realtime and hardware clock
> > at the same time, we will only be able to request one of them.
> 
> This will any way fail at gpiolib-cdev layer. Do we want to add it here
> 
> as well?
> 

I can't speak for Linus, but I'm fine with it as is as it allows the tool
to be used to exercise the sanity check in the kernel.

Cheers,
Kent.


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-28 23:59     ` Dipen Patel
  2021-07-30  7:01       ` Dipen Patel
@ 2021-07-31 15:43       ` Kent Gibson
  2021-08-03 22:40         ` Dipen Patel
  2021-08-07  2:41         ` Dipen Patel
  1 sibling, 2 replies; 75+ messages in thread
From: Kent Gibson @ 2021-07-31 15:43 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
> Thanks Kent for the review comment. My responses inline.
> 
> On 7/1/21 7:21 AM, Kent Gibson wrote:
> > On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
> >> Tegra194 device has multiple HTE instances also known as GTE
> >> (Generic hardware Timestamping Engine) which can timestamp subset of
> >> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> >> and exposes timestamping ability on those lines to the consumers
> >> through HTE subsystem.
> >>
> >> Also, with this patch, added:
> >> - documentation about this provider and its capabilities at
> >> Documentation/hte.
> >> - Compilation support in Makefile and Kconfig
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >> ---
> >>  Documentation/hte/index.rst        |  21 ++
> >>  Documentation/hte/tegra194-hte.rst |  65 ++++
> >>  Documentation/index.rst            |   1 +
> >>  drivers/hte/Kconfig                |  12 +
> >>  drivers/hte/Makefile               |   1 +
> >>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
> >>  6 files changed, 654 insertions(+)
> >>  create mode 100644 Documentation/hte/index.rst
> >>  create mode 100644 Documentation/hte/tegra194-hte.rst
> >>  create mode 100644 drivers/hte/hte-tegra194.c
> >>
> >> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
> >> new file mode 100644
> >> index 000000000000..f311ebec6b47
> >> --- /dev/null
> >> +++ b/Documentation/hte/index.rst
> >> @@ -0,0 +1,21 @@
> >> +.. SPDX-License-Identifier: GPL-2.0
> >> +
> >> +============================================
> >> +The Linux Hardware Timestamping Engine (HTE)
> >> +============================================
> >> +
> >> +The HTE Subsystem
> >> +=================
> >> +
> >> +.. toctree::
> >> +   :maxdepth: 1
> >> +
> >> +   hte
> >> +
> >> +HTE Tegra Provider
> >> +==================
> >> +
> >> +.. toctree::
> >> +   :maxdepth: 1
> >> +
> >> +   tegra194-hte
> >> \ No newline at end of file
> >> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> >> new file mode 100644
> >> index 000000000000..c23eaafcf080
> >> --- /dev/null
> >> +++ b/Documentation/hte/tegra194-hte.rst
> >> @@ -0,0 +1,65 @@
> >> +HTE Kernel provider driver
> >> +==========================
> >> +
> >> +Description
> >> +-----------
> >> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
> >> +known as generic timestamping engine (GTE). This provider driver implements
> >> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
> >> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> >> +driver converts clock tick rate to nano seconds before storing it as timestamp
> >> +value.
> >> +
> >> +GPIO GTE
> >> +--------
> >> +
> >> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
> >> +needs to be configured as input and IRQ needs to ba enabled as well. The only
> >> +always on (AON) gpio controller instance supports timestamping GPIOs in
> >> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
> >> +controller as it requires very specific bits to be set in GPIO config register.
> >> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
> >> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
> >> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
> >> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
> >> +subsystem and GPIO GTE for in kernel consumers.
> >> +
> >> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
> >> +
> >> +	To enable HTE on given GPIO line.
> >> +
> >> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
> >> +
> >> +	To retrieve hardwre timestamp in nano seconds.
> >> +
> >> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
> >> +
> >> +	To query if HTE is enabled on the given GPIO.
> >> +
> >> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> >> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
> >> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
> >> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
> >> +in nano second.
> >> +
> > <snip>
> >
> >> +
> >> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
> >> +{
> >> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
> >> +	u64 tsc;
> >> +	int dir;
> >> +	struct hte_ts_data el;
> >> +
> >> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
> >> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
> >> +		HTE_TESTATUS_OCCUPANCY_MASK) {
> >> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
> >> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
> >> +		tsc = (((u64)tsh << 32) | tsl);
> >> +
> >> +		src = tegra_hte_readl(gs, HTE_TESRC);
> >> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
> >> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
> >> +
> >> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
> >> +		cv = tegra_hte_readl(gs, HTE_TECCV);
> >> +		acv = pv ^ cv;
> >> +		while (acv) {
> >> +			bit_index = __builtin_ctz(acv);
> >> +			if ((pv >> bit_index) & BIT(0))
> >> +				dir = HTE_EVENT_RISING_EDGE;
> >> +			else
> >> +				dir = HTE_EVENT_FALLING_EDGE;
> >> +
> >> +			line_id = bit_index + (slice << 5);
> >> +			el.dir = dir;
> >> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
> >> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
> >> +					      sizeof(el));
> >> +			acv &= ~BIT(bit_index);
> >> +		}
> >> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
> >> +	}
> >> +}
> > What happens when the hte_push_ts_ns_atomic() fails?
> > The timestamp will be quietly dropped?
> > What happens when the interrupt corresponding to that dropped timestamp
> > asks for it?  The irq handler thread will block until it can get a
> > timestamp from the subsequent interrupt?
> 
> Two things happen, 1) at the push, HTE core increments seq counter
> 
> 2) If the consumer has provided callback, it will either call that callback
> 
> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
> 
> view of dropped ts. However, I see the problem with the consumers not
> 
> providing callback, in that case, push_ts* API just wakes up process without
> 
> indicating why (assuming notify variable is true or else there is a chance for
> 
> the thread to block forever). One easy approach I can think of for now is to
> 
> make callback mandatory (which is optional right now), I will have to rethink
> 
> that scenario and will push corrected version next RFC version.
> 
> Thanks for pointing out.
> 

I'm not sure you understood my question, which was intended to
demonstrate how an overflow here would break your gpio integration, but I
am certain that I don't understand your answer.

Using the callback to signal fifo overflow to the consumer is crazy.
If the consumer is too busy to service the fifo then they probably wont
be prepared to deal with the callback either. And the primary purpose of
the fifo is to decouple the producer and consumer, so requiring a callback
defeats the whole purpose of having the fifo there in the first place.

> >
> > Which brings me back to the concern I have with the approach used in
> > the hte/gpiolib integration - how do you guarantee that the timestamp
> > returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
> > being handled, particularly in the face of errors such as:
> >  - overflows of the timestamp FIFO in the chip
> 
> I currently do not have any indication mechanism as the providers
> 
> I am dealing with right now does not have overflow hardware detection
> 
> support. If the chip supports, it should be easy to integrate that feature.
> 
> I will provide some hook function or change in push_* API to accommodate
> 
> this in next version of RFC.
> 
> >  - overflows of software FIFOs as here
> 
> HTE core records sequence counter as well it callsback the consumer with
> 
> HTE_TS_DROPPED.
> 
> >  - lost interupts (if the hw generates interrupts faster than the CPU
> >    can service them)
> 
> For this, I have no idea unless hardware supports some sort of mechanism
> 
> to catch that. For the current providers, as soon as it detects changes on lines
> 
> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
> 
> set in that hw fifo. This interrupt is different than the lines of actual device
> 
> that is why I said I have no idea how we can tackle that. Let me know if there
> 
> is any idea or reference of the codes which does tackle this.
> 

As far as I am aware there is no solution, given your suggested
architecture.

Your architecture is inherently fragile, as you try to use one stream
of data (the timestamp fifo) to provide supplementary info for another
(the physical irq).  Guaranteeing that the two are synchronised is
impossible - even if you can get them synced at some point, they can
fall out of sync without any indication.
That is a recipe for Ingenuity flight 6.

My solution would be to use the hte timestamp fifo as the event source,
rather than the physical irq.  With only one event source the 
synchronisation problem disappears.  As to how to implement that,
gpiolib-cdev would request a line from the hte subsystem and register
and event handler for it, much as it does currently with the irq
subsystem. That event handler would translate the hte events into gpio
events.

You still have to deal with possible fifo overflows, but if the fifo
overflows result in discarding the oldest event, rather than the most
recent, then everything comes out in the wash.  If not then the final
event in a burst may not correspond to the actual state so you need
some additional mechanism to address that.
Either way the consumer needs to be aware that events may be lost - but
with the event seqno for consumers to detect those lost events we
already have that covered.

> 
> Regarding HTE/GPIOLIB integration comment:
> 
> You are right, currently, I have only tsc field returned from struct hte_ts_data
> 
> to gpiolib. If I can extend that to return hte_ts_data structure which has seq
> 
> counter, which I believe can be used to track the overflow situation. The
> 
> dropped scenario can be easily tracked if gpiolib can be notified with above
> 
> mentioned DROP event through callback. If that is the case, is it ok to have
> 
> some sort of callback per gpio in gpiolib?
> 

Even better if you can provide the whole struct hte_ts_data so we have
the direction as well (assuming all hte providers provide direction?).
Otherwise gpiolib-cdev may need to read the physical line state and that
may have changed since the hardware captured the event.
In the solution I outlined above, the hte_ts_data would be provided to
the event handler registered by gpiolib-cdev.
And in this case you could skip buffering the event in hte - it could be
passed to the event handler as soon as it is read from the hardware - 
gpiolib-cdev does its own buffering of gpio events.

> 
> Any idea how I can integrate callback notification with gpiolib if you do not agree on
> 
> above callback suggestion?
> 

See above.  But this is just my take, so I would get feedback from the
relevant maintainers or SMEs before you act on anything suggested above.

Cheers,
Kent.


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 01/11] Documentation: Add HTE subsystem guide
  2021-07-27 23:44     ` Dipen Patel
@ 2021-08-01 15:24       ` Jonathan Cameron
  0 siblings, 0 replies; 75+ messages in thread
From: Jonathan Cameron @ 2021-08-01 15:24 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Tue, 27 Jul 2021 16:44:30 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> On 7/4/21 11:55 AM, Jonathan Cameron wrote:
> > On Fri, 25 Jun 2021 16:55:22 -0700
> > Dipen Patel <dipenp@nvidia.com> wrote:
> >  
> >> Adding hte document which can help understand various APIs implemented
> >> in HTE framework for the HTE producers and the consumers.
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>  
> > Some editorial stuff inline. (I can't resist even on RFCs)
> >
> > Certainly interesting. I'm running a bit tight on time today, so not sure how
> > much of the code I'll get a chance to look at.  Will try to get to it soon though.
> >
> > Jonathan  
> Thanks Jonathan for the review comment and time. My answers inline.
Hi Dipen,

A few follow up comments inline.

> >  
> >> ---
> >>  Documentation/hte/hte.rst | 198 ++++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 198 insertions(+)
> >>  create mode 100644 Documentation/hte/hte.rst
> >>
> >> diff --git a/Documentation/hte/hte.rst b/Documentation/hte/hte.rst
> >> new file mode 100644
> >> index 000000000000..11744dbc6d16
> >> --- /dev/null
> >> +++ b/Documentation/hte/hte.rst
> >> @@ -0,0 +1,198 @@
> >> +============================================
> >> +The Linux Hardware Timestamping Engine (HTE)
> >> +============================================
> >> +
> >> +:Author: Dipen Patel
> >> +
> >> +Introduction
> >> +------------
> >> +
> >> +The certain devices have the built in hardware timestamping engine which can  
> > Certain devices have built in hardware timestamping engines which can
> >  
> >> +monitor sets of system signals, lines, buses etc... in realtime for the state  
> > for state changes;
> >  
> >> +change; upon detecting the change it can automatically store the timestamp at  
> > they can  
> Will add above in next RFC version2.
> >  
> >> +the moment of occurrence. Such functionality may help achieve better accuracy
> >> +in obtaining timestamp than using software counterparts i.e. ktime and friends.
> >> +
> >> +This document describes the API that can be used by hardware timestamping
> >> +engine provider and consumer drivers that want to use the hardware timestamping
> >> +engine (HTE) framework.
> >> +
> >> +The HTE framework APIs for the providers
> >> +----------------------------------------
> >> +Each driver must #include <linux/hte.h>. The ``linux/hte.h`` declares the
> >> +following functions for the provider:
> >> +
> >> +.. c:function:: int hte_register_chip( struct hte_chip *chip )
> >> +		int hte_unregister_chip( struct hte_chip *chip )
> >> +
> >> +	The provider uses these APIs to un/register itself with HTE framework.
> >> +
> >> +.. c:function:: int hte_push_ts_ns_atomic( const struct hte_chip *chip, u32 xlated_id, struct hte_ts_data *data, size_t n )
> >> +
> >> +	The provider pushes timestamp data in nano seconds unit using this API.
> >> +
> >> +The detail about parameters and API usage are described in each functions
> >> +definitions in ``drivers/hte/hte.c`` file.
> >> +
> >> +The HTE framework APIs for the consumers
> >> +----------------------------------------
> >> +The consumers use following APIs to control the line for the timestamp:
> >> +  
> > When documenting APIs you may well be better including a reference to the files
> > themselves and using kernel doc there.  The documentation build can then pull that
> > in when creating the html docs etc (and crucially you don't have to provide the
> > same docs in two places.).   Having them here is very convenient for the RFC however :)  
> You mean to omit description here and put reference to file like ``drivers/hte/hte.c``?

Exactly. You can cross reference to kernel-doc from within the rst. That means the
documentation will be in the generated html etc and available in the source code,
cutting down on duplication and chances of them disagreeing.  The disadvantage
is it isn't quite as useful to provide an easily reviewable single document when
discussing the design.

What can be useful in a docs file though is to describe the 'flow' of how these
functions might be used.  That can be harder to do inline in comments etc.

> >  
> >> +.. c:function:: int hte_release_ts( struct hte_ts_desc *desc )
> >> +		int devm_hte_release_ts( struct device *dev, struct hte_ts_desc *desc )
> >> +
> >> +	The consumer uses API to release specified desc from timestamping.
> >> +	The API frees resources associated with the desc and disables the
> >> +	timestamping on it. The later is managed version of the same API.
> >> +
> >> +.. c:function:: struct hte_ts_desc *of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
> >> +		struct hte_ts_desc *devm_of_hte_request_ts( struct device *dev, const char *label, void (*cb)(enum hte_notify n) )
> >> +
> >> +	The consumers can use above request APIs to request real timestamp
> >> +	capability on specified entity. The later is resource managed version
> >> +	of the of_hte_request_ts API. Both the APIs expect consumer to follow
> >> +	device tree bindings for the HTE consumer. The details about binding
> >> +	is in ``Documentation/devicetree/bindings/hte/hte-consumer.yaml``.
> >> +
> >> +.. c:function:: struct hte_ts_desc *hte_req_ts_by_dt_node( struct device_node *of_node, unsigned int id, void (*cb)(enum hte_notify n) )
> >> +
> >> +	The consumer can request timestamping directly specifying provider
> >> +	device tree node.  
> > When does this make sense?  
> 
> This is needed when provider has dependencies on other IP within chip, for example
> 
> tegra chip GPIO HTE has to talk to GPIO controller to fully enable HTE functionality.

I'd expect that to be done via a device tree handle at the consumer end.  So you'd be
requesting based on your own struct device pointer (and the firmware node
underneath though that would ideally not be visible in this interface).

Similar to how regulator and other provider / consumer firmware description works.

Whilst it's not immediately clear what this would map to in other firmware types, you
should also try to avoid exposing device tree specific interfaces, in favour of
generic ones (see include/property.h)

> 
> >  
> >> +
> >> +.. c:function:: int hte_enable_ts( struct hte_ts_desc *desc )
> >> +.. c:function:: int hte_disable_ts( struct hte_ts_desc *desc )
> >> +
> >> +	The consumer can enable/disable timestamping on given desc.
> >> +
> >> +.. c:function:: int hte_retrieve_ts_ns( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
> >> +		int hte_retrieve_ts_ns_wait( const struct hte_ts_desc *desc, struct hte_ts_data *el, size_t n )
> >> +
> >> +	The consumer uses above two API versions to get/retrieve timestamp data
> >> +	for the given desc. The later is blocking version.
> >> +
> >> +.. c:function:: hte_get_clk_src_info(const struct hte_line_desc *desc, struct hte_clk_info *ci)
> >> +
> >> +	The consumer retrieves clock source information that provider uses to
> >> +	timestamp entity in the structure hte_clk_info. This information
> >> +	specifies clock rate in HZ and clock.
> >> +
> >> +The details on struct hte_clk_info
> >> +-----------------------------------
> >> +This structure presents detail of the hardware clock that provider uses for
> >> +realtime timestamping purposes. The consumer can use hte_get_clk_src_info API
> >> +to get the information in hte_clk_info structure. It has hz and type parameters
> >> +where hz represents clock rate in HZ and type is clock type of clockid_t and
> >> +of CLOCK_* family (for example, CLOCK_MONOTONIC).
> >> +
> >> +The consumers calling of_hte_request_ts or hte_req_ts_by_dt_node APIs with
> >> +cb parameter set, usually will call hte_retrieve_ts (non blocking
> >> +version) after being notified by the callbacks from HTE subsystem. The
> >> +consumers calling those requests APIs with cb parameter NULL, usually will call
> >> +hte_retrieve_ts_wait API.
> >> +
> >> +The HTE subsystem provides software buffer per requested id/entity to store
> >> +timestamp data (struct hte_ts_data type). The consumers can manage the buffer.
> >> +It also provides buffer watermark which can notify (if cb parameter is provided
> >> +during request API call) consumer or unblock consumers calling
> >> +hte_retrieve_ts_wait API. The following APIs are used to manipulate the
> >> +software buffer:  
> > Have you come across any devices that have a hardware fifo for these timestamps?
> > It's moderately common on sensor hubs to do so, and then you get into a fun question
> > of how to manage the watermark.  You don't want to pull from the hardware too early,
> > but conversely you can get out of sync between the software and hardware buffers if
> > someone reasons less than 'watermark' samples from the software buffer.
> >
> > Anyhow, it can be entertaining.  So in those cases it can be simpler to explicitly provide
> > control of two separate watermarks.  
> 
> The provider I have dealt with had single hardware FIFO to store timestamps
> 
> indiscriminately. I am sure this will come up in future in which case we can
> 
> expand it to separate watermark.

Just to check I've understood this correctly.
You do have a hardware fifo and it has a fixed watermark? (perhaps of 1 timestamp?)
If so, indeed one for the future.

> 
> >  
> >> +
> >> +.. c:function:: int hte_set_buf_len( const struct hte_ts_desc *desc,unsigned int len )
> >> +		int hte_get_buf_len( const struct hte_ts_desc *desc )
> >> +
> >> +	The consumer uses above APIs to set/get software buffer depth.  
> > What happens if there is content when it is resized?  
> 
> I have described in the hte_set_buf_len API description. To summarize, you can
> 
> follow certain sequences to consume old data if you still care. Otherwise this
> 
> is a destructive API.

OK.  You might want to think about blocking this from changing if the timestamping
is currently enabled (assuming you don't already!)  Otherwise you tend to get
entertaining race conditions that consumers will need to deal with - probably by
doing the locking at their end.

> 
> >  
> >> +
> >> +.. c:function:: int hte_set_buf_watermark( const struct hte_ts_desc *desc, unsigned int val )
> >> +		int hte_get_buf_watermark( const struct hte_ts_desc *desc )
> >> +
> >> +	The consumer uses above APIs to set/get software threshold, threshold
> >> +	can be used to notity or unblock waiting consumer when data becomes
> >> +	available equal or above to threshold value.
> >> +
> >> +.. c:function:: size_t hte_available_ts( const struct hte_ts_desc *desc )
> >> +
> >> +	The consumer uses above API to get available timestamp data stored
> >> +	in the software buffer for the desc.
> >> +
> >> +The detail about parameters and API usage are described in each functions
> >> +definitions in ``drivers/hte/hte.c`` file.
> >> +
> >> +The HTE timestamp element detail
> >> +--------------------------------
> >> +The struct hte_ts_data, declared at ``include/linux/hte.h``, is used to pass
> >> +timestamp details between the consumers and the providers. It expresses
> >> +timestamp data in nano second in u64 data type.  
> > I'd suggest s64 to match with kernel timestamp format.  
> Make sense, I will update in next revision.
> >  
> >> For now all the HTE APIs
> >> +using struct hte_ts_data requires tsc to be in nano seconds. The timestamp
> >> +element structure stores below information along with timestamp data::
> >> +
> >> + struct hte_ts_data {
> >> +	/*
> >> +	 * Timestamp value
> >> +	 */
> >> +	u64 tsc;
> >> +	/*
> >> +	 * The sequence counter, keep track of the number of timestamps.
> >> +	 * It can be used to check if data is dropped in between.
> >> +	 */  
> > Is this a hardware feature?  A bit unusual to have this rather than simple
> > overflow flag to indicate we dropped an unknown number of samples.  
> Its software feature. I Believe having seq helps consumer to backtrack.

Will be interesting to see how this works out, particularly as I expect you
will get hardware that only tells you it's dropped something, but not how much.
That's what all the sensor hardware with timestamps currently does on overflow
Often you can control if old or new samples are dropped, but either way you will
get a hole of unknown size.

> >  
> >> +	u64 seq;
> >> +	/* Direction of the event, i.e. falling or rising */
> >> +	int dir;  
> > Given an even could do more than that potentially, or indeed not be able to
> > tell if it was rising or falling, I would suggest an enum to which we can add
> > more options as needed.  
> I have two defines in hte.h for now. I can convert them into enum type.

yikes, I should proof read my comments before sending! Glad you figured out what
I meant.  

> >  
> >> + };
> >> +
> >> +The typical hte_ts_data data life cycle::
> >> +In this example the provider provides timestamp in nano seconds and for the
> >> +GPIO line::
> >> +
> >> + - Monitors GPIO line change.
> >> + - Detects the state change on GPIO line.
> >> + - Converts timestamps in nano seconds and stores it in tsc.
> >> + - Stores GPIO direction in dir variable if the provider has that hardware
> >> + capability.  
> > We definitely want to know if it does or not.  How does an application query that?  
> Its stored in dir field of the hte_ts_data structure.

I wasn't clear in this comment.  We need a way to know the hardware does not support
providing the direction and hence the dir field is not valid.  Is there a way to
find that out from a consumer driver?

Thanks,

Jonathan

> >  
> >> + - Pushes this hte_timestamp_el object to HTE subsystem.
> >> + - HTE subsystem increments seq counter and stores it in software buffer
> >> + dedicated to requested GPIO line.  
> > Ah. So that seq counter is only for software drops if the fifo fills up.  
> Yes.
> >  
> >> + - Waiting consumer gets notified.
> >> + - The consumer calls the retrieve timestamp API.
> >> +
> >> +HTE subsystem debugfs attributes
> >> +--------------------------------
> >> +HTE subsystem creates debugfs attributes at ``/sys/kernel/debug/hte/``.
> >> +It also creates line/signal related debugfs attributes at
> >> +``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> >> +
> >> +`ts_requested`
> >> +		The total number of entities requested from the given provider,
> >> +		where entity is the provider specific and could represent
> >> +		lines, GPIO, chip signals, buses etc...
> >> +                The attribute will be availble at
> >> +		``/sys/kernel/debug/hte/<provider>/``.
> >> +
> >> +		Read only value
> >> +
> >> +`total_ts`
> >> +		The total number of entities supported by the provider.
> >> +                The attribute will be availble at
> >> +		``/sys/kernel/debug/hte/<provider>/``.
> >> +
> >> +		Read only value
> >> +
> >> +`ts_buffer_depth`
> >> +		The software buffer lenth to store timestamp data.
> >> +                The attribute will be availble at
> >> +		``/sys/kernel/debug/hte/<provider>/<label or id>/``.
> >> +
> >> +		Read only value
> >> +
> >> +`ts_buffer_watermark`
> >> +		The software buffer watermark or threshold.
> >> +                The attribute will be availble at
> >> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> >> +
> >> +		Read only value
> >> +
> >> +`dropped_timestamps`
> >> +		The dropped timestamps for a given line.
> >> +                The attribute will be availble at
> >> +		``/sys/kernel/debug/hte/<provider>/<label or line id>/``.
> >> +
> >> +		Read only value  


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-28  4:38     ` Dipen Patel
@ 2021-08-01 16:13       ` Jonathan Cameron
  2021-09-14  5:43         ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Jonathan Cameron @ 2021-08-01 16:13 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Tue, 27 Jul 2021 21:38:45 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> On 7/4/21 1:15 PM, Jonathan Cameron wrote:
> > On Fri, 25 Jun 2021 16:55:23 -0700
> > Dipen Patel <dipenp@nvidia.com> wrote:
> >  
> >> Some devices can timestamp system lines/signals/Buses in real-time
> >> using the hardware counter or other hardware means which can give
> >> finer granularity and help avoid jitter introduced by software means
> >> of timestamping. To utilize such functionality there has to be
> >> framework where such devices can register themselves as producers or
> >> providers so that the consumers or clients devices can request specific
> >> line from the providers. This patch introduces such subsystem as
> >> hardware timestamping engine (HTE).
> >>
> >> It provides below APIs for the provider:
> >> - hte_register_chip() -- To register the HTE chip.
> >> - hte_unregister_chip() -- To unregister the HTE chip.
> >> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
> >>
> >> It provides below APIs for the consumer:
> >> - of_hte_request_ts() -- To request timestamp functionality.
> >> - devm_of_hte_request_ts() -- Managed version of the above.
> >> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
> >> using HTE provider dt node.
> >> - devm_hte_release_ts() -- The managed version to release timestamp
> >> functionality and associated resources.
> >> - hte_retrieve_ts_ns() -- To retrieve timestamps.
> >> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
> >> - hte_enable_ts() -- To disable timestamp functionality.
> >> - hte_disable_ts() -- To enable timestamp functionality.
> >> - hte_available_ts() -- To query available timestamp data.
> >> - hte_release_ts() -- To release timestamp functionality and its
> >> associated resources.
> >> - hte_get_clk_src_info() -- To query clock source information from
> >> the provider
> >>
> >> It provides centralized software buffer management per requested id to
> >> store the timestamp data for the consumers as below:
> >> - hte_set_buf_len() -- To set the buffer length.
> >> - hte_get_buf_len() -- To get the buffer length.
> >> - hte_set_buf_watermark() -- To set the software threshold/watermark.
> >> - hte_get_buf_watermark() -- To get the software threshold/watermark.
> >>
> >> The detail about parameters and API usage are described in each
> >> functions definitions in drivers/hte/hte.c file.
> >>
> >> The patch adds compilation support in Makefile and menu options in
> >> Kconfig.
> >>
> >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>  
> > Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
> > around what this is doing + it is an RFC :)  
> Thanks for the review comments. My responses inline.

You are welcome, some follow up responses inline.
I've tried to crop this down a bit so only kept the bits we are discussing.

> >> +
> >> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
> >> +{
> >> +	u32 ts_id;
> >> +	struct hte_device *gdev;
> >> +	struct hte_ts_info *ei;
> >> +	int ret;
> >> +
> >> +	if (!desc)
> >> +		return -EINVAL;
> >> +
> >> +	ei = (struct hte_ts_info *)desc->data_subsys;  
> > As above, no need to cast - though it rather implies the type of data_subsys
> > should not be void *.  
> 
> desc is public facing structure, I wanted to make subsystem related
> 
> information opaque that is why I had it void *.
> 

you can keep it opaque, just have a forwards definition of
struct hte_ts_desc;
which just means it is defined somewhere.  You can have that in the header with
the definition hidden away.

It will only need to have a visible complete definition when you dereference it inside
the the core.

Mind you, I'm suggesting allowing it to be embedded in another structure anyway which
would require you to have it exposed.  Perhaps this desire to keep it opaque is
a reason to not take that suggestion but it isn't relevant for this one.


> >> + */
> >> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> >> +					   const char *label,
> >> +					   void (*cb)(enum hte_notify n))
> >> +{
> >> +
> >> +	struct hte_ts_desc **ptr, *desc;
> >> +
> >> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);  
> > Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
> > and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
> > last kernel cycle.  
> I based my patches from linux-next/master. Not sure if that has
> 
> dev_add_action_or_reset

typo on my part was meant to be

devm_add_action_or_reset()

> 
> >

> >> +
> >> +/**
> >> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
> >> + * node directly, where meaning of the entity is provider specific, for example
> >> + * lines, signals, GPIOs, buses etc...
> >> + *
> >> + * @of_node: HTE provider device node.
> >> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
> >> + * @cb: Optional callback to notify.
> >> + *
> >> + * Context: Holds mutex lock, can not be called from atomic context.  
> > What mutex and why?  If it is one you can check is held even better.  
> 
> ___hte_req_ts holds the mutex lock to serialize multiple consumers
> 
> requesting same entity.

Add that detail to the comment.

> 
> >  
> >> + * Returns: ts descriptor on success or error pointers.
> >> + */
> >> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> >> +					  unsigned int id,
> >> +					  void (*cb)(enum hte_notify n))
> >> +{
> >> +	struct hte_device *gdev;
> >> +	struct hte_ts_desc *desc;
> >> +	int ret;
> >> +	u32 xlated_id;
> >> +
> >> +	gdev = of_node_to_htedevice(of_node);
> >> +	if (IS_ERR(gdev))
> >> +		return ERR_PTR(-ENOTSUPP);
> >> +
> >> +	if (!gdev->chip || !gdev->chip->ops)
> >> +		return ERR_PTR(-ENOTSUPP);
> >> +
> >> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> >> +	if (!desc) {
> >> +		ret = -ENOMEM;
> >> +		goto out_put_device;
> >> +	}  
> > Pass a desc pointer into this function rather than allocating the structure
> > in here.  That lets the caller embed that structure inside one of it's own
> > structures if it wants to, resulting in fewer small allocations which is always good.
> >
> > It's far from obvious that the caller needs to free desc.  
> 
> Are you suggesting to shift burden of allocation/deallocation (static or dynamic)
> 
> at client/consumer side?

It's been a while so I've forgotten how this works, but 'probably' yes...
If a function creates some sort of record (of fixed known size and type) then
letting that be passed in + filled in by the function is normally more efficient
than having an allocation in here.   Chances are the consumer will just have
it embedded in an existing state structure and not need to do any explicit
allocation / deallocation.  Disadvantage is you can't keep it opaque.

> 
> >  
> >> +
> >> +	desc->con_id = id;
> >> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
> >> +	if (ret < 0) {
> >> +		dev_err(gdev->chip->dev,
> >> +			"failed to xlate id: %d\n", id);
> >> +		goto out_free_desc;
> >> +	}
> >> +
> >> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> >> +	if (ret < 0) {
> >> +		dev_err(gdev->chip->dev,
> >> +			"failed to request id: %d\n", id);
> >> +		goto out_free_desc;
> >> +	}
> >> +
> >> +	return desc;
> >> +
> >> +out_free_desc:
> >> +	kfree(desc);
> >> +
> >> +out_put_device:
> >> +	return ERR_PTR(ret);
> >> +}
> >> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
> >> +
> >> +/**
> >> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
> >> + * information of the desc.
> >> + *
> >> + * @desc: ts descriptor, same as returned from request API.
> >> + *
> >> + * Context: Any context.
> >> + * Returns: 0 on success else negative error code on failure.
> >> + */
> >> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> >> +			 struct hte_clk_info *ci)
> >> +{
> >> +	struct hte_chip *chip;
> >> +	struct hte_ts_info *ei;
> >> +
> >> +	if (!desc || !desc->data_subsys || !ci) {
> >> +		pr_debug("%s:%d\n", __func__, __LINE__);
> >> +		return -EINVAL;
> >> +	}
> >> +
> >> +	ei = desc->data_subsys;
> >> +	if (!ei || !ei->gdev || !ei->gdev->chip)
> >> +		return -EINVAL;
> >> +
> >> +	chip = ei->gdev->chip;
> >> +	if (!chip->ops->get_clk_src_info)
> >> +		return -ENOTSUPP;
> >> +
> >> +	return chip->ops->get_clk_src_info(chip, ci);
> >> +}
> >> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
> >> +
> >> +static inline void hte_add_to_device_list(struct hte_device *gdev)
> >> +{
> >> +	struct hte_device *prev;  
> > Needs to take an appropriate lock as you may have concurrent calls.  
> 
> There is spin_lock held from register API from where this gets
> called.

Great. I'd missed that.

> 
> >  
> >> +
> >> +	if (list_empty(&hte_devices)) {
> >> +		list_add_tail(&gdev->list, &hte_devices);  
> > Needs a comment. I've no idea why you might want to only add it if there were
> > no other hte_devices already there.
> >  
> >> +		return;
> >> +	}
> >> +
> >> +	prev = list_last_entry(&hte_devices, struct hte_device, list);  
> > Why woud you do this?  
> 
> Thanks for pointing out. I definitely missed cleaning this up. Now, I will
> 
> remove this function in next RFC version as one line can be added directly
> 
> in register API.
> 
> >  
> >> +	list_add_tail(&gdev->list, &hte_devices);
> >> +}
> >> +
> >> +/**
> >> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
> >> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
> >> + * using this API from its ISR or atomic context.
> >> + *
> >> + * @chip: The HTE chip, used during the registration.
> >> + * @xlated_id: entity id understood by both subsystem and provider, usually this
> >> + * is obtained from xlate callback during request API.
> >> + * @data: timestamp data.
> >> + * @n: Size of the data.
> >> + *
> >> + * Context: Atomic.
> >> + * Returns: 0 on success or a negative error code on failure.
> >> + */
> >> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> >> +			  struct hte_ts_data *data, size_t n)
> >> +{
> >> +	unsigned int ret;
> >> +	bool notify;
> >> +	size_t el_avail;
> >> +	struct hte_ts_buf *buffer;
> >> +	struct hte_ts_info *ei;
> >> +
> >> +	if (!chip || !data || !chip->gdev)
> >> +		return -EINVAL;
> >> +
> >> +	if (xlated_id > chip->nlines)
> >> +		return -EINVAL;
> >> +
> >> +	ei = &chip->gdev->ei[xlated_id];
> >> +
> >> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
> >> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
> >> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
> >> +		return -EINVAL;
> >> +	}
> >> +
> >> +	/* timestamp sequence counter, start from 0 */
> >> +	data->seq = ei->seq++;
> >> +
> >> +	buffer = ei->buf;
> >> +	el_avail = buffer->access->el_available(buffer);
> >> +	ret = buffer->access->store(buffer, data, n);  
> > If we are doing this from the hte core, why is buffer definition in the scope of the
> > drivers rather than the core?  That seems backwards to me.  
> 
> I do not understand this comment. The buffer definition is in scope of hte core
> 
> as it is the only entity that manages it.

I think I figured that out later and forgot to come back and edit this comment.
However...

In that case, why is it an ops function?  Don't introduce abstraction
until you need it. Will be simpler and easier to review if you just
call those functions directly for now. e.g.

	ret = hs_ts_store_to_buf(buffer, data, n);

Chances are you'll never introduce another buffer choice.
For a long time I thought we'd have both fifo and ring options in IIO
but it turned out no one really cared. We do have an ops structure, but
that's because in IIO the buffer interface is used for two things:
1) Pushing to a kfifo that is going to userspace.
2) Pushing to a callback function owned by a consumer.
and there is a rather fiddly data demux on the front end to ensure each
of those only gets the data requested via that path - at least with timestamps
there is only one type of data!

Hmm, thinking about this raises an interesting question.
Why do we want a kfifo here at all for HTE?  You could
just call a callback function registered by the consumer of that
kfifo directly.  If that consumer then wants to buffer then of
course it can, but it not (perhaps it only cares about the latest
value and will drop the rest) then it can chose not to.  Maybe
it's just gathering stats rather than caring about individual
timestamps?  Probably lots of other things that might happen in
the consumer that I've not thought of.  We need a buffer if
userspace becomes involved, but here IIRC that's not (yet) true.

> 
> >  
> >> +	if (ret != n) {
> >> +		atomic_inc(&ei->dropped_ts);
> >> +		if (ei->cb)
> >> +			ei->cb(HTE_TS_DROPPED);
> >> +		return -ENOMEM;
> >> +	}

...

> >  
> >> +
> >> +/**
> >> + * struct hte_ts_data - HTE timestamp data.
> >> + * The provider uses and fills timestamp related details during push_timestamp
> >> + * API call. The consumer uses during retrieve_timestamp API call.
> >> + *
> >> + * @tsc: Timestamp value.
> >> + * @seq: Sequence counter of the timestamps.
> >> + * @dir: Direction of the event at the time of timestamp.
> >> + */
> >> +struct hte_ts_data {
> >> +	u64 tsc;
> >> +	u64 seq;
> >> +	int dir;
> >> +};
> >> +
> >> +/**
> >> + * struct hte_clk_info - Clock source info that HTE provider uses.
> >> + * The provider uses hardware clock as a source to timestamp real time. This
> >> + * structure presents the clock information to consumers. 
> >> + *
> >> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
> >> + * @type: Clock type. CLOCK_* types.  
> > So this is something we got a it wrong in IIO. It's much better to define
> > a subset of clocks that can be potentially used.  There are some that make
> > absolutely no sense and consumers really don't want to have to deal with them.  
> Is there anything I have to change here?

Yes - specify which clocks would make sense.  You might not need to explicitly
allow only those, but that might also be worthwhile. Otherwise, the chances are
you'll end up with a bunch of special purpose code in consumers on the basis
they might get CLOCK_TAI or similar and have to deal with it.
As for exactly which clocks do make sense, that's one which may take some figuring
out. Probably REALTIME, MONOTONIC and BOOTTIME depending on whether you care
what happens when the time of the system gets adjusted, or whether it carries
on measuring time across suspend.   Very application dependent but there are some
you can definitely rule out. Don't repeat my mistake of leaving it vague
(which incidentally was a follow up to picking a silly clock to use for timestamps
 before we allowed it to be configured).

> >    
> >> + */
> >> +struct hte_clk_info {
> >> +	u64 hz;
> >> +	clockid_t type;
> >> +};
> >> +
> >> +/**
> >> + * HTE subsystem notifications for the consumers.
> >> + *
> >> + * @HTE_TS_AVAIL: Timestamps available notification.
> >> + * @HTE_TS_DROPPED: Timestamps dropped notification.  
> > Something I've missed so far is whether drops are in a kfifo or a ring
> > fashion.  I'm guess that's stated somewhere, but it might be useful to have
> > it here.  
> Dropped are from kfifo if kfifo does not have space.

Ok, perhaps expand the comment?

...

> 
> >  
> >> + *
> >> + * xlated_id parameter is used to communicate between HTE subsystem and the
> >> + * providers. It is the same id returned during xlate API call and translated
> >> + * by the provider. This may be helpful as both subsystem and provider locate
> >> + * the requested entity in constant time, where entity could be anything from
> >> + * lines, signals, events, buses etc.. that providers support.
> >> + */
> >> +struct hte_ops {
> >> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
> >> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
> >> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
> >> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
> >> +	int (*get_clk_src_info)(struct hte_chip *chip,
> >> +				struct hte_clk_info *ci);
> >> +};
> >> +
> >> +/**
> >> + * struct hte_chip - Abstract HTE chip structure.
> >> + * @name: functional name of the HTE IP block.
> >> + * @dev: device providing the HTE.  
> > Unclear naming.  Is this the parent device, or one associated with the HTE itself?
> > I'm guessing today you don't have one associated with the HTE, but it is plausible you
> > might gain on in future to make it fit nicely in the device model as a function of another
> > device.  
> 
> This is provider's device, could be &pdev->dev or any dev provider deems fit hence the
> 
> generic name.

Ok, for now this works as a name, but I wonder if you will end up growing another
layer in the device model as would happen for majority of subsystems.
You may end up doing so when adding support to query the provider via a handle
in the dt of the consumer.  It could probably be avoided, but throwing this into
a class might make your life easier as you can use more standard infrastructure.

> 
> >  
> >> + * @ops: callbacks for this HTE.
> >> + * @nlines: number of lines/signals supported by this chip.
> >> + * @xlate: Callback which translates consumer supplied logical ids to
> >> + * physical ids, return from 0 for the success and negative for the
> >> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
> >> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
> >> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
> >> + * @data: chip specific private data.
> >> + */
> >> +struct hte_chip {
> >> +	const char *name;
> >> +	struct device *dev;
> >> +	const struct hte_ops *ops;
> >> +	u32 nlines;
> >> +	int (*xlate)(struct hte_chip *gc,
> >> +		     const struct of_phandle_args *args,
> >> +		     struct hte_ts_desc *desc, u32 *xlated_id);
> >> +	u8 of_hte_n_cells;
> >> +
> >> +	/* only used internally by the HTE framework */
> >> +	struct hte_device *gdev;
> >> +	void *data;
> >> +};
...

Jonathan

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-07-28  5:12       ` Dipen Patel
@ 2021-08-01 16:48         ` Jonathan Cameron
  0 siblings, 0 replies; 75+ messages in thread
From: Jonathan Cameron @ 2021-08-01 16:48 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt, William Breathitt Gray

On Tue, 27 Jul 2021 22:12:18 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> On 7/4/21 1:45 PM, Jonathan Cameron wrote:
> > On Sun, 4 Jul 2021 21:15:25 +0100
> > Jonathan Cameron <jic23@kernel.org> wrote:
> >  
> >> On Fri, 25 Jun 2021 16:55:23 -0700
> >> Dipen Patel <dipenp@nvidia.com> wrote:
> >>  
> >>> Some devices can timestamp system lines/signals/Buses in real-time
> >>> using the hardware counter or other hardware means which can give
> >>> finer granularity and help avoid jitter introduced by software means
> >>> of timestamping. To utilize such functionality there has to be
> >>> framework where such devices can register themselves as producers or
> >>> providers so that the consumers or clients devices can request specific
> >>> line from the providers. This patch introduces such subsystem as
> >>> hardware timestamping engine (HTE).
> >>>
> >>> It provides below APIs for the provider:
> >>> - hte_register_chip() -- To register the HTE chip.
> >>> - hte_unregister_chip() -- To unregister the HTE chip.
> >>> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
> >>>
> >>> It provides below APIs for the consumer:
> >>> - of_hte_request_ts() -- To request timestamp functionality.
> >>> - devm_of_hte_request_ts() -- Managed version of the above.
> >>> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
> >>> using HTE provider dt node.
> >>> - devm_hte_release_ts() -- The managed version to release timestamp
> >>> functionality and associated resources.
> >>> - hte_retrieve_ts_ns() -- To retrieve timestamps.
> >>> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
> >>> - hte_enable_ts() -- To disable timestamp functionality.
> >>> - hte_disable_ts() -- To enable timestamp functionality.
> >>> - hte_available_ts() -- To query available timestamp data.
> >>> - hte_release_ts() -- To release timestamp functionality and its
> >>> associated resources.
> >>> - hte_get_clk_src_info() -- To query clock source information from
> >>> the provider
> >>>
> >>> It provides centralized software buffer management per requested id to
> >>> store the timestamp data for the consumers as below:
> >>> - hte_set_buf_len() -- To set the buffer length.
> >>> - hte_get_buf_len() -- To get the buffer length.
> >>> - hte_set_buf_watermark() -- To set the software threshold/watermark.
> >>> - hte_get_buf_watermark() -- To get the software threshold/watermark.
> >>>
> >>> The detail about parameters and API usage are described in each
> >>> functions definitions in drivers/hte/hte.c file.
> >>>
> >>> The patch adds compilation support in Makefile and menu options in
> >>> Kconfig.
> >>>
> >>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>    
> >> Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
> >> around what this is doing + it is an RFC :)  
> > Having read on through the rest of the series, one of the biggest things that became
> > clear is you have more layers of abstraction in here than make sense.  Squash things
> > together so you have fewer allocations (and hence fewer error paths etc).
> > Don't introduce ops functions unless you have more than one answer to what they
> > are (and a clear justification for those).   It's easy to add layers of indirection
> > later, but for now they just make your code harder to read.  Reality is that the buffer
> > is a kfifo, make it so everywhere, rather than pretending otherwise.  
> 
> I can remove buffer abstraction for sure. Do you find any other not needed
> 
> abstractions that I can rethink of?

Ah, I should have read both replies before responding to either.

It was a while ago so I can't remember which others I noticed.

Looking quickly at it again, I think the lifetimes of the desc and buffer
are the same (note that I'd normally expect a release() to be a result
of reference counting rather that directly called which confused me a little whilst
looking at this code).  You could make that explicit and embed the buffer in
the desc.

> 
> >
> > I thought for a while that you were allowing a different buffer implementation for
> > each of the your hte chips and was very confused as to why that would make sense.
> >
> > Anyhow, it's interesting.  I'm not sure yet if the way it all fits together makes
> > sense and you will almost certainly want to support hardware FIFOs and those tend
> > to want to be partly exposed through to the consumer.  
> 
> For now I am dealing with the providers which has hardware fifo but it is
> not per event/entity i.e. it stores timestamps from all the enabled entities.
> For such providers it is not feasible to expose hardware fifo. Can you point
> to some references where it is exposed so that if needed I can accommodate
> similar changes for any future providers.

At the moment all I have to go on is what has been done in sensor hubs where it
is a single purpose timestamp. An example is drivers/iio/imu/st_lsm6dsx/
which has hardware timestamping support (and some nasty code to align that
with system timestamps).

You may still be able to do some stuff with your hwfifo.   For example if only
one event is currently enabled, you could match the hwfifo watermark to the
software one and hence enable batching of handling the timestamps.

> 
> >
> > As to similar devices.  Lots of sensorhubs have timestamping facilities but it is
> > tightly coupled to the data streams, so probably doesn't make sense to map to a
> > generic subsystem like this.  You'll hit some of the same issues as those though
> > when you try to align these timestamps with system ones etc.  
> What mechanism does sensorhub/iio now have to retrieve timestamp?

They are just another type of channel giving a measurement of time alongside
other elements of a channel scan (ADC channels etC), so for userspace you can
get them via the chrdev / buffered interface.  In kernel you can get them via
the consumer buffer interface (drivers/iio/buffers/industrialio-buffer-cb.c)
though interestingly I don't think any in kernel consumers use the timestamp channels.
The actually mangling of a hardware timestamp into one aligned with the
system timestamps is done in each driver.

> >
> > otherwise, some of the counter devices are closer to this.  Perhaps the ti ecap?
> > https://www.ti.com/lit/ug/spru807b/spru807b.pdf?ts=1625431159791&ref_url=https%253A%252F%252Fwww.google.com%252F
> >
> > That has a short hardware buffer and can be used for absolute timestamp grabbing
> > on rising triggers etc.  
> 
> I certainly have to read about this device. Initial target was to address in chip
> 
> HTEs and see (based on upstream comments) how can it be expanded to other
> 
> devices.

Understood. It's possible some of the sensor hubs will timestamp things other than
their main flow but no one has enabled driver support yet if they do. Some sensorhubs
are tightly integrated with the application processor so you may run into some of
those.

Good luck!

Jonathan


> 
> >
> > Jonathan
> >
> >  
> >>> ---
> >>>  drivers/Kconfig      |    2 +
> >>>  drivers/Makefile     |    1 +
> >>>  drivers/hte/Kconfig  |   22 +
> >>>  drivers/hte/Makefile |    1 +
> >>>  drivers/hte/hte.c    | 1368 ++++++++++++++++++++++++++++++++++++++++++
> >>>  include/linux/hte.h  |  278 +++++++++
> >>>  6 files changed, 1672 insertions(+)
> >>>  create mode 100644 drivers/hte/Kconfig
> >>>  create mode 100644 drivers/hte/Makefile
> >>>  create mode 100644 drivers/hte/hte.c
> >>>  create mode 100644 include/linux/hte.h
> >>>
> >>> diff --git a/drivers/Kconfig b/drivers/Kconfig
> >>> index 47980c6b1945..9b078964974b 100644
> >>> --- a/drivers/Kconfig
> >>> +++ b/drivers/Kconfig
> >>> @@ -238,4 +238,6 @@ source "drivers/interconnect/Kconfig"
> >>>  source "drivers/counter/Kconfig"
> >>>  
> >>>  source "drivers/most/Kconfig"
> >>> +
> >>> +source "drivers/hte/Kconfig"
> >>>  endmenu
> >>> diff --git a/drivers/Makefile b/drivers/Makefile
> >>> index 5a6d613e868d..0a996a698e4c 100644
> >>> --- a/drivers/Makefile
> >>> +++ b/drivers/Makefile
> >>> @@ -190,3 +190,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
> >>>  obj-$(CONFIG_INTERCONNECT)	+= interconnect/
> >>>  obj-$(CONFIG_COUNTER)		+= counter/
> >>>  obj-$(CONFIG_MOST)		+= most/
> >>> +obj-$(CONFIG_HTE)		+= hte/
> >>> diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
> >>> new file mode 100644
> >>> index 000000000000..394e112f7dfb
> >>> --- /dev/null
> >>> +++ b/drivers/hte/Kconfig
> >>> @@ -0,0 +1,22 @@
> >>> +# SPDX-License-Identifier: GPL-2.0-only
> >>> +menuconfig HTE
> >>> +        bool "Hardware Timestamping Engine (HTE) Support"
> >>> +        help
> >>> +          Hardware Timestamping Engine (HTE) Support.    
> >> Tidy this up, but think that's already been commented on.
> >>  
> >>> +
> >>> +          Some devices provide hardware timestamping engine which can timestamp
> >>> +	  certain device lines/signals in realtime. This way to provide
> >>> +	  hardware assisted timestamp to generic signals like GPIOs, IRQs lines
> >>> +	  comes with benefit for the applications like autonomous machines
> >>> +	  needing accurate timestamping event with less jitter.
> >>> +
> >>> +	  This framework provides a generic interface to such HTE devices
> >>> +          within the Linux kernel. It provides an API to register and
> >>> +	  unregister a HTE provider chip, configurable sw buffer to
> >>> +	  store the timestamps, push the timestamp from the HTE providers and
> >>> +	  retrieve timestamps for the consumers. It also provides means for the
> >>> +	  consumers to request signals it wishes to hardware timestamp and
> >>> +	  release them if not required.
> >>> +
> >>> +          If unsure, say no.
> >>> +
> >>> diff --git a/drivers/hte/Makefile b/drivers/hte/Makefile
> >>> new file mode 100644
> >>> index 000000000000..9899dbe516f7
> >>> --- /dev/null
> >>> +++ b/drivers/hte/Makefile
> >>> @@ -0,0 +1 @@
> >>> +obj-$(CONFIG_HTE)		+= hte.o
> >>> diff --git a/drivers/hte/hte.c b/drivers/hte/hte.c
> >>> new file mode 100644
> >>> index 000000000000..c53260d1e250
> >>> --- /dev/null
> >>> +++ b/drivers/hte/hte.c
> >>> @@ -0,0 +1,1368 @@
> >>> +// SPDX-License-Identifier: GPL-2.0
> >>> +/*
> >>> + * Copyright (c) 2021 NVIDIA Corporation
> >>> + *
> >>> + * Author: Dipen Patel <dipenp@nvidia.com>
> >>> + */
> >>> +
> >>> +#include <linux/kernel.h>
> >>> +#include <linux/module.h>
> >>> +#include <linux/err.h>
> >>> +#include <linux/slab.h>
> >>> +#include <linux/of.h>
> >>> +#include <linux/of_device.h>
> >>> +#include <linux/kfifo.h>
> >>> +#include <linux/mutex.h>
> >>> +#include <linux/sched.h>
> >>> +#include <linux/uaccess.h>
> >>> +#include <linux/hte.h>
> >>> +#include <linux/delay.h>
> >>> +#include <linux/debugfs.h>
> >>> +
> >>> +/* Global list of the HTE devices */
> >>> +static DEFINE_SPINLOCK(hte_lock);
> >>> +static LIST_HEAD(hte_devices);
> >>> +
> >>> +enum {
> >>> +	HTE_TS_REGISTERED,
> >>> +	HTE_TS_DISABLE,
> >>> +};
> >>> +
> >>> +/* Default FIFO depth */
> >>> +#define HTE_EV_FIFO_EL		32
> >>> +
> >>> +#define HTE_TS_NAME_LEN		10
> >>> +
> >>> +struct hte_ts_buf;
> >>> +
> >>> +/**
> >>> + * struct hte_ts_buf_acc_func - Software buffer management functions.
> >>> + * @store: Store timestamp from atomic context as providers most likely
> >>> + * be pushing timestamps from their interrupt handlers.
> >>> + * @read: Read timestamps from the buffer.
> >>> + * @el_available: Available timestamps to retrieve. The client can use this to
> >>> + * query available elements so that it can pre-allocate internal buffer to send
> >>> + * to during hte_retrieve_ts_ns API.
> >>> + * @set_length: Set length/depth of the buffer.
> >>> + * @get_length: Get length/depth of the buffer.
> >>> + * @set_watermark: Set software threshold of the buffer.
> >>> + * @get_watermark: Get software threshold of the buffer.
> >>> + * @release: Release/free buffer.
> >>> + * @reset: Reset the buffer.
> >>> + */
> >>> +struct hte_ts_buf_acc_func {
> >>> +	unsigned int (*store)(struct hte_ts_buf *buf, void *data, size_t n);
> >>> +	int (*read)(struct hte_ts_buf *buf, unsigned char *data, size_t n,
> >>> +		    size_t *copied);
> >>> +	size_t (*el_available)(struct hte_ts_buf *buf);
> >>> +	int (*set_length)(struct hte_ts_buf *buf,
> >>> +			  size_t length, size_t bpd);
> >>> +	size_t (*get_length)(struct hte_ts_buf *buf);
> >>> +	int (*set_watermark)(struct hte_ts_buf *buf,
> >>> +			     size_t val);
> >>> +	size_t (*get_watermark)(struct hte_ts_buf *buf);
> >>> +	void (*release)(struct hte_ts_buf *buf);
> >>> +	void (*reset)(struct hte_ts_buf *buf);
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_ts_buf - Software buffer per requested id or entity to store
> >>> + * timestamps.
> >>> + *
> >>> + * @datum_len: Buffer depth or number of elements.
> >>> + * @bytes_per_datum: Element size in bytes.
> >>> + * @watermark: Software threshold at which client will be notified.
> >>> + * @valid: Validity of the buffer.
> >>> + * @pollq: Waitqueue for the blocking clients.
> >>> + * @access: Various buffer management functions.
> >>> + */
> >>> +struct hte_ts_buf {
> >>> +	size_t datum_len;
> >>> +	size_t bytes_per_datum;
> >>> +	size_t watermark;
> >>> +	bool valid;
> >>> +	wait_queue_head_t pollq;
> >>> +	const struct hte_ts_buf_acc_func *access;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_ts_info - Information related to requested timestamp.
> >>> + *
> >>> + * @xlated_id: Timestamp ID as understood between HTE subsys and HTE provider,
> >>> + * See xlate callback API.
> >>> + * @flags: Flags holding state informations.
> >>> + * @seq: Timestamp sequence counter.
> >>> + * @dropped_ts: Dropped timestamps.
> >>> + * @cb: Callback to notify clients.
> >>> + * @mlock: Lock during timestamp request/release APIs.
> >>> + * @ts_dbg_root: Root for the debug fs.
> >>> + * @gdev: HTE abstract device that this timestamp belongs to.
> >>> + * @buf: Per requested timestamp software buffer.
> >>> + * @desc: Timestamp descriptor understood between clients and HTE subsystem.
> >>> + */
> >>> +struct hte_ts_info {
> >>> +	u32 xlated_id;
> >>> +	unsigned long flags;
> >>> +	u64 seq;
> >>> +	atomic_t dropped_ts;
> >>> +	void (*cb)(enum hte_notify n);
> >>> +	struct mutex mlock;
> >>> +	struct dentry *ts_dbg_root;
> >>> +	struct hte_device *gdev;
> >>> +	struct hte_ts_buf *buf;  
> > Where there is one instance, just embed it.  Lots of small allocations just make
> > for less readable code.  
> Which specific instance you are alluding to desc and buf types?
> >  
> >>> +	struct hte_ts_desc *desc;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_device - HTE abstract device
> >>> + * @nlines: Number of entities this device supports.
> >>> + * @ts_req: Total number of entities requested.
> >>> + * @ei: Timestamp information.
> >>> + * @sdev: Device used at various debug prints.
> >>> + * @dbg_root: Root directory for debug fs.
> >>> + * @list: List node for internal use.    
> >> Be more specific of what sort of internal use.
> >>  
> >>> + * @chip: HTE chip providing this HTE device.
> >>> + * @owner: helps prevent removal of modules when in use.
> >>> + */
> >>> +struct hte_device {
> >>> +	u32 nlines;
> >>> +	atomic_t ts_req;
> >>> +	struct hte_ts_info *ei;
> >>> +	struct device *sdev;
> >>> +	struct dentry *dbg_root;
> >>> +	struct list_head list;
> >>> +	struct hte_chip *chip;
> >>> +	struct module *owner;
> >>> +};
> >>> +
> >>> +/* Buffer management functions */
> >>> +
> >>> +/**
> >>> + * struct hte_kfifo - Software buffer wrapper.
> >>> + * @buffer: Abstract buffer device.
> >>> + * @gkf: Actual software buffer type, this case its FIFO.
> >>> + */
> >>> +struct hte_kfifo {
> >>> +	struct hte_ts_buf buffer;
> >>> +	struct kfifo gkf;
> >>> +};
> >>> +
> >>> +#define buf_to_kfifo(r) container_of(r, struct hte_kfifo, buffer)
> >>> +
> >>> +static unsigned int hte_ts_store_to_buf(struct hte_ts_buf *r, void *data,
> >>> +					size_t n)
> >>> +{
> >>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> >>> +
> >>> +	if (unlikely(!r->valid))
> >>> +		return 0;
> >>> +
> >>> +	return kfifo_in(&kf->gkf, (unsigned char *)data, n);
> >>> +}
> >>> +
> >>> +static inline int hte_ts_buf_read(struct hte_ts_buf *r,
> >>> +				  unsigned char *buf, size_t n,
> >>> +				  size_t *copied)
> >>> +{
> >>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> >>> +
> >>> +	if ((!r->valid) || (n < kfifo_esize(&kf->gkf)))
> >>> +		return -EINVAL;
> >>> +
> >>> +	*copied = kfifo_out(&kf->gkf, buf, n);
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static size_t hte_ts_buf_el_available(struct hte_ts_buf *r)
> >>> +{
> >>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> >>> +
> >>> +	if (!r->valid)
> >>> +		return 0;
> >>> +
> >>> +	return (kfifo_len(&kf->gkf) / r->bytes_per_datum);
> >>> +}
> >>> +
> >>> +static int hte_ts_buf_set_length(struct hte_ts_buf *r,
> >>> +				 size_t length, size_t bpd)
> >>> +{
> >>> +	int ret = 0;
> >>> +	struct hte_kfifo *buf;
> >>> +
> >>> +	if ((length == 0) || (bpd == 0) || !r)
> >>> +		return -EINVAL;
> >>> +
> >>> +	buf = buf_to_kfifo(r);
> >>> +
> >>> +	if (r->datum_len != length) {
> >>> +		if (r->valid)
> >>> +			kfifo_free(&buf->gkf);
> >>> +		r->valid = false;
> >>> +		r->datum_len = length;
> >>> +		r->bytes_per_datum = bpd;
> >>> +		ret = kfifo_alloc(&buf->gkf, length * bpd, GFP_KERNEL);
> >>> +		if (!ret)
> >>> +			r->valid = true;
> >>> +	}
> >>> +
> >>> +	return ret;
> >>> +}
> >>> +
> >>> +static inline size_t hte_ts_buf_get_length(struct hte_ts_buf *r)
> >>> +{
> >>> +	if ((!r->valid) || !r->datum_len)
> >>> +		return 0;
> >>> +
> >>> +	return r->datum_len;
> >>> +}
> >>> +
> >>> +static inline int hte_ts_buf_set_watermark(struct hte_ts_buf *r, size_t val)
> >>> +{
> >>> +	if ((!r->valid) || (val > r->datum_len))
> >>> +		return -EINVAL;
> >>> +
> >>> +	r->watermark = val;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static inline size_t hte_ts_buf_get_watermark(struct hte_ts_buf *r)
> >>> +{
> >>> +	if (!r->valid)
> >>> +		return 0;
> >>> +
> >>> +	return r->watermark;
> >>> +}
> >>> +
> >>> +static inline void hte_ts_buf_release(struct hte_ts_buf *r)
> >>> +{
> >>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> >>> +
> >>> +	r->valid = false;
> >>> +	kfifo_free(&kf->gkf);
> >>> +	kfree(kf);
> >>> +}
> >>> +
> >>> +static inline void hte_ts_buf_reset(struct hte_ts_buf *r)
> >>> +{
> >>> +	struct hte_kfifo *kf = buf_to_kfifo(r);
> >>> +
> >>> +	if (!r->valid)
> >>> +		return;
> >>> +
> >>> +	kfifo_reset(&kf->gkf);
> >>> +}
> >>> +
> >>> +static const struct hte_ts_buf_acc_func kfifo_access_funcs = {
> >>> +	.store = &hte_ts_store_to_buf,
> >>> +	.read = &hte_ts_buf_read,
> >>> +	.el_available = &hte_ts_buf_el_available,
> >>> +	.set_length = &hte_ts_buf_set_length,
> >>> +	.get_length = &hte_ts_buf_get_length,
> >>> +	.set_watermark = &hte_ts_buf_set_watermark,
> >>> +	.get_watermark = &hte_ts_buf_get_watermark,
> >>> +	.release = &hte_ts_buf_release,
> >>> +	.reset = &hte_ts_buf_reset,
> >>> +};
> >>> +
> >>> +static struct hte_ts_buf *hte_ts_buf_allocate(void)
> >>> +{
> >>> +	struct hte_kfifo *kf;
> >>> +
> >>> +	kf = kzalloc(sizeof(*kf), GFP_KERNEL);
> >>> +	if (!kf)
> >>> +		return ERR_PTR(-ENOMEM);
> >>> +
> >>> +	init_waitqueue_head(&kf->buffer.pollq);
> >>> +	kf->buffer.watermark = 1;
> >>> +	kf->buffer.datum_len = 0;
> >>> +	kf->buffer.valid = false;
> >>> +	kf->buffer.access = &kfifo_access_funcs;  
> > Why do you have this level abstraction?  I would suggest you flatten all this
> > into direct calls until you have some clear need for multiple buffer types.
> >
> > A long long time ago (10 years or more) we had similar abstractions in IIO because
> > we though it would be helpful to support ring buffers and kfifos.  It wasn't, we ended
> > up ripping them all out because all they resulted in was a more complex code base
> > when in reality everyone was happy with a kfifo.  
> 
> I agree. Initially I thought future providers may need different type of buffers so had
> 
> abstraction layer. But I guess its ok to not worry about that for now. I will remove this
> 
> in next RFC version.
> 
> >
> > Jonathan
> >
> >  
> >>> +
> >>> +	return &kf->buffer;
> >>> +}
> >>> +/* End of buffer management */
> >>> +
> >>> +/* Debugfs management */
> >>> +
> >>> +#ifdef CONFIG_DEBUG_FS
> >>> +
> >>> +static struct dentry *hte_root;
> >>> +
> >>> +static void __init hte_subsys_dbgfs_init(void)
> >>> +{
> >>> +	/* creates /sys/kernel/debug/hte/ */
> >>> +	hte_root = debugfs_create_dir("hte", NULL);
> >>> +}
> >>> +subsys_initcall(hte_subsys_dbgfs_init);
> >>> +
> >>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> >>> +{
> >>> +	const struct hte_chip *chip = gdev->chip;
> >>> +	const char *name = chip->name ? chip->name : dev_name(chip->dev);
> >>> +
> >>> +	gdev->dbg_root = debugfs_create_dir(name, hte_root);
> >>> +	if (!gdev->dbg_root)
> >>> +		return;
> >>> +
> >>> +	debugfs_create_atomic_t("ts_requested", 0444, gdev->dbg_root,
> >>> +				&gdev->ts_req);
> >>> +	debugfs_create_u32("total_ts", 0444, gdev->dbg_root,
> >>> +			   &gdev->nlines);
> >>> +}
> >>> +
> >>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> >>> +{
> >>> +	if (!ei->gdev->dbg_root || !name)
> >>> +		return;
> >>> +
> >>> +	ei->ts_dbg_root = debugfs_create_dir(name, ei->gdev->dbg_root);
> >>> +	if (!ei->ts_dbg_root)
> >>> +		return;
> >>> +
> >>> +	debugfs_create_size_t("ts_buffer_depth", 0444, ei->ts_dbg_root,
> >>> +			      &ei->buf->datum_len);
> >>> +	debugfs_create_size_t("ts_buffer_watermark", 0444, ei->ts_dbg_root,
> >>> +			      &ei->buf->watermark);
> >>> +	debugfs_create_atomic_t("dropped_timestamps", 0444, ei->ts_dbg_root,
> >>> +				&ei->dropped_ts);
> >>> +}
> >>> +
> >>> +static inline void hte_dbgfs_deinit(struct dentry *root)
> >>> +{
> >>> +	if (!root)
> >>> +		return;
> >>> +
> >>> +	debugfs_remove_recursive(root);
> >>> +}
> >>> +
> >>> +#else
> >>> +
> >>> +static void hte_chip_dbgfs_init(struct hte_device *gdev)
> >>> +{
> >>> +}
> >>> +
> >>> +static void hte_ts_dbgfs_init(const char *name, struct hte_ts_info *ei)
> >>> +{
> >>> +}
> >>> +
> >>> +static inline void hte_dbgfs_deinit(struct dentry *root)
> >>> +{
> >>> +}
> >>> +
> >>> +#endif
> >>> +/* end of debugfs management*/
> >>> +
> >>> +/* Driver APIs */
> >>> +
> >>> +/**
> >>> + * hte_release_ts() - Consumer calls this API to release the entity, where
> >>> + * entity could be anything providers support, like lines, signals, buses,
> >>> + * etc...
> >>> + *
> >>> + * The correct sequence to call this API is as below:
> >>> + * 1) Call hte_disable_ts, this stops the timestamp push from the provider.
> >>> + * 2) Retrieve timestamps by calling non blocking hte_retrieve_ts_ns API if you
> >>> + * still care about the data.
> >>> + * 3) Call this API.
> >>> + * Above sequence makes sure that entity gets released race free.
> >>> + *
> >>> + * @desc: timestamp descriptor, this is the same as returned by the request API.
> >>> + *
> >>> + * Context: hte_dbgfs_deinit() function call may use sleeping locks,
> >>> + *	    not suitable from atomic context in that case.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_release_ts(struct hte_ts_desc *desc)
> >>> +{
> >>> +	u32 id;
> >>> +	int ret = 0;
> >>> +	struct hte_device *gdev;
> >>> +	struct hte_ts_info *ei;
> >>> +	struct hte_ts_buf *buf;
> >>> +
> >>> +	if (!desc)
> >>> +		return -EINVAL;
> >>> +
> >>> +	ei = (struct hte_ts_info *)desc->data_subsys;    
> >> As data_subsys is void * you don't need to explicitly cast it to another pointer type.
> >>  
> >>> +
> >>> +	if (!ei || !ei->gdev || !ei->buf)
> >>> +		return -EINVAL;
> >>> +
> >>> +	gdev = ei->gdev;
> >>> +	buf = ei->buf;
> >>> +	id = desc->con_id;
> >>> +
> >>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> >>> +		dev_info(gdev->sdev, "id:%d is not registered", id);
> >>> +		return -EUSERS;
> >>> +	}
> >>> +
> >>> +	ret = gdev->chip->ops->release(gdev->chip, ei->xlated_id);
> >>> +	if (ret) {
> >>> +		dev_err(gdev->sdev, "id: %d free failed\n", id);
> >>> +		goto out;
> >>> +	}
> >>> +
> >>> +	atomic_dec(&gdev->ts_req);
> >>> +	atomic_set(&ei->dropped_ts, 0);
> >>> +
> >>> +	kfree(desc->name);
> >>> +	kfree(desc);
> >>> +	ei->desc = NULL;
> >>> +	ei->seq = 0;
> >>> +	buf->access->release(buf);
> >>> +
> >>> +	hte_dbgfs_deinit(ei->ts_dbg_root);
> >>> +	module_put(gdev->owner);
> >>> +
> >>> +	clear_bit(HTE_TS_REGISTERED, &ei->flags);
> >>> +
> >>> +out:
> >>> +	dev_dbg(gdev->sdev, "%s: id: %d\n", __func__, id);
> >>> +	return ret;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_release_ts);
> >>> +
> >>> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
> >>> +{
> >>> +	u32 ts_id;
> >>> +	struct hte_device *gdev;
> >>> +	struct hte_ts_info *ei;
> >>> +	int ret;
> >>> +
> >>> +	if (!desc)
> >>> +		return -EINVAL;
> >>> +
> >>> +	ei = (struct hte_ts_info *)desc->data_subsys;    
> >> As above, no need to cast - though it rather implies the type of data_subsys
> >> should not be void *.
> >>  
> >>> +
> >>> +	if (!ei || !ei->gdev)
> >>> +		return -EINVAL;
> >>> +
> >>> +	gdev = ei->gdev;
> >>> +	ts_id = desc->con_id;
> >>> +
> >>> +	mutex_lock(&ei->mlock);
> >>> +
> >>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> >>> +		dev_dbg(gdev->sdev, "id:%d is not registered", ts_id);
> >>> +		ret = -EUSERS;
> >>> +		goto out;
> >>> +	}
> >>> +
> >>> +	if (en) {
> >>> +		if (!test_bit(HTE_TS_DISABLE, &ei->flags)) {
> >>> +			ret = 0;
> >>> +			goto out;
> >>> +		}
> >>> +		ret = gdev->chip->ops->enable(gdev->chip, ei->xlated_id);
> >>> +		if (ret) {
> >>> +			dev_warn(gdev->sdev, "id: %d enable failed\n",
> >>> +				 ts_id);
> >>> +			goto out;
> >>> +		}
> >>> +
> >>> +		clear_bit(HTE_TS_DISABLE, &ei->flags);
> >>> +		ret = 0;    
> >> ret is already 0 so no point in setting it again.
> >>  
> >>> +	} else {
> >>> +		if (test_bit(HTE_TS_DISABLE, &ei->flags)) {
> >>> +			ret = 0;
> >>> +			goto out;
> >>> +		}
> >>> +		ret = gdev->chip->ops->disable(gdev->chip, ei->xlated_id);
> >>> +		if (ret) {
> >>> +			dev_warn(gdev->sdev, "id: %d disable failed\n",
> >>> +				 ts_id);
> >>> +			goto out;
> >>> +		}
> >>> +
> >>> +		set_bit(HTE_TS_DISABLE, &ei->flags);
> >>> +		ret = 0;
> >>> +	}
> >>> +
> >>> +out:
> >>> +	mutex_unlock(&ei->mlock);
> >>> +	return ret;
> >>> +}
> >>> +
> >>> +/**
> >>> + * hte_disable_ts() - Disable timestamp on given descriptor.
> >>> + *
> >>> + * @desc: ts descriptor, this is the same as returned by the request API.
> >>> + *
> >>> + * Context: Holds mutex lock, not suitable from atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_disable_ts(struct hte_ts_desc *desc)
> >>> +{
> >>> +	return hte_ts_dis_en_common(desc, false);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_disable_ts);
> >>> +
> >>> +/**
> >>> + * hte_enable_ts() - Enable timestamp on given descriptor.
> >>> + *
> >>> + * @desc: ts descriptor, this is the same as returned by the request API.
> >>> + *
> >>> + * Context: Holds mutex lock, not suitable from atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_enable_ts(struct hte_ts_desc *desc)
> >>> +{
> >>> +	return hte_ts_dis_en_common(desc, true);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_enable_ts);
> >>> +
> >>> +static int hte_simple_xlate(struct hte_chip *gc,
> >>> +			    const struct of_phandle_args *args,
> >>> +			    struct hte_ts_desc *desc,
> >>> +			    u32 *id)
> >>> +{
> >>> +	if (!id || !desc || !gc)
> >>> +		return -EINVAL;
> >>> +
> >>> +	/*
> >>> +	 * For the providers which do not have any internal mappings between
> >>> +	 * logically exposed ids and actual ids, will set both
> >>> +	 * the same.
> >>> +	 *
> >>> +	 * In case there is a internal mapping needed, providers will need to
> >>> +	 * provide its own xlate function where con_id will be sent as
> >>> +	 * args[0] and it will return xlated id. Later xlated id will be
> >>> +	 * used for any future exchanges between provider and subsystems.
> >>> +	 */
> >>> +
> >>> +	if (args) {
> >>> +		if (gc->of_hte_n_cells < 1)
> >>> +			return -EINVAL;
> >>> +
> >>> +		if (args->args_count != gc->of_hte_n_cells)
> >>> +			return -EINVAL;
> >>> +
> >>> +		*id = args->args[0];
> >>> +		desc->con_id = *id;
> >>> +	} else {
> >>> +		*id = desc->con_id;
> >>> +	}
> >>> +
> >>> +	if (desc->con_id > gc->nlines)
> >>> +		return -EINVAL;
> >>> +
> >>> +	desc->data_subsys = NULL;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static struct hte_device *of_node_to_htedevice(struct device_node *np)
> >>> +{
> >>> +	struct hte_device *gdev;
> >>> +
> >>> +	spin_lock(&hte_lock);
> >>> +
> >>> +	list_for_each_entry(gdev, &hte_devices, list)
> >>> +		if (gdev->chip && gdev->chip->dev &&
> >>> +		    gdev->chip->dev->of_node == np) {
> >>> +			spin_unlock(&hte_lock);
> >>> +			return gdev;
> >>> +		}
> >>> +
> >>> +	spin_unlock(&hte_lock);
> >>> +
> >>> +	return ERR_PTR(-ENODEV);
> >>> +}
> >>> +
> >>> +static int ___hte_req_ts(struct hte_device *gdev, struct hte_ts_desc *desc,
> >>> +			 u32 xlated_id, void (*cb)(enum hte_notify n))
> >>> +{
> >>> +	struct hte_ts_info *ei;
> >>> +	struct hte_ts_buf *buf;
> >>> +	int ret;
> >>> +	u32 con_id = desc->con_id;
> >>> +
> >>> +	if (!try_module_get(gdev->owner))
> >>> +		return -ENODEV;
> >>> +
> >>> +	ei = &gdev->ei[xlated_id];
> >>> +	ei->xlated_id = xlated_id;
> >>> +
> >>> +	/*
> >>> +	 * There a chance that multiple consumers requesting same entity,
> >>> +	 * lock here.
> >>> +	 */
> >>> +	mutex_lock(&ei->mlock);
> >>> +
> >>> +	if (test_bit(HTE_TS_REGISTERED, &ei->flags)) {
> >>> +		dev_dbg(gdev->chip->dev, "id:%u is already registered",
> >>> +			xlated_id);
> >>> +		ret = -EUSERS;
> >>> +		goto unlock;
> >>> +	}
> >>> +
> >>> +	buf = hte_ts_buf_allocate();
> >>> +	if (IS_ERR(buf)) {
> >>> +		dev_err(gdev->chip->dev, "Buffer allocation failed");
> >>> +		ret = PTR_ERR(buf);
> >>> +		goto unlock;
> >>> +	}
> >>> +
> >>> +	/* Set default here, let consumer decide how much to set later */
> >>> +	ret = buf->access->set_length(buf, HTE_EV_FIFO_EL,
> >>> +				      sizeof(struct hte_ts_data));
> >>> +    
> >> It's good to keep to consistent style of no line break between a statement
> >> and it's error check.
> >>  
> >>> +	if (ret) {
> >>> +		dev_err(gdev->chip->dev, "Fifo set length failed");
> >>> +		goto buf_rel;
> >>> +	}
> >>> +
> >>> +	buf->access->reset(buf);
> >>> +	buf->valid = true;
> >>> +
> >>> +	ei->buf = buf;
> >>> +	ei->cb = cb;
> >>> +
> >>> +	ret = gdev->chip->ops->request(gdev->chip, xlated_id);
> >>> +	if (ret < 0) {
> >>> +		dev_err(gdev->chip->dev, "ts request failed\n");
> >>> +		goto buf_rel;
> >>> +	}
> >>> +
> >>> +	desc->data_subsys = ei;
> >>> +	ei->desc = desc;
> >>> +
> >>> +	atomic_inc(&gdev->ts_req);
> >>> +	set_bit(HTE_TS_REGISTERED, &ei->flags);
> >>> +	mutex_unlock(&ei->mlock);
> >>> +
> >>> +	if (!desc->name) {
> >>> +		desc->name = kzalloc(HTE_TS_NAME_LEN, GFP_KERNEL);
> >>> +		if (desc->name)
> >>> +			scnprintf(desc->name, HTE_TS_NAME_LEN, "ts_%u",
> >>> +				  con_id);
> >>> +	}
> >>> +
> >>> +	hte_ts_dbgfs_init(desc->name, ei);
> >>> +
> >>> +	dev_dbg(gdev->chip->dev, "%s: id: %u, xlated id:%u",
> >>> +		__func__, con_id, xlated_id);
> >>> +
> >>> +	return 0;
> >>> +
> >>> +buf_rel:
> >>> +	buf->access->release(buf);
> >>> +unlock:
> >>> +	module_put(gdev->owner);
> >>> +	mutex_unlock(&ei->mlock);
> >>> +
> >>> +	return ret;
> >>> +}
> >>> +
> >>> +static struct hte_device *of_hte_dev_get(struct device *dev,
> >>> +					 struct device_node *np,
> >>> +					 const char *label,
> >>> +					 struct of_phandle_args *args)
> >>> +{
> >>> +	struct hte_device *gdev = NULL;
> >>> +	int index = 0;
> >>> +	int err;
> >>> +
> >>> +	if (label) {
> >>> +		index = of_property_match_string(np, "hte-names", label);
> >>> +		if (index < 0)
> >>> +			return ERR_PTR(index);
> >>> +	}
> >>> +
> >>> +	err = of_parse_phandle_with_args(np, "htes", "#hte-cells", index,
> >>> +					 args);
> >>> +	if (err) {
> >>> +		pr_err("%s(): can't parse \"htes\" property\n", __func__);
> >>> +		return ERR_PTR(err);
> >>> +	}
> >>> +
> >>> +	gdev = of_node_to_htedevice(args->np);
> >>> +	if (IS_ERR(gdev)) {
> >>> +		pr_err("%s(): HTE chip not found\n", __func__);
> >>> +		of_node_put(args->np);
> >>> +		return gdev;
> >>> +	}
> >>> +
> >>> +	return gdev;
> >>> +}
> >>> +
> >>> +static struct hte_ts_desc *__hte_req_ts(struct device *dev,
> >>> +					struct device_node *np,
> >>> +					const char *label,
> >>> +					void (*cb)(enum hte_notify n))
> >>> +{
> >>> +	struct hte_device *gdev = NULL;
> >>> +	struct hte_ts_desc *desc;
> >>> +	struct of_phandle_args args;
> >>> +	int ret;
> >>> +	u32 xlated_id;
> >>> +
> >>> +	gdev = of_hte_dev_get(dev, np, label, &args);
> >>> +	if (IS_ERR(gdev))
> >>> +		return ERR_CAST(gdev);
> >>> +
> >>> +	if (!gdev->chip) {
> >>> +		pr_debug("requested id does not have provider\n");
> >>> +		return ERR_PTR(-ENODEV);
> >>> +	}
> >>> +
> >>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> >>> +	if (!desc)
> >>> +		return ERR_PTR(-ENOMEM);
> >>> +
> >>> +	ret = gdev->chip->xlate(gdev->chip, &args, desc, &xlated_id);
> >>> +	if (ret < 0)
> >>> +		goto put;
> >>> +
> >>> +	desc->name = NULL;
> >>> +	if (label)
> >>> +		desc->name = kstrdup(label, GFP_KERNEL);
> >>> +
> >>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> >>> +	if (ret < 0)
> >>> +		goto put;
> >>> +
> >>> +	return desc;
> >>> +
> >>> +put:
> >>> +	of_node_put(args.np);
> >>> +	kfree(desc);
> >>> +
> >>> +	return ERR_PTR(ret);
> >>> +}
> >>> +
> >>> +/**
> >>> + * of_hte_request_ts() - Consumer calls this API to request the HTE facility
> >>> + * on the specified entity, where entity is provider specific for example,
> >>> + * GPIO lines, signals, buses etc...
> >>> + *
> >>> + * @dev: Consumer device.
> >>> + * @label: Optional label.
> >>> + * @cb: Optional notify callback to consumer when data is pushed by the
> >>> + * provider.
> >>> + *
> >>> + * Context: Holds mutex lock, not suitable from atomic context.
> >>> + * Returns: Timestamp descriptor on success or error ptr on failure.
> >>> + */
> >>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev,
> >>> +				      const char *label,
> >>> +				      void (*cb)(enum hte_notify n))
> >>> +{
> >>> +
> >>> +	if (dev && dev->of_node)
> >>> +		return __hte_req_ts(dev, dev->of_node, label, cb);
> >>> +	else
> >>> +		return ERR_PTR(-EOPNOTSUPP);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(of_hte_request_ts);
> >>> +
> >>> +static int devm_hte_ts_match_desc(struct device *dev, void *res, void *data)    
> >> I'm not seeing what is devm about this.
> >>  
> >>> +{
> >>> +	struct hte_ts_desc **p = res;
> >>> +
> >>> +	if (WARN_ON(!p || !*p))
> >>> +		return 0;
> >>> +
> >>> +	return *p == data;
> >>> +}
> >>> +
> >>> +static void __devm_hte_release_ts(struct device *dev, void *res)
> >>> +{
> >>> +	hte_release_ts(*(struct hte_ts_desc **)res);
> >>> +}
> >>> +
> >>> +/**
> >>> + * devm_hte_release_ts() - Resource managed hte_release_ts().    
> >> I'd not introduce this until you have a user.  It very rarely actually makes
> >> sense to call a devm release manually. Not having one makes people think harder
> >> about it.
> >>  
> >>> + * @dev: HTE consumer/client device.
> >>> + * @desc: HTE ts descriptor.
> >>> + *
> >>> + * Release timestamp functionality and its resources previously allocated using
> >>> + * of_hte_request_ts(). Calling this function is usually not needed because
> >>> + * devm-allocated resources are automatically released on driver detach.
> >>> + *
> >>> + * Context: Same as hte_release_ts() function.
> >>> + * Returns: 0 on success otherwise negative error code.
> >>> + */
> >>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc)
> >>> +{
> >>> +	return devres_release(dev, __devm_hte_release_ts,
> >>> +			      devm_hte_ts_match_desc, desc);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(devm_hte_release_ts);
> >>> +
> >>> +/**
> >>> + * devm_of_hte_request_ts() - Resource managed of_hte_request_ts().    
> >> If it's kernel-doc it needs to give no warnings when you point the kernel-doc
> >> scripts at it.  They insist on full parameter documentation.
> >>  
> >>> + */
> >>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> >>> +					   const char *label,
> >>> +					   void (*cb)(enum hte_notify n))
> >>> +{
> >>> +
> >>> +	struct hte_ts_desc **ptr, *desc;
> >>> +
> >>> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);    
> >> Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
> >> and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
> >> last kernel cycle.
> >>
> >>  
> >>> +	if (!ptr)
> >>> +		return ERR_PTR(-ENOMEM);
> >>> +
> >>> +	desc = of_hte_request_ts(dev, label, cb);
> >>> +	if (!IS_ERR(desc)) {
> >>> +		*ptr = desc;
> >>> +		devres_add(dev, ptr);
> >>> +	} else {
> >>> +		devres_free(ptr);
> >>> +	}
> >>> +	return desc;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(devm_of_hte_request_ts);
> >>> +
> >>> +static struct hte_ts_info *hte_para_check(const struct hte_ts_desc *desc,
> >>> +					  size_t val)    
> >> Not a good name or indeed combination of different things.
> >> hte_desc_to_info() and some separate check on val would be better.
> >>  
> >>> +{
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	if (!desc || !desc->data_subsys || !val) {
> >>> +		pr_debug("%s:%d: val :%lu\n", __func__, __LINE__, val);
> >>> +		return NULL;
> >>> +	}
> >>> +
> >>> +	ei = desc->data_subsys;
> >>> +	if (!ei || !ei->buf) {
> >>> +		pr_debug("%s:%d\n", __func__, __LINE__);
> >>> +		return NULL;
> >>> +	}
> >>> +
> >>> +	return ei;
> >>> +}
> >>> +
> >>> +static inline bool hte_ts_buf_wait(struct hte_ts_buf *buffer, size_t to_read)
> >>> +{
> >>> +	size_t el_avail;
> >>> +
> >>> +	el_avail = buffer->access->el_available(buffer);
> >>> +
> >>> +	return (el_avail >= to_read) ? false : true;    
> >> return el_avail < to_read;
> >>  
> >>> +}
> >>> +
> >>> +static int _hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> >>> +			       struct hte_ts_data *el, size_t n, bool block)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +	int ret;
> >>> +	size_t to_read, copied;
> >>> +
> >>> +	ei = hte_para_check(desc, n);
> >>> +	if (!ei)
> >>> +		return -EINVAL;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +
> >>> +	to_read = min_t(size_t, n, buffer->watermark);    
> >> Needs a comment as not obvious why you'd read the min of that requested or
> >> the watermark if there might be more available.
> >>  
> >>> +
> >>> +	do {
> >>> +		if (hte_ts_buf_wait(buffer, to_read)) {
> >>> +			if (!block) {
> >>> +				/* Possibly early here to retrieve, try again */
> >>> +				dev_dbg(ei->gdev->chip->dev, "%s: %d\n",
> >>> +					__func__, ret);
> >>> +				return -EAGAIN;
> >>> +			}
> >>> +			ret = wait_event_interruptible(buffer->pollq,
> >>> +					!hte_ts_buf_wait(buffer, to_read));
> >>> +			if (ret)
> >>> +				return ret;
> >>> +		}
> >>> +		ret = buffer->access->read(buffer, (void *)el,    
> >> If you have to cast to a void * that usually means something is wrong in your definitions.
> >> Why is it needed here?  Looks like read has an inappropriate definition.
> >>  
> >>> +					   n * buffer->bytes_per_datum,
> >>> +					   &copied);
> >>> +		if (ret < 0)
> >>> +			return ret;
> >>> +
> >>> +		if (copied > 0)
> >>> +			return 0;
> >>> +		else if (copied == 0 && !block)
> >>> +			return -EAGAIN;
> >>> +	} while (copied == 0);
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +/**
> >>> + * hte_retrieve_ts_ns() - Consumer calls this API to retrieve timestamp in
> >>> + * nano seconds i.e. el->tsc will be in ns.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + * @el: buffer to store the timestamp details.
> >>> + * @n: Number of struct hte_timestamp_el elements.
> >>> + *
> >>> + * Context: Can be called from the atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc,
> >>> +		       struct hte_ts_data *el, size_t n)
> >>> +{
> >>> +	return _hte_retrieve_ts_ns(desc, el, n, false);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns);
> >>> +
> >>> +/**
> >>> + * hte_retrieve_ts_ns_wait() - Blocking version of the hte_retrieve_ts_ns.
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + * @el: buffer to store the timestamp data.
> >>> + * @n: Number of struct hte_ts_data data.
> >>> + *
> >>> + * Context: Can not be called from the atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> >>> +			    struct hte_ts_data *el, size_t n)
> >>> +{
> >>> +	return _hte_retrieve_ts_ns(desc, el, n, true);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_retrieve_ts_ns_wait);
> >>> +
> >>> +/**
> >>> + * hte_set_buf_len() - Consumer calls this API to set timestamp software buffer
> >>> + * depth.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + * @len: New length/depth.
> >>> + *
> >>> + * The correct sequence to set buffer length is as below:
> >>> + * 1) Disable timestamp by calling hte_disable_ts API.
> >>> + * 2) Optionally retrieve all the timestamps by calling non blocking
> >>> + *    hte_retrieve_ts_ns() API. This step only needed if you still care about
> >>> + *    the data.
> >>> + * 3) Call this API.
> >>> + * 4) Enable timestamp by calling hte_enable_ts API.
> >>> + *
> >>> + * This API destroys previously allocated buffer and creates new one, because
> >>> + * of that, it is mandatory to follow above sequence to make sure there is no
> >>> + * race between various other APIs in the subsystem.    
> >> Good docs.  This is why I mentioned in review of docs patch that it is better
> >> to just have that refer to the kernel-doc in these files.  Keep all this good
> >> information in one place.
> >>  
> >>> + *
> >>> + * By default during the request API call, HTE subsystem allocates software
> >>> + * buffer with predefined length, this API gives flexibility to adjust the
> >>> + * length according to consumer's need.
> >>> + *
> >>> + * Context: Can not be called from atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +	int ret;
> >>> +
> >>> +	ei = hte_para_check(desc, len);
> >>> +	if (!ei)
> >>> +		return -EINVAL;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +	ret = buffer->access->set_length(buffer, len,
> >>> +					 sizeof(struct hte_ts_data));
> >>> +	if (ret)
> >>> +		dev_err(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);    
> >> Not point in printing things line __func__ manually in dev_err() etc.
> >> Dynamic debug includes that and gives far more information + control of this.
> >>  
> >>> +
> >>> +	return ret;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_set_buf_len);
> >>> +
> >>> +/**
> >>> + * hte_get_buf_len() - Consumer calls this API to get timestamp software buffer
> >>> + * depth or length.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + *
> >>> + * Context: Any context.
> >>> + * Returns: Positive length on success or 0 on failure.
> >>> + */
> >>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	ei = hte_para_check(desc, 1);
> >>> +	if (!ei)
> >>> +		return 0;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +
> >>> +	return buffer->access->get_length(buffer);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_get_buf_len);
> >>> +
> >>> +/**
> >>> + * hte_available_ts() - Returns total available timestamps.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + *
> >>> + * The API helps consumers to pre-allocate its internal buffer required
> >>> + * during hte_retrieve_ts_ns call.
> >>> + *
> >>> + * Context: Any context.
> >>> + * Returns: Positive value if elements are available else 0. The value is
> >>> + * number of total available struct hte_timestamp_el elements available not
> >>> + * the size in bytes.
> >>> + */
> >>> +size_t hte_available_ts(const struct hte_ts_desc *desc)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	ei = hte_para_check(desc, 1);
> >>> +	if (!ei)
> >>> +		return 0;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +
> >>> +	return buffer->access->el_available(buffer);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_available_ts);
> >>> +
> >>> +/**
> >>> + * hte_set_buf_watermark() - Consumer calls this API to set timestamp software
> >>> + * buffer watermark. The correct sequence to call this API is as below:
> >>> + * 1) Disable timestamp by calling hte_disable_ts API.
> >>> + * 2) Call this API.
> >>> + * 3) Enable timestamp by calling hte_enable_ts API.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + * @val: New watermark.
> >>> + *
> >>> + * By default during the request API call, HTE subsystem sets watermark as 1,
> >>> + * this API gives flexibility to adjust the watermark according to consumer's
> >>> + * need. The consumers will get notification through callback registered during
> >>> + * request API either when timestamp is dropped or watermark is reached or will
> >>> + * wait till watermark is reached. Refer hte_retrieve_ts_ns() and
> >>> + * hte_push_ts_ns_atomic() APIs to understand how watermark is used.
> >>> + *
> >>> + * Context: Any context.    
> >> You have no way of knowing that as will depend on the driver - I'd definitely
> >> suggest not from atomic context, but then that would be crazy so you are better
> >>  off not documenting any specific requirement at all.
> >>  
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +	int ret;
> >>> +
> >>> +	ei = hte_para_check(desc, val);
> >>> +	if (!ei)
> >>> +		return -EINVAL;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +	ret = buffer->access->set_watermark(buffer, val);
> >>> +	if (ret)
> >>> +		dev_dbg(ei->gdev->chip->dev, "%s: ret:%d\n", __func__, ret);
> >>> +
> >>> +	return ret;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_set_buf_watermark);
> >>> +
> >>> +/**
> >>> + * hte_get_buf_watermark() - Consumer calls this API to get software
> >>> + * buffer watermark.
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + *
> >>> + * Context: Any context.
> >>> + * Returns: Positive current watermark on success or 0 on failure.
> >>> + */
> >>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc)
> >>> +{
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	ei = hte_para_check(desc, 1);
> >>> +	if (!ei)
> >>> +		return 0;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +
> >>> +	return buffer->access->get_watermark(buffer);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_get_buf_watermark);
> >>> +
> >>> +/**
> >>> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
> >>> + * node directly, where meaning of the entity is provider specific, for example
> >>> + * lines, signals, GPIOs, buses etc...
> >>> + *
> >>> + * @of_node: HTE provider device node.
> >>> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
> >>> + * @cb: Optional callback to notify.
> >>> + *
> >>> + * Context: Holds mutex lock, can not be called from atomic context.    
> >> What mutex and why?  If it is one you can check is held even better.
> >>  
> >>> + * Returns: ts descriptor on success or error pointers.
> >>> + */
> >>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> >>> +					  unsigned int id,
> >>> +					  void (*cb)(enum hte_notify n))
> >>> +{
> >>> +	struct hte_device *gdev;
> >>> +	struct hte_ts_desc *desc;
> >>> +	int ret;
> >>> +	u32 xlated_id;
> >>> +
> >>> +	gdev = of_node_to_htedevice(of_node);
> >>> +	if (IS_ERR(gdev))
> >>> +		return ERR_PTR(-ENOTSUPP);
> >>> +
> >>> +	if (!gdev->chip || !gdev->chip->ops)
> >>> +		return ERR_PTR(-ENOTSUPP);
> >>> +
> >>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
> >>> +	if (!desc) {
> >>> +		ret = -ENOMEM;
> >>> +		goto out_put_device;
> >>> +	}    
> >> Pass a desc pointer into this function rather than allocating the structure
> >> in here.  That lets the caller embed that structure inside one of it's own
> >> structures if it wants to, resulting in fewer small allocations which is always good.
> >>
> >> It's far from obvious that the caller needs to free desc.
> >>  
> >>> +
> >>> +	desc->con_id = id;
> >>> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
> >>> +	if (ret < 0) {
> >>> +		dev_err(gdev->chip->dev,
> >>> +			"failed to xlate id: %d\n", id);
> >>> +		goto out_free_desc;
> >>> +	}
> >>> +
> >>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
> >>> +	if (ret < 0) {
> >>> +		dev_err(gdev->chip->dev,
> >>> +			"failed to request id: %d\n", id);
> >>> +		goto out_free_desc;
> >>> +	}
> >>> +
> >>> +	return desc;
> >>> +
> >>> +out_free_desc:
> >>> +	kfree(desc);
> >>> +
> >>> +out_put_device:
> >>> +	return ERR_PTR(ret);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
> >>> +
> >>> +/**
> >>> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
> >>> + * information of the desc.
> >>> + *
> >>> + * @desc: ts descriptor, same as returned from request API.
> >>> + *
> >>> + * Context: Any context.
> >>> + * Returns: 0 on success else negative error code on failure.
> >>> + */
> >>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> >>> +			 struct hte_clk_info *ci)
> >>> +{
> >>> +	struct hte_chip *chip;
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	if (!desc || !desc->data_subsys || !ci) {
> >>> +		pr_debug("%s:%d\n", __func__, __LINE__);
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	ei = desc->data_subsys;
> >>> +	if (!ei || !ei->gdev || !ei->gdev->chip)
> >>> +		return -EINVAL;
> >>> +
> >>> +	chip = ei->gdev->chip;
> >>> +	if (!chip->ops->get_clk_src_info)
> >>> +		return -ENOTSUPP;
> >>> +
> >>> +	return chip->ops->get_clk_src_info(chip, ci);
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
> >>> +
> >>> +static inline void hte_add_to_device_list(struct hte_device *gdev)
> >>> +{
> >>> +	struct hte_device *prev;    
> >> Needs to take an appropriate lock as you may have concurrent calls.
> >>  
> >>> +
> >>> +	if (list_empty(&hte_devices)) {
> >>> +		list_add_tail(&gdev->list, &hte_devices);    
> >> Needs a comment. I've no idea why you might want to only add it if there were
> >> no other hte_devices already there.
> >>  
> >>> +		return;
> >>> +	}
> >>> +
> >>> +	prev = list_last_entry(&hte_devices, struct hte_device, list);    
> >> Why woud you do this?
> >>  
> >>> +	list_add_tail(&gdev->list, &hte_devices);
> >>> +}
> >>> +
> >>> +/**
> >>> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
> >>> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
> >>> + * using this API from its ISR or atomic context.
> >>> + *
> >>> + * @chip: The HTE chip, used during the registration.
> >>> + * @xlated_id: entity id understood by both subsystem and provider, usually this
> >>> + * is obtained from xlate callback during request API.
> >>> + * @data: timestamp data.
> >>> + * @n: Size of the data.
> >>> + *
> >>> + * Context: Atomic.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> >>> +			  struct hte_ts_data *data, size_t n)
> >>> +{
> >>> +	unsigned int ret;
> >>> +	bool notify;
> >>> +	size_t el_avail;
> >>> +	struct hte_ts_buf *buffer;
> >>> +	struct hte_ts_info *ei;
> >>> +
> >>> +	if (!chip || !data || !chip->gdev)
> >>> +		return -EINVAL;
> >>> +
> >>> +	if (xlated_id > chip->nlines)
> >>> +		return -EINVAL;
> >>> +
> >>> +	ei = &chip->gdev->ei[xlated_id];
> >>> +
> >>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
> >>> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
> >>> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	/* timestamp sequence counter, start from 0 */
> >>> +	data->seq = ei->seq++;
> >>> +
> >>> +	buffer = ei->buf;
> >>> +	el_avail = buffer->access->el_available(buffer);  
> >>> +	ret = buffer->access->store(buffer, data, n);    
> >> If we are doing this from the hte core, why is buffer definition in the scope of the
> >> drivers rather than the core?  That seems backwards to me.
> >>  
> >>> +	if (ret != n) {
> >>> +		atomic_inc(&ei->dropped_ts);
> >>> +		if (ei->cb)
> >>> +			ei->cb(HTE_TS_DROPPED);
> >>> +		return -ENOMEM;
> >>> +	}
> >>> +
> >>> +	notify = ((el_avail + 1) >= buffer->watermark) ? true : false;    
> >> You push n but only check on el_avail + 1 here.  
> >> Also, this is the same as
> >>
> >> 	notify = ((el_avail + 1) >= buffer->watermark;
> >>
> >>  
> >>> +
> >>> +	/*
> >>> +	 * If there is a callback, its consumer's job to retrieve the timestamp.
> >>> +	 * For the rest, wake up the process.
> >>> +	 */
> >>> +	if (notify && ei->cb) {
> >>> +		ei->cb(HTE_TS_AVAIL);
> >>> +		return 0;    
> >> Given you return 0 anyway, might as well not have this line.
> >>  
> >>> +	} else if (notify) {
> >>> +		wake_up_interruptible(&buffer->pollq);
> >>> +	}
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_push_ts_ns_atomic);
> >>> +
> >>> +/**
> >>> + * hte_register_chip() - Used by provider to register a HTE chip.
> >>> + * @chip: the HTE chip to add to subsystem.
> >>> + *
> >>> + * Context: Can not be called from atomic context.    
> >> Whilst true, I'd think that was common sense as it would be insane
> >> to register something like this from atomic context.  So I'd say no
> >> need to comment on it!  Keep those comments for things that
> >> might be used like that.
> >>  
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_register_chip(struct hte_chip *chip)
> >>> +{
> >>> +	struct hte_device *gdev;
> >>> +	int ret;
> >>> +	u32 i;
> >>> +
> >>> +	if (!chip || !chip->dev || !chip->dev->of_node)
> >>> +		return -EINVAL;
> >>> +
> >>> +	if (!chip->ops || !chip->ops->request || !chip->ops->release) {
> >>> +		dev_err(chip->dev, "Driver needs to provide ops\n");
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
> >>> +	if (!gdev)
> >>> +		return -ENOMEM;
> >>> +
> >>> +	gdev->chip = chip;
> >>> +	chip->gdev = gdev;
> >>> +	gdev->nlines = chip->nlines;
> >>> +	gdev->sdev = chip->dev;
> >>> +
> >>> +	/*
> >>> +	 * Allocate all the supported entities here at once, this will have
> >>> +	 * following advantages:
> >>> +	 * When provider pushes timestamp, it can then just send the
> >>> +	 * xlated_id, subsystem will use it as an index which
> >>> +	 * gives us the constant time access; this is important as mostly
> >>> +	 * providers will be pushing the timestamps from their ISR.
> >>> +	 */
> >>> +	gdev->ei = kcalloc(chip->nlines, sizeof(struct hte_ts_info),
> >>> +			   GFP_KERNEL);    
> >> I'd be tempted to do this as a 0 length element at the end of gdev
> >> then do the allocation in one go use struct_size() etc to work out
> >> how long it is.  Cuts down on allocations + error paths to deal with
> >> for no obvious disadvantage.
> >>  
> >>> +	if (!gdev->ei) {
> >>> +		ret = -ENOMEM;
> >>> +		goto err_free_gdev;
> >>> +	}
> >>> +
> >>> +	for (i = 0; i < chip->nlines; i++) {
> >>> +		gdev->ei[i].flags = 0;    
> >> zero allocated, so don't bother setting things to 0 where it's a fairly obvious
> >> base state.  If you set something to 0 to act as some form of documentation then
> >> that's fine, but I don't think that's true here.
> >>  
> >>> +		gdev->ei[i].gdev = gdev;
> >>> +		gdev->ei[i].seq = 0;
> >>> +		mutex_init(&gdev->ei[i].mlock);
> >>> +	}
> >>> +
> >>> +	if (chip->dev->driver)
> >>> +		gdev->owner = chip->dev->driver->owner;
> >>> +	else
> >>> +		gdev->owner = THIS_MODULE;
> >>> +
> >>> +	if (!chip->xlate) {
> >>> +		chip->xlate = hte_simple_xlate;
> >>> +		/* Just a id number to monitor */
> >>> +		chip->of_hte_n_cells = 1;
> >>> +	}
> >>> +
> >>> +	of_node_get(chip->dev->of_node);
> >>> +
> >>> +	INIT_LIST_HEAD(&gdev->list);
> >>> +
> >>> +	spin_lock(&hte_lock);
> >>> +	hte_add_to_device_list(gdev);
> >>> +	spin_unlock(&hte_lock);
> >>> +
> >>> +	hte_chip_dbgfs_init(gdev);
> >>> +
> >>> +	dev_dbg(chip->dev, "Added hte chip\n");
> >>> +	return 0;
> >>> +
> >>> +err_free_gdev:
> >>> +	kfree(gdev);
> >>> +
> >>> +	return ret;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_register_chip);
> >>> +
> >>> +/**
> >>> + * hte_unregister_chip() - Used by the provider to remove a HTE chip.
> >>> + * @chip: the HTE chip to remove.
> >>> + *
> >>> + * Context: Can not be called from atomic context.
> >>> + * Returns: 0 on success or a negative error code on failure.
> >>> + */
> >>> +int hte_unregister_chip(struct hte_chip *chip)
> >>> +{
> >>> +	struct hte_device *gdev = chip->gdev;
> >>> +
> >>> +	spin_lock(&hte_lock);
> >>> +	list_del(&gdev->list);
> >>> +	spin_unlock(&hte_lock);
> >>> +
> >>> +	gdev->chip = NULL;
> >>> +
> >>> +	of_node_put(chip->dev->of_node);
> >>> +	hte_dbgfs_deinit(gdev->dbg_root);
> >>> +	kfree(gdev->ei);
> >>> +	kfree(gdev);
> >>> +
> >>> +	dev_dbg(chip->dev, "Removed hte chip\n");
> >>> +	return 0;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(hte_unregister_chip);
> >>> +
> >>> +/* Driver APIs ends */    
> >> Don't bother with file layout type comments.  They don't add that much and tend
> >> to rot horribly over time as people move code around in files.
> >>  
> >>> diff --git a/include/linux/hte.h b/include/linux/hte.h
> >>> new file mode 100644
> >>> index 000000000000..e1737579d4c4
> >>> --- /dev/null
> >>> +++ b/include/linux/hte.h
> >>> @@ -0,0 +1,278 @@
> >>> +// SPDX-License-Identifier: GPL-2.0
> >>> +/*
> >>> + * Copyright (c) 2021 NVIDIA Corporation
> >>> + *
> >>> + * Author: Dipen Patel <dipenp@nvidia.com>
> >>> + */
> >>> +
> >>> +#ifndef __LINUX_HTE_H
> >>> +#define __LINUX_HTE_H
> >>> +
> >>> +struct hte_chip;
> >>> +struct hte_device;
> >>> +struct of_phandle_args;
> >>> +
> >>> +/**
> >>> + * Used by providers to indicate the direction of the timestamp.
> >>> + */
> >>> +#define HTE_EVENT_RISING_EDGE          0x1
> >>> +#define HTE_EVENT_FALLING_EDGE         0x2    
> >> Use an enum rather than a define for this as it's a value that can take a
> >> set of distinct values.  Also, provide a name for 'I've no idea' which
> >> I'm guessing is 0 currently.
> >>  
> >>> +
> >>> +/**
> >>> + * struct hte_ts_data - HTE timestamp data.
> >>> + * The provider uses and fills timestamp related details during push_timestamp
> >>> + * API call. The consumer uses during retrieve_timestamp API call.
> >>> + *
> >>> + * @tsc: Timestamp value.
> >>> + * @seq: Sequence counter of the timestamps.
> >>> + * @dir: Direction of the event at the time of timestamp.
> >>> + */
> >>> +struct hte_ts_data {
> >>> +	u64 tsc;
> >>> +	u64 seq;
> >>> +	int dir;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_clk_info - Clock source info that HTE provider uses.
> >>> + * The provider uses hardware clock as a source to timestamp real time. This
> >>> + * structure presents the clock information to consumers. 
> >>> + *
> >>> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
> >>> + * @type: Clock type. CLOCK_* types.    
> >> So this is something we got a it wrong in IIO. It's much better to define
> >> a subset of clocks that can be potentially used.  There are some that make
> >> absolutely no sense and consumers really don't want to have to deal with them.
> >>    
> >>> + */
> >>> +struct hte_clk_info {
> >>> +	u64 hz;
> >>> +	clockid_t type;
> >>> +};
> >>> +
> >>> +/**
> >>> + * HTE subsystem notifications for the consumers.
> >>> + *
> >>> + * @HTE_TS_AVAIL: Timestamps available notification.
> >>> + * @HTE_TS_DROPPED: Timestamps dropped notification.    
> >> Something I've missed so far is whether drops are in a kfifo or a ring
> >> fashion.  I'm guess that's stated somewhere, but it might be useful to have
> >> it here.
> >>  
> >>> + */
> >>> +enum hte_notify {
> >>> +	HTE_TS_AVAIL = 1,
> >>> +	HTE_TS_DROPPED,
> >>> +	HTE_NUM_NOTIFIER,
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_ts_desc - HTE timestamp descriptor, this structure will be
> >>> + * communication token between consumers to subsystem and subsystem to
> >>> + * providers.
> >>> + *
> >>> + * @con_id: This is the same id sent in request APIs.
> >>> + * @name: Descriptive name of the entity that is being monitored for the
> >>> + * realtime timestamping.
> >>> + * @data_subsys: Subsystem's private data relate to requested con_id.
> >>> + */
> >>> +struct hte_ts_desc {
> >>> +	u32 con_id;
> >>> +	char *name;
> >>> +	void *data_subsys;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_ops - HTE operations set by providers.
> >>> + *
> >>> + * @request: Hook for requesting a HTE timestamp. Returns 0 on success,
> >>> + * non-zero for failures.
> >>> + * @release: Hook for releasing a HTE timestamp. Returns 0 on success,
> >>> + * non-zero for failures.
> >>> + * @enable: Hook to enable the specified timestamp. Returns 0 on success,
> >>> + * non-zero for failures.
> >>> + * @disable: Hook to disable specified timestamp. Returns 0 on success,
> >>> + * non-zero for failures.
> >>> + * @get_clk_src_info: Optional hook to get the clock information provider uses
> >>> + * to timestamp. Returns 0 for success and negative error code for failure. On
> >>> + * success HTE subsystem fills up provided struct hte_clk_info.    
> >> Why optional?  Consumers will probably need that information.
> >>  
> >>> + *
> >>> + * xlated_id parameter is used to communicate between HTE subsystem and the
> >>> + * providers. It is the same id returned during xlate API call and translated
> >>> + * by the provider. This may be helpful as both subsystem and provider locate
> >>> + * the requested entity in constant time, where entity could be anything from
> >>> + * lines, signals, events, buses etc.. that providers support.
> >>> + */
> >>> +struct hte_ops {
> >>> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
> >>> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
> >>> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
> >>> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
> >>> +	int (*get_clk_src_info)(struct hte_chip *chip,
> >>> +				struct hte_clk_info *ci);
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct hte_chip - Abstract HTE chip structure.
> >>> + * @name: functional name of the HTE IP block.
> >>> + * @dev: device providing the HTE.    
> >> Unclear naming.  Is this the parent device, or one associated with the HTE itself?
> >> I'm guessing today you don't have one associated with the HTE, but it is plausible you
> >> might gain on in future to make it fit nicely in the device model as a function of another
> >> device.
> >>  
> >>> + * @ops: callbacks for this HTE.
> >>> + * @nlines: number of lines/signals supported by this chip.
> >>> + * @xlate: Callback which translates consumer supplied logical ids to
> >>> + * physical ids, return from 0 for the success and negative for the
> >>> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
> >>> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
> >>> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
> >>> + * @data: chip specific private data.
> >>> + */
> >>> +struct hte_chip {
> >>> +	const char *name;
> >>> +	struct device *dev;
> >>> +	const struct hte_ops *ops;
> >>> +	u32 nlines;
> >>> +	int (*xlate)(struct hte_chip *gc,
> >>> +		     const struct of_phandle_args *args,
> >>> +		     struct hte_ts_desc *desc, u32 *xlated_id);
> >>> +	u8 of_hte_n_cells;
> >>> +
> >>> +	/* only used internally by the HTE framework */
> >>> +	struct hte_device *gdev;
> >>> +	void *data;
> >>> +};
> >>> +
> >>> +#if IS_ENABLED(CONFIG_HTE)
> >>> +/* HTE APIs for the providers */
> >>> +int hte_register_chip(struct hte_chip *chip);
> >>> +int hte_unregister_chip(struct hte_chip *chip);
> >>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
> >>> +			  struct hte_ts_data *data, size_t n);
> >>> +
> >>> +/* HTE APIs for the consumers */
> >>> +
> >>> +int hte_release_ts(struct hte_ts_desc *desc);
> >>> +struct hte_ts_desc *of_hte_request_ts(struct device *dev, const char *label,
> >>> +				      void (*cb)(enum hte_notify n));
> >>> +
> >>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
> >>> +					   const char *label,
> >>> +					   void (*cb)(enum hte_notify n));
> >>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
> >>> +					  unsigned int id,
> >>> +					  void (*cb)(enum hte_notify n));
> >>> +int devm_hte_release_ts(struct device *dev, struct hte_ts_desc *desc);
> >>> +int hte_retrieve_ts_ns(const struct hte_ts_desc *desc, struct hte_ts_data *el,
> >>> +		       size_t n);
> >>> +int hte_retrieve_ts_ns_wait(const struct hte_ts_desc *desc,
> >>> +			    struct hte_ts_data *el, size_t n);
> >>> +int hte_set_buf_len(const struct hte_ts_desc *desc, size_t len);
> >>> +size_t hte_get_buf_len(const struct hte_ts_desc *desc);
> >>> +int hte_set_buf_watermark(const struct hte_ts_desc *desc, size_t val);
> >>> +size_t hte_get_buf_watermark(const struct hte_ts_desc *desc);
> >>> +size_t hte_available_ts(const struct hte_ts_desc *desc);
> >>> +int hte_enable_ts(struct hte_ts_desc *desc);
> >>> +int hte_disable_ts(struct hte_ts_desc *desc);
> >>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
> >>> +			 struct hte_clk_info *ci);
> >>> +
> >>>    


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-07-30  2:33     ` Dipen Patel
@ 2021-08-03 16:42       ` Jon Hunter
  2021-08-03 22:51         ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Jon Hunter @ 2021-08-03 16:42 UTC (permalink / raw)
  To: Dipen Patel, thierry.reding, linux-kernel, linux-tegra,
	linux-gpio, linus.walleij, bgolaszewski, warthog618, devicetree,
	linux-doc, robh+dt


On 30/07/2021 03:33, Dipen Patel wrote:
> 
> On 7/9/21 1:30 AM, Jon Hunter wrote:
>> On 26/06/2021 00:55, Dipen Patel wrote:
>>> This patch adds new clock type for the GPIO controller which can
>>> timestamp gpio lines using hardware means. To expose such
>>> functionalities to the userspace, code has been added in this patch
>>> where during line create call, it checks for new clock type and if
>>> requested, calls hardware timestamp related API from gpiolib.c.
>>> During line change event, it retrieves timestamp in nano seconds by
>>> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
>>> it disables this functionality by calling gpiod_hw_timestamp_control.
>>>
>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>> ---
>>>   drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>>>   include/uapi/linux/gpio.h   |  1 +
>>>   2 files changed, 64 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>> index 1631727bf0da..9f98c727e937 100644
>>> --- a/drivers/gpio/gpiolib-cdev.c
>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>> @@ -518,6 +518,7 @@ struct linereq {
>>>   	 GPIO_V2_LINE_DRIVE_FLAGS | \
>>>   	 GPIO_V2_LINE_EDGE_FLAGS | \
>>>   	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>> +	 GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>   	 GPIO_V2_LINE_BIAS_FLAGS)
>>>   
>>>   static void linereq_put_event(struct linereq *lr,
>>> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>>>   
>>>   static u64 line_event_timestamp(struct line *line)
>>>   {
>>> +	bool block;
>>> +
>>>   	if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>>>   		return ktime_get_real_ns();
>>>   
>>> +	if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>> +		if (irq_count())
>>> +			block = false;
>>> +		else
>>> +			block = true;
>>> +
>>> +		return gpiod_get_hw_timestamp(line->desc, block);
>>> +	}
>>> +
>>>   	return ktime_get_ns();
>>>   }
>>
>> Looking at line_event_timestamp() and the callers of this function, it
>> appears that this should always return nanoseconds. Does
>> gpiod_get_hw_timestamp() return nanoseconds?
> Yes, it returns in ns to align with line_event_timestamp.


It might be worth updating the function name to 
gpiod_get_hw_timestamp_ns() so that this is clear.

Jon

--
nvpublic

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-08-03 22:41         ` Dipen Patel
@ 2021-08-03 22:38           ` Kent Gibson
  0 siblings, 0 replies; 75+ messages in thread
From: Kent Gibson @ 2021-08-03 22:38 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt,
	andriy.shevchenko

On Tue, Aug 03, 2021 at 03:41:56PM -0700, Dipen Patel wrote:
> 
> On 7/30/21 11:05 PM, Kent Gibson wrote:
> > On Thu, Jul 29, 2021 at 08:07:15PM -0700, Dipen Patel wrote:
> >> On 7/1/21 7:24 AM, Kent Gibson wrote:
> > <snip>
> >>>>  			ret = gpiod_direction_output(desc, val);
> >>>>  			if (ret)
> >>>>  				return ret;
> >>>> @@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
> >>>>  					polarity_change);
> >>>>  			if (ret)
> >>>>  				return ret;
> >>>> +
> >>>> +			/* Check if new config sets hardware assisted clock */
> >>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
> >>>> +				ret = gpiod_hw_timestamp_control(desc, true);
> >>>> +				if (ret)
> >>>> +					return ret;
> >>>> +			}
> >>>>  		}
> >>>>  
> >>> The error code here can come from the pinctrl timestamp_control(), so it
> >>> should be sanitised before being returned to userspace.
> >> I do not understand what do you mean by sanitise. I just followed what
> >>
> >> gpiod_direction_output did just above which also returns ret from gpio
> >>
> >> driver code similar to timestamp_control API.
> >>
> > In this context, sanitise means convert any kernel internal error codes
> > to their userspace equivalent before returning them to userspace.
> >
> > Fair enough with the gpiod_direction_output() comparison.  I was thinking
> > of a patch Andy recently submitted[1] to sanitise gpiod_request(), which
> > can sometimes return EPROBE_DEFER.  But I guess we can wait until we find
> > a case of a driver returning an internal error code and add a sanitiser
> > then.
> Make sense, I will add sanity check
> >

But I said don't bother yet.  And you need to know what errors to sanitise
before you sanitise them - unless you want to run through all the
possibilities that can be returned to userspace.

Cheers,
Kent.


^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-31 15:43       ` Kent Gibson
@ 2021-08-03 22:40         ` Dipen Patel
  2021-08-03 23:02           ` Kent Gibson
  2021-08-07  2:41         ` Dipen Patel
  1 sibling, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-08-03 22:40 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 7/31/21 8:43 AM, Kent Gibson wrote:
> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
>> Thanks Kent for the review comment. My responses inline.
>>
>> On 7/1/21 7:21 AM, Kent Gibson wrote:
>>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>>>> Tegra194 device has multiple HTE instances also known as GTE
>>>> (Generic hardware Timestamping Engine) which can timestamp subset of
>>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>>>> and exposes timestamping ability on those lines to the consumers
>>>> through HTE subsystem.
>>>>
>>>> Also, with this patch, added:
>>>> - documentation about this provider and its capabilities at
>>>> Documentation/hte.
>>>> - Compilation support in Makefile and Kconfig
>>>>
>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>> ---
>>>>  Documentation/hte/index.rst        |  21 ++
>>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>>>  Documentation/index.rst            |   1 +
>>>>  drivers/hte/Kconfig                |  12 +
>>>>  drivers/hte/Makefile               |   1 +
>>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>>>  6 files changed, 654 insertions(+)
>>>>  create mode 100644 Documentation/hte/index.rst
>>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>>>  create mode 100644 drivers/hte/hte-tegra194.c
>>>>
>>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>>>> new file mode 100644
>>>> index 000000000000..f311ebec6b47
>>>> --- /dev/null
>>>> +++ b/Documentation/hte/index.rst
>>>> @@ -0,0 +1,21 @@
>>>> +.. SPDX-License-Identifier: GPL-2.0
>>>> +
>>>> +============================================
>>>> +The Linux Hardware Timestamping Engine (HTE)
>>>> +============================================
>>>> +
>>>> +The HTE Subsystem
>>>> +=================
>>>> +
>>>> +.. toctree::
>>>> +   :maxdepth: 1
>>>> +
>>>> +   hte
>>>> +
>>>> +HTE Tegra Provider
>>>> +==================
>>>> +
>>>> +.. toctree::
>>>> +   :maxdepth: 1
>>>> +
>>>> +   tegra194-hte
>>>> \ No newline at end of file
>>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>>>> new file mode 100644
>>>> index 000000000000..c23eaafcf080
>>>> --- /dev/null
>>>> +++ b/Documentation/hte/tegra194-hte.rst
>>>> @@ -0,0 +1,65 @@
>>>> +HTE Kernel provider driver
>>>> +==========================
>>>> +
>>>> +Description
>>>> +-----------
>>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>>>> +known as generic timestamping engine (GTE). This provider driver implements
>>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>>>> +value.
>>>> +
>>>> +GPIO GTE
>>>> +--------
>>>> +
>>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>>>> +controller as it requires very specific bits to be set in GPIO config register.
>>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>>>> +subsystem and GPIO GTE for in kernel consumers.
>>>> +
>>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>>>> +
>>>> +	To enable HTE on given GPIO line.
>>>> +
>>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>>>> +
>>>> +	To retrieve hardwre timestamp in nano seconds.
>>>> +
>>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>>>> +
>>>> +	To query if HTE is enabled on the given GPIO.
>>>> +
>>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>>>> +in nano second.
>>>> +
>>> <snip>
>>>
>>>> +
>>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>>>> +{
>>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>>>> +	u64 tsc;
>>>> +	int dir;
>>>> +	struct hte_ts_data el;
>>>> +
>>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>>>> +		tsc = (((u64)tsh << 32) | tsl);
>>>> +
>>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>>>> +
>>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>>>> +		acv = pv ^ cv;
>>>> +		while (acv) {
>>>> +			bit_index = __builtin_ctz(acv);
>>>> +			if ((pv >> bit_index) & BIT(0))
>>>> +				dir = HTE_EVENT_RISING_EDGE;
>>>> +			else
>>>> +				dir = HTE_EVENT_FALLING_EDGE;
>>>> +
>>>> +			line_id = bit_index + (slice << 5);
>>>> +			el.dir = dir;
>>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>>>> +					      sizeof(el));
>>>> +			acv &= ~BIT(bit_index);
>>>> +		}
>>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>>>> +	}
>>>> +}
>>> What happens when the hte_push_ts_ns_atomic() fails?
>>> The timestamp will be quietly dropped?
>>> What happens when the interrupt corresponding to that dropped timestamp
>>> asks for it?  The irq handler thread will block until it can get a
>>> timestamp from the subsequent interrupt?
>> Two things happen, 1) at the push, HTE core increments seq counter
>>
>> 2) If the consumer has provided callback, it will either call that callback
>>
>> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
>>
>> view of dropped ts. However, I see the problem with the consumers not
>>
>> providing callback, in that case, push_ts* API just wakes up process without
>>
>> indicating why (assuming notify variable is true or else there is a chance for
>>
>> the thread to block forever). One easy approach I can think of for now is to
>>
>> make callback mandatory (which is optional right now), I will have to rethink
>>
>> that scenario and will push corrected version next RFC version.
>>
>> Thanks for pointing out.
>>
> I'm not sure you understood my question, which was intended to
> demonstrate how an overflow here would break your gpio integration, but I
> am certain that I don't understand your answer.
>
> Using the callback to signal fifo overflow to the consumer is crazy.
> If the consumer is too busy to service the fifo then they probably wont
> be prepared to deal with the callback either. And the primary purpose of
> the fifo is to decouple the producer and consumer, so requiring a callback
> defeats the whole purpose of having the fifo there in the first place.
>
>>> Which brings me back to the concern I have with the approach used in
>>> the hte/gpiolib integration - how do you guarantee that the timestamp
>>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
>>> being handled, particularly in the face of errors such as:
>>>  - overflows of the timestamp FIFO in the chip
>> I currently do not have any indication mechanism as the providers
>>
>> I am dealing with right now does not have overflow hardware detection
>>
>> support. If the chip supports, it should be easy to integrate that feature.
>>
>> I will provide some hook function or change in push_* API to accommodate
>>
>> this in next version of RFC.
>>
>>>  - overflows of software FIFOs as here
>> HTE core records sequence counter as well it callsback the consumer with
>>
>> HTE_TS_DROPPED.
>>
>>>  - lost interupts (if the hw generates interrupts faster than the CPU
>>>    can service them)
>> For this, I have no idea unless hardware supports some sort of mechanism
>>
>> to catch that. For the current providers, as soon as it detects changes on lines
>>
>> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
>>
>> set in that hw fifo. This interrupt is different than the lines of actual device
>>
>> that is why I said I have no idea how we can tackle that. Let me know if there
>>
>> is any idea or reference of the codes which does tackle this.
>>
> As far as I am aware there is no solution, given your suggested
> architecture.
>
> Your architecture is inherently fragile, as you try to use one stream
> of data (the timestamp fifo) to provide supplementary info for another
> (the physical irq).  Guaranteeing that the two are synchronised is
> impossible - even if you can get them synced at some point, they can
> fall out of sync without any indication.
> That is a recipe for Ingenuity flight 6.
>
> My solution would be to use the hte timestamp fifo as the event source,
> rather than the physical irq.  With only one event source the 
> synchronisation problem disappears.  As to how to implement that,
> gpiolib-cdev would request a line from the hte subsystem and register
> and event handler for it, much as it does currently with the irq
> subsystem. 
Regarding "

much as it does currently with the irq
subsystem

" Statment, do you mean edge_irq_handler?
> That event handler would translate the hte events into gpio
> events.
>
> You still have to deal with possible fifo overflows, but if the fifo
> overflows result in discarding the oldest event, rather than the most
> recent, then everything comes out in the wash.  If not then the final
> event in a burst may not correspond to the actual state so you need
> some additional mechanism to address that.
> Either way the consumer needs to be aware that events may be lost - but
> with the event seqno for consumers to detect those lost events we
> already have that covered.
>
>> Regarding HTE/GPIOLIB integration comment:
>>
>> You are right, currently, I have only tsc field returned from struct hte_ts_data
>>
>> to gpiolib. If I can extend that to return hte_ts_data structure which has seq
>>
>> counter, which I believe can be used to track the overflow situation. The
>>
>> dropped scenario can be easily tracked if gpiolib can be notified with above
>>
>> mentioned DROP event through callback. If that is the case, is it ok to have
>>
>> some sort of callback per gpio in gpiolib?
>>
> Even better if you can provide the whole struct hte_ts_data so we have
> the direction as well (assuming all hte providers provide direction?).
> Otherwise gpiolib-cdev may need to read the physical line state and that
> may have changed since the hardware captured the event.
> In the solution I outlined above, the hte_ts_data would be provided to
> the event handler registered by gpiolib-cdev.

How is this event handler different then cdev providing callback to

hte core? I am guessing even cdev registers event handler with HTE

it is some sort of function  pointer so does callbacks.

> And in this case you could skip buffering the event in hte - it could be
> passed to the event handler as soon as it is read from the hardware - 
> gpiolib-cdev does its own buffering of gpio events.
>
>> Any idea how I can integrate callback notification with gpiolib if you do not agree on
>>
>> above callback suggestion?
>>
> See above.  But this is just my take, so I would get feedback from the
> relevant maintainers or SMEs before you act on anything suggested above.
>
> Cheers,
> Kent.
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-07-31  6:05       ` Kent Gibson
@ 2021-08-03 22:41         ` Dipen Patel
  2021-08-03 22:38           ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-08-03 22:41 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt,
	andriy.shevchenko


On 7/30/21 11:05 PM, Kent Gibson wrote:
> On Thu, Jul 29, 2021 at 08:07:15PM -0700, Dipen Patel wrote:
>> On 7/1/21 7:24 AM, Kent Gibson wrote:
> <snip>
>>>>  			ret = gpiod_direction_output(desc, val);
>>>>  			if (ret)
>>>>  				return ret;
>>>> @@ -1152,6 +1186,13 @@ static long linereq_set_config_unlocked(struct linereq *lr,
>>>>  					polarity_change);
>>>>  			if (ret)
>>>>  				return ret;
>>>> +
>>>> +			/* Check if new config sets hardware assisted clock */
>>>> +			if (flags & GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE) {
>>>> +				ret = gpiod_hw_timestamp_control(desc, true);
>>>> +				if (ret)
>>>> +					return ret;
>>>> +			}
>>>>  		}
>>>>  
>>> The error code here can come from the pinctrl timestamp_control(), so it
>>> should be sanitised before being returned to userspace.
>> I do not understand what do you mean by sanitise. I just followed what
>>
>> gpiod_direction_output did just above which also returns ret from gpio
>>
>> driver code similar to timestamp_control API.
>>
> In this context, sanitise means convert any kernel internal error codes
> to their userspace equivalent before returning them to userspace.
>
> Fair enough with the gpiod_direction_output() comparison.  I was thinking
> of a patch Andy recently submitted[1] to sanitise gpiod_request(), which
> can sometimes return EPROBE_DEFER.  But I guess we can wait until we find
> a case of a driver returning an internal error code and add a sanitiser
> then.
Make sense, I will add sanity check
>
> Cheers,
> Kent.
>
> [1] https://www.spinics.net/lists/linux-gpio/msg60998.html
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-08-03 16:42       ` Jon Hunter
@ 2021-08-03 22:51         ` Dipen Patel
  2021-08-03 23:09           ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-08-03 22:51 UTC (permalink / raw)
  To: Jon Hunter, thierry.reding, linux-kernel, linux-tegra,
	linux-gpio, linus.walleij, bgolaszewski, warthog618, devicetree,
	linux-doc, robh+dt


On 8/3/21 9:42 AM, Jon Hunter wrote:
>
> On 30/07/2021 03:33, Dipen Patel wrote:
>>
>> On 7/9/21 1:30 AM, Jon Hunter wrote:
>>> On 26/06/2021 00:55, Dipen Patel wrote:
>>>> This patch adds new clock type for the GPIO controller which can
>>>> timestamp gpio lines using hardware means. To expose such
>>>> functionalities to the userspace, code has been added in this patch
>>>> where during line create call, it checks for new clock type and if
>>>> requested, calls hardware timestamp related API from gpiolib.c.
>>>> During line change event, it retrieves timestamp in nano seconds by
>>>> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
>>>> it disables this functionality by calling gpiod_hw_timestamp_control.
>>>>
>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>> ---
>>>>   drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
>>>>   include/uapi/linux/gpio.h   |  1 +
>>>>   2 files changed, 64 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
>>>> index 1631727bf0da..9f98c727e937 100644
>>>> --- a/drivers/gpio/gpiolib-cdev.c
>>>> +++ b/drivers/gpio/gpiolib-cdev.c
>>>> @@ -518,6 +518,7 @@ struct linereq {
>>>>        GPIO_V2_LINE_DRIVE_FLAGS | \
>>>>        GPIO_V2_LINE_EDGE_FLAGS | \
>>>>        GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
>>>> +     GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
>>>>        GPIO_V2_LINE_BIAS_FLAGS)
>>>>     static void linereq_put_event(struct linereq *lr,
>>>> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
>>>>     static u64 line_event_timestamp(struct line *line)
>>>>   {
>>>> +    bool block;
>>>> +
>>>>       if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
>>>>           return ktime_get_real_ns();
>>>>   +    if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
>>>> +        if (irq_count())
>>>> +            block = false;
>>>> +        else
>>>> +            block = true;
>>>> +
>>>> +        return gpiod_get_hw_timestamp(line->desc, block);
>>>> +    }
>>>> +
>>>>       return ktime_get_ns();
>>>>   }
>>>
>>> Looking at line_event_timestamp() and the callers of this function, it
>>> appears that this should always return nanoseconds. Does
>>> gpiod_get_hw_timestamp() return nanoseconds?
>> Yes, it returns in ns to align with line_event_timestamp.
>
>
> It might be worth updating the function name to gpiod_get_hw_timestamp_ns() so that this is clear.
Wouldn't be sufficient to right into its description rather embed in API name?
>
> Jon
>
> -- 
> nvpublic

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-03 22:40         ` Dipen Patel
@ 2021-08-03 23:02           ` Kent Gibson
  0 siblings, 0 replies; 75+ messages in thread
From: Kent Gibson @ 2021-08-03 23:02 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Tue, Aug 03, 2021 at 03:40:50PM -0700, Dipen Patel wrote:
> 
> On 7/31/21 8:43 AM, Kent Gibson wrote:
> > On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
> >> Thanks Kent for the review comment. My responses inline.
> >>
> >> On 7/1/21 7:21 AM, Kent Gibson wrote:
> >>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
> >>>> Tegra194 device has multiple HTE instances also known as GTE
> >>>> (Generic hardware Timestamping Engine) which can timestamp subset of
> >>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> >>>> and exposes timestamping ability on those lines to the consumers
> >>>> through HTE subsystem.
> >>>>
> >>>> Also, with this patch, added:
> >>>> - documentation about this provider and its capabilities at
> >>>> Documentation/hte.
> >>>> - Compilation support in Makefile and Kconfig
> >>>>
> >>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >>>> ---
> >>>>  Documentation/hte/index.rst        |  21 ++
> >>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
> >>>>  Documentation/index.rst            |   1 +
> >>>>  drivers/hte/Kconfig                |  12 +
> >>>>  drivers/hte/Makefile               |   1 +
> >>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
> >>>>  6 files changed, 654 insertions(+)
> >>>>  create mode 100644 Documentation/hte/index.rst
> >>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
> >>>>  create mode 100644 drivers/hte/hte-tegra194.c
> >>>>
> >>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
> >>>> new file mode 100644
> >>>> index 000000000000..f311ebec6b47
> >>>> --- /dev/null
> >>>> +++ b/Documentation/hte/index.rst
> >>>> @@ -0,0 +1,21 @@
> >>>> +.. SPDX-License-Identifier: GPL-2.0
> >>>> +
> >>>> +============================================
> >>>> +The Linux Hardware Timestamping Engine (HTE)
> >>>> +============================================
> >>>> +
> >>>> +The HTE Subsystem
> >>>> +=================
> >>>> +
> >>>> +.. toctree::
> >>>> +   :maxdepth: 1
> >>>> +
> >>>> +   hte
> >>>> +
> >>>> +HTE Tegra Provider
> >>>> +==================
> >>>> +
> >>>> +.. toctree::
> >>>> +   :maxdepth: 1
> >>>> +
> >>>> +   tegra194-hte
> >>>> \ No newline at end of file
> >>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> >>>> new file mode 100644
> >>>> index 000000000000..c23eaafcf080
> >>>> --- /dev/null
> >>>> +++ b/Documentation/hte/tegra194-hte.rst
> >>>> @@ -0,0 +1,65 @@
> >>>> +HTE Kernel provider driver
> >>>> +==========================
> >>>> +
> >>>> +Description
> >>>> +-----------
> >>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
> >>>> +known as generic timestamping engine (GTE). This provider driver implements
> >>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
> >>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> >>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
> >>>> +value.
> >>>> +
> >>>> +GPIO GTE
> >>>> +--------
> >>>> +
> >>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
> >>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
> >>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
> >>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
> >>>> +controller as it requires very specific bits to be set in GPIO config register.
> >>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
> >>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
> >>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
> >>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
> >>>> +subsystem and GPIO GTE for in kernel consumers.
> >>>> +
> >>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
> >>>> +
> >>>> +	To enable HTE on given GPIO line.
> >>>> +
> >>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
> >>>> +
> >>>> +	To retrieve hardwre timestamp in nano seconds.
> >>>> +
> >>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
> >>>> +
> >>>> +	To query if HTE is enabled on the given GPIO.
> >>>> +
> >>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> >>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
> >>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
> >>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
> >>>> +in nano second.
> >>>> +
> >>> <snip>
> >>>
> >>>> +
> >>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
> >>>> +{
> >>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
> >>>> +	u64 tsc;
> >>>> +	int dir;
> >>>> +	struct hte_ts_data el;
> >>>> +
> >>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
> >>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
> >>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
> >>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
> >>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
> >>>> +		tsc = (((u64)tsh << 32) | tsl);
> >>>> +
> >>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
> >>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
> >>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
> >>>> +
> >>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
> >>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
> >>>> +		acv = pv ^ cv;
> >>>> +		while (acv) {
> >>>> +			bit_index = __builtin_ctz(acv);
> >>>> +			if ((pv >> bit_index) & BIT(0))
> >>>> +				dir = HTE_EVENT_RISING_EDGE;
> >>>> +			else
> >>>> +				dir = HTE_EVENT_FALLING_EDGE;
> >>>> +
> >>>> +			line_id = bit_index + (slice << 5);
> >>>> +			el.dir = dir;
> >>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
> >>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
> >>>> +					      sizeof(el));
> >>>> +			acv &= ~BIT(bit_index);
> >>>> +		}
> >>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
> >>>> +	}
> >>>> +}
> >>> What happens when the hte_push_ts_ns_atomic() fails?
> >>> The timestamp will be quietly dropped?
> >>> What happens when the interrupt corresponding to that dropped timestamp
> >>> asks for it?  The irq handler thread will block until it can get a
> >>> timestamp from the subsequent interrupt?
> >> Two things happen, 1) at the push, HTE core increments seq counter
> >>
> >> 2) If the consumer has provided callback, it will either call that callback
> >>
> >> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
> >>
> >> view of dropped ts. However, I see the problem with the consumers not
> >>
> >> providing callback, in that case, push_ts* API just wakes up process without
> >>
> >> indicating why (assuming notify variable is true or else there is a chance for
> >>
> >> the thread to block forever). One easy approach I can think of for now is to
> >>
> >> make callback mandatory (which is optional right now), I will have to rethink
> >>
> >> that scenario and will push corrected version next RFC version.
> >>
> >> Thanks for pointing out.
> >>
> > I'm not sure you understood my question, which was intended to
> > demonstrate how an overflow here would break your gpio integration, but I
> > am certain that I don't understand your answer.
> >
> > Using the callback to signal fifo overflow to the consumer is crazy.
> > If the consumer is too busy to service the fifo then they probably wont
> > be prepared to deal with the callback either. And the primary purpose of
> > the fifo is to decouple the producer and consumer, so requiring a callback
> > defeats the whole purpose of having the fifo there in the first place.
> >
> >>> Which brings me back to the concern I have with the approach used in
> >>> the hte/gpiolib integration - how do you guarantee that the timestamp
> >>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
> >>> being handled, particularly in the face of errors such as:
> >>>  - overflows of the timestamp FIFO in the chip
> >> I currently do not have any indication mechanism as the providers
> >>
> >> I am dealing with right now does not have overflow hardware detection
> >>
> >> support. If the chip supports, it should be easy to integrate that feature.
> >>
> >> I will provide some hook function or change in push_* API to accommodate
> >>
> >> this in next version of RFC.
> >>
> >>>  - overflows of software FIFOs as here
> >> HTE core records sequence counter as well it callsback the consumer with
> >>
> >> HTE_TS_DROPPED.
> >>
> >>>  - lost interupts (if the hw generates interrupts faster than the CPU
> >>>    can service them)
> >> For this, I have no idea unless hardware supports some sort of mechanism
> >>
> >> to catch that. For the current providers, as soon as it detects changes on lines
> >>
> >> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
> >>
> >> set in that hw fifo. This interrupt is different than the lines of actual device
> >>
> >> that is why I said I have no idea how we can tackle that. Let me know if there
> >>
> >> is any idea or reference of the codes which does tackle this.
> >>
> > As far as I am aware there is no solution, given your suggested
> > architecture.
> >
> > Your architecture is inherently fragile, as you try to use one stream
> > of data (the timestamp fifo) to provide supplementary info for another
> > (the physical irq).  Guaranteeing that the two are synchronised is
> > impossible - even if you can get them synced at some point, they can
> > fall out of sync without any indication.
> > That is a recipe for Ingenuity flight 6.
> >
> > My solution would be to use the hte timestamp fifo as the event source,
> > rather than the physical irq.  With only one event source the 
> > synchronisation problem disappears.  As to how to implement that,
> > gpiolib-cdev would request a line from the hte subsystem and register
> > and event handler for it, much as it does currently with the irq
> > subsystem. 
> Regarding "
> 
> much as it does currently with the irq
> subsystem
> 
> " Statment, do you mean edge_irq_handler?

I mean that style of API.  Obviously it would be a new handler function.
But it would perform the same as edge_irq_handler and edge_irq_thread,
just with a different event source.

> > That event handler would translate the hte events into gpio
> > events.
> >
> > You still have to deal with possible fifo overflows, but if the fifo
> > overflows result in discarding the oldest event, rather than the most
> > recent, then everything comes out in the wash.  If not then the final
> > event in a burst may not correspond to the actual state so you need
> > some additional mechanism to address that.
> > Either way the consumer needs to be aware that events may be lost - but
> > with the event seqno for consumers to detect those lost events we
> > already have that covered.
> >
> >> Regarding HTE/GPIOLIB integration comment:
> >>
> >> You are right, currently, I have only tsc field returned from struct hte_ts_data
> >>
> >> to gpiolib. If I can extend that to return hte_ts_data structure which has seq
> >>
> >> counter, which I believe can be used to track the overflow situation. The
> >>
> >> dropped scenario can be easily tracked if gpiolib can be notified with above
> >>
> >> mentioned DROP event through callback. If that is the case, is it ok to have
> >>
> >> some sort of callback per gpio in gpiolib?
> >>
> > Even better if you can provide the whole struct hte_ts_data so we have
> > the direction as well (assuming all hte providers provide direction?).
> > Otherwise gpiolib-cdev may need to read the physical line state and that
> > may have changed since the hardware captured the event.
> > In the solution I outlined above, the hte_ts_data would be provided to
> > the event handler registered by gpiolib-cdev.
> 
> How is this event handler different then cdev providing callback to
> 
> hte core? I am guessing even cdev registers event handler with HTE
> 
> it is some sort of function  pointer so does callbacks.
> 

If you mean your proposed callbacks, well for starters it wouldn't pass
it a DROPPED event.

But other than that registering a handler it essentially a callback.
Your existing callback is at interrupt context, right?
The irq subsystem also has provision for handling the event at
interrupt context or thread context - gpiolib-cdev uses both.
You might want to do the same here - depends on what your expected
consumers would prefer.

Way back when you initially proposed this I said "this is an irq problem",
meaning that it makes sense to me that this should be integrated with irq,
and provide functions to return additional detail to the irq handlers,
such as the event timestamp.
Not sure what the irq guys think of that - it may be simpler and
clearer to provide a separate subsystem.
Either way, a hte subsystem that provides an irq-like API might be a
good way to start.

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type
  2021-08-03 22:51         ` Dipen Patel
@ 2021-08-03 23:09           ` Kent Gibson
  0 siblings, 0 replies; 75+ messages in thread
From: Kent Gibson @ 2021-08-03 23:09 UTC (permalink / raw)
  To: Dipen Patel
  Cc: Jon Hunter, thierry.reding, linux-kernel, linux-tegra,
	linux-gpio, linus.walleij, bgolaszewski, devicetree, linux-doc,
	robh+dt

On Tue, Aug 03, 2021 at 03:51:31PM -0700, Dipen Patel wrote:
> 
> On 8/3/21 9:42 AM, Jon Hunter wrote:
> >
> > On 30/07/2021 03:33, Dipen Patel wrote:
> >>
> >> On 7/9/21 1:30 AM, Jon Hunter wrote:
> >>> On 26/06/2021 00:55, Dipen Patel wrote:
> >>>> This patch adds new clock type for the GPIO controller which can
> >>>> timestamp gpio lines using hardware means. To expose such
> >>>> functionalities to the userspace, code has been added in this patch
> >>>> where during line create call, it checks for new clock type and if
> >>>> requested, calls hardware timestamp related API from gpiolib.c.
> >>>> During line change event, it retrieves timestamp in nano seconds by
> >>>> calling gpiod_get_hw_timestamp API from gpiolib.c. At the line release,
> >>>> it disables this functionality by calling gpiod_hw_timestamp_control.
> >>>>
> >>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >>>> ---
> >>>>   drivers/gpio/gpiolib-cdev.c | 65 +++++++++++++++++++++++++++++++++++--
> >>>>   include/uapi/linux/gpio.h   |  1 +
> >>>>   2 files changed, 64 insertions(+), 2 deletions(-)
> >>>>
> >>>> diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
> >>>> index 1631727bf0da..9f98c727e937 100644
> >>>> --- a/drivers/gpio/gpiolib-cdev.c
> >>>> +++ b/drivers/gpio/gpiolib-cdev.c
> >>>> @@ -518,6 +518,7 @@ struct linereq {
> >>>>        GPIO_V2_LINE_DRIVE_FLAGS | \
> >>>>        GPIO_V2_LINE_EDGE_FLAGS | \
> >>>>        GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME | \
> >>>> +     GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE | \
> >>>>        GPIO_V2_LINE_BIAS_FLAGS)
> >>>>     static void linereq_put_event(struct linereq *lr,
> >>>> @@ -540,9 +541,20 @@ static void linereq_put_event(struct linereq *lr,
> >>>>     static u64 line_event_timestamp(struct line *line)
> >>>>   {
> >>>> +    bool block;
> >>>> +
> >>>>       if (test_bit(FLAG_EVENT_CLOCK_REALTIME, &line->desc->flags))
> >>>>           return ktime_get_real_ns();
> >>>>   +    if (test_bit(FLAG_EVENT_CLOCK_HARDWARE, &line->desc->flags)) {
> >>>> +        if (irq_count())
> >>>> +            block = false;
> >>>> +        else
> >>>> +            block = true;
> >>>> +
> >>>> +        return gpiod_get_hw_timestamp(line->desc, block);
> >>>> +    }
> >>>> +
> >>>>       return ktime_get_ns();
> >>>>   }
> >>>
> >>> Looking at line_event_timestamp() and the callers of this function, it
> >>> appears that this should always return nanoseconds. Does
> >>> gpiod_get_hw_timestamp() return nanoseconds?
> >> Yes, it returns in ns to align with line_event_timestamp.
> >
> >
> > It might be worth updating the function name to gpiod_get_hw_timestamp_ns() so that this is clear.
> Wouldn't be sufficient to right into its description rather embed in API name?
> >

Adding a suffix identifying the timestamp resolution to variable and
function names is pretty standard in the kernel.
It makes the code easier to read as you don't have to keep checking the
documentation.

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-07-31 15:43       ` Kent Gibson
  2021-08-03 22:40         ` Dipen Patel
@ 2021-08-07  2:41         ` Dipen Patel
  2021-08-07  3:07           ` Kent Gibson
  1 sibling, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-08-07  2:41 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 7/31/21 8:43 AM, Kent Gibson wrote:
> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
>> Thanks Kent for the review comment. My responses inline.
>>
>> On 7/1/21 7:21 AM, Kent Gibson wrote:
>>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>>>> Tegra194 device has multiple HTE instances also known as GTE
>>>> (Generic hardware Timestamping Engine) which can timestamp subset of
>>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>>>> and exposes timestamping ability on those lines to the consumers
>>>> through HTE subsystem.
>>>>
>>>> Also, with this patch, added:
>>>> - documentation about this provider and its capabilities at
>>>> Documentation/hte.
>>>> - Compilation support in Makefile and Kconfig
>>>>
>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>> ---
>>>>  Documentation/hte/index.rst        |  21 ++
>>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>>>  Documentation/index.rst            |   1 +
>>>>  drivers/hte/Kconfig                |  12 +
>>>>  drivers/hte/Makefile               |   1 +
>>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>>>  6 files changed, 654 insertions(+)
>>>>  create mode 100644 Documentation/hte/index.rst
>>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>>>  create mode 100644 drivers/hte/hte-tegra194.c
>>>>
>>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>>>> new file mode 100644
>>>> index 000000000000..f311ebec6b47
>>>> --- /dev/null
>>>> +++ b/Documentation/hte/index.rst
>>>> @@ -0,0 +1,21 @@
>>>> +.. SPDX-License-Identifier: GPL-2.0
>>>> +
>>>> +============================================
>>>> +The Linux Hardware Timestamping Engine (HTE)
>>>> +============================================
>>>> +
>>>> +The HTE Subsystem
>>>> +=================
>>>> +
>>>> +.. toctree::
>>>> +   :maxdepth: 1
>>>> +
>>>> +   hte
>>>> +
>>>> +HTE Tegra Provider
>>>> +==================
>>>> +
>>>> +.. toctree::
>>>> +   :maxdepth: 1
>>>> +
>>>> +   tegra194-hte
>>>> \ No newline at end of file
>>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>>>> new file mode 100644
>>>> index 000000000000..c23eaafcf080
>>>> --- /dev/null
>>>> +++ b/Documentation/hte/tegra194-hte.rst
>>>> @@ -0,0 +1,65 @@
>>>> +HTE Kernel provider driver
>>>> +==========================
>>>> +
>>>> +Description
>>>> +-----------
>>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>>>> +known as generic timestamping engine (GTE). This provider driver implements
>>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>>>> +value.
>>>> +
>>>> +GPIO GTE
>>>> +--------
>>>> +
>>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>>>> +controller as it requires very specific bits to be set in GPIO config register.
>>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>>>> +subsystem and GPIO GTE for in kernel consumers.
>>>> +
>>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>>>> +
>>>> +	To enable HTE on given GPIO line.
>>>> +
>>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>>>> +
>>>> +	To retrieve hardwre timestamp in nano seconds.
>>>> +
>>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>>>> +
>>>> +	To query if HTE is enabled on the given GPIO.
>>>> +
>>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>>>> +in nano second.
>>>> +
>>> <snip>
>>>
>>>> +
>>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>>>> +{
>>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>>>> +	u64 tsc;
>>>> +	int dir;
>>>> +	struct hte_ts_data el;
>>>> +
>>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>>>> +		tsc = (((u64)tsh << 32) | tsl);
>>>> +
>>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>>>> +
>>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>>>> +		acv = pv ^ cv;
>>>> +		while (acv) {
>>>> +			bit_index = __builtin_ctz(acv);
>>>> +			if ((pv >> bit_index) & BIT(0))
>>>> +				dir = HTE_EVENT_RISING_EDGE;
>>>> +			else
>>>> +				dir = HTE_EVENT_FALLING_EDGE;
>>>> +
>>>> +			line_id = bit_index + (slice << 5);
>>>> +			el.dir = dir;
>>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>>>> +					      sizeof(el));
>>>> +			acv &= ~BIT(bit_index);
>>>> +		}
>>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>>>> +	}
>>>> +}
>>> What happens when the hte_push_ts_ns_atomic() fails?
>>> The timestamp will be quietly dropped?
>>> What happens when the interrupt corresponding to that dropped timestamp
>>> asks for it?  The irq handler thread will block until it can get a
>>> timestamp from the subsequent interrupt?
>> Two things happen, 1) at the push, HTE core increments seq counter
>>
>> 2) If the consumer has provided callback, it will either call that callback
>>
>> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
>>
>> view of dropped ts. However, I see the problem with the consumers not
>>
>> providing callback, in that case, push_ts* API just wakes up process without
>>
>> indicating why (assuming notify variable is true or else there is a chance for
>>
>> the thread to block forever). One easy approach I can think of for now is to
>>
>> make callback mandatory (which is optional right now), I will have to rethink
>>
>> that scenario and will push corrected version next RFC version.
>>
>> Thanks for pointing out.
>>
> I'm not sure you understood my question, which was intended to
> demonstrate how an overflow here would break your gpio integration, but I
> am certain that I don't understand your answer.
>
> Using the callback to signal fifo overflow to the consumer is crazy.
> If the consumer is too busy to service the fifo then they probably wont
> be prepared to deal with the callback either. And the primary purpose of
> the fifo is to decouple the producer and consumer, so requiring a callback
> defeats the whole purpose of having the fifo there in the first place.
>
>>> Which brings me back to the concern I have with the approach used in
>>> the hte/gpiolib integration - how do you guarantee that the timestamp
>>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
>>> being handled, particularly in the face of errors such as:
>>>  - overflows of the timestamp FIFO in the chip
>> I currently do not have any indication mechanism as the providers
>>
>> I am dealing with right now does not have overflow hardware detection
>>
>> support. If the chip supports, it should be easy to integrate that feature.
>>
>> I will provide some hook function or change in push_* API to accommodate
>>
>> this in next version of RFC.
>>
>>>  - overflows of software FIFOs as here
>> HTE core records sequence counter as well it callsback the consumer with
>>
>> HTE_TS_DROPPED.
>>
>>>  - lost interupts (if the hw generates interrupts faster than the CPU
>>>    can service them)
>> For this, I have no idea unless hardware supports some sort of mechanism
>>
>> to catch that. For the current providers, as soon as it detects changes on lines
>>
>> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
>>
>> set in that hw fifo. This interrupt is different than the lines of actual device
>>
>> that is why I said I have no idea how we can tackle that. Let me know if there
>>
>> is any idea or reference of the codes which does tackle this.
>>
> As far as I am aware there is no solution, given your suggested
> architecture.
>
> Your architecture is inherently fragile, as you try to use one stream
> of data (the timestamp fifo) to provide supplementary info for another
> (the physical irq).  Guaranteeing that the two are synchronised is
> impossible - even if you can get them synced at some point, they can
> fall out of sync without any indication.
> That is a recipe for Ingenuity flight 6.
>
> My solution would be to use the hte timestamp fifo as the event source,
> rather than the physical irq.  With only one event source the 
> synchronisation problem disappears.  As to how to implement that,
> gpiolib-cdev would request a line from the hte subsystem and register
> and event handler for it, much as it does currently with the irq
> subsystem. That event handler would translate the hte events into gpio
> events.

I have to circle back to here regarding the event handler thing. I

surely did not understand fifo as event source rather than physical irq

part? I believe you are suggesting to have somekind of buffer abstraction

layer for the hardware fifo similar to what I have with software buffer and

register handler to that buffer, right?


The current implementation I have (not with gpiolib/HTE integration)

is partially simlar to event handler mechanism except that it does not send data

with it. See hte-tegra194-irq-test.c in this patch.


Coming back to gpiolib/hte integration part and your suggestion about

providing event handler during hte registration. I have below doubts:

1. When HTE calls this provided hte_handler, will it store data into

hte->timestamp_ns directly, I am guessing yes.

2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and

hte_handler, for the worst case scenario as below?

2.a edge_irq_handler runs first, checks some kind of flag to see if

we are using hardware clock and if yes, directly accesses timestamp_ns

instead of calling line_event_timestamp.

2.b finds timestamp_ns to be invalid since it ran first before hte event handler did.

2.c returns and invokes edge_irq_thread.

2.c.1 Here, should edge_irq_thread wait in cdev till hte handler to be called? If yes,

Doesn't it have case where it will wait forever till hte handler gets called, also not

to mention keeping irq line disabled since IRQF_ONESHOT is specified, could be possible

when provider has gone rogue?

3. I am guessing there will not be dropped event in this suggestion since are

directly sending data without buffering in HTE, that is the good part I believe.


>
> You still have to deal with possible fifo overflows, but if the fifo
> overflows result in discarding the oldest event, rather than the most
> recent, then everything comes out in the wash.  If not then the final
> event in a burst may not correspond to the actual state so you need
> some additional mechanism to address that.
> Either way the consumer needs to be aware that events may be lost - but
> with the event seqno for consumers to detect those lost events we
> already have that covered.

Correct (for the seqno part), you already have seqno, cdev does not need

struct hte_ts_data's u64 seq counter.


On similar note, I was looking at the linereq_put_event

function and I have below doubts:

1. IIUC, you are discarding oldest data when fifo is full, right?

2. There is no indication to waiting client if overflow is true beside pr_debug print.

2.a Does this not block waiting client infinitely since there is no wake_up_poll call

in case of overflow == 1?

2.c If I have missed, what current mechanism cdev provides to client beside seqno

to indicate there is a drop and if there is a drop, what it does to re-sync?


>
>> Regarding HTE/GPIOLIB integration comment:
>>
>> You are right, currently, I have only tsc field returned from struct hte_ts_data
>>
>> to gpiolib. If I can extend that to return hte_ts_data structure which has seq
>>
>> counter, which I believe can be used to track the overflow situation. The
>>
>> dropped scenario can be easily tracked if gpiolib can be notified with above
>>
>> mentioned DROP event through callback. If that is the case, is it ok to have
>>
>> some sort of callback per gpio in gpiolib?
>>
> Even better if you can provide the whole struct hte_ts_data so we have
> the direction as well (assuming all hte providers provide direction?).
> Otherwise gpiolib-cdev may need to read the physical line state and that
> may have changed since the hardware captured the event.
> In the solution I outlined above, the hte_ts_data would be provided to
> the event handler registered by gpiolib-cdev.
> And in this case you could skip buffering the event in hte - it could be
> passed to the event handler as soon as it is read from the hardware - 
> gpiolib-cdev does its own buffering of gpio events.
>
>> Any idea how I can integrate callback notification with gpiolib if you do not agree on
>>
>> above callback suggestion?
>>
> See above.  But this is just my take, so I would get feedback from the
> relevant maintainers or SMEs before you act on anything suggested above.
>
> Cheers,
> Kent.
>

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  2:41         ` Dipen Patel
@ 2021-08-07  3:07           ` Kent Gibson
  2021-08-07  4:52             ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-08-07  3:07 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
> 
> On 7/31/21 8:43 AM, Kent Gibson wrote:
> > On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
> >> Thanks Kent for the review comment. My responses inline.
> >>
> >> On 7/1/21 7:21 AM, Kent Gibson wrote:
> >>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
> >>>> Tegra194 device has multiple HTE instances also known as GTE
> >>>> (Generic hardware Timestamping Engine) which can timestamp subset of
> >>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
> >>>> and exposes timestamping ability on those lines to the consumers
> >>>> through HTE subsystem.
> >>>>
> >>>> Also, with this patch, added:
> >>>> - documentation about this provider and its capabilities at
> >>>> Documentation/hte.
> >>>> - Compilation support in Makefile and Kconfig
> >>>>
> >>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> >>>> ---
> >>>>  Documentation/hte/index.rst        |  21 ++
> >>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
> >>>>  Documentation/index.rst            |   1 +
> >>>>  drivers/hte/Kconfig                |  12 +
> >>>>  drivers/hte/Makefile               |   1 +
> >>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
> >>>>  6 files changed, 654 insertions(+)
> >>>>  create mode 100644 Documentation/hte/index.rst
> >>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
> >>>>  create mode 100644 drivers/hte/hte-tegra194.c
> >>>>
> >>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
> >>>> new file mode 100644
> >>>> index 000000000000..f311ebec6b47
> >>>> --- /dev/null
> >>>> +++ b/Documentation/hte/index.rst
> >>>> @@ -0,0 +1,21 @@
> >>>> +.. SPDX-License-Identifier: GPL-2.0
> >>>> +
> >>>> +============================================
> >>>> +The Linux Hardware Timestamping Engine (HTE)
> >>>> +============================================
> >>>> +
> >>>> +The HTE Subsystem
> >>>> +=================
> >>>> +
> >>>> +.. toctree::
> >>>> +   :maxdepth: 1
> >>>> +
> >>>> +   hte
> >>>> +
> >>>> +HTE Tegra Provider
> >>>> +==================
> >>>> +
> >>>> +.. toctree::
> >>>> +   :maxdepth: 1
> >>>> +
> >>>> +   tegra194-hte
> >>>> \ No newline at end of file
> >>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
> >>>> new file mode 100644
> >>>> index 000000000000..c23eaafcf080
> >>>> --- /dev/null
> >>>> +++ b/Documentation/hte/tegra194-hte.rst
> >>>> @@ -0,0 +1,65 @@
> >>>> +HTE Kernel provider driver
> >>>> +==========================
> >>>> +
> >>>> +Description
> >>>> +-----------
> >>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
> >>>> +known as generic timestamping engine (GTE). This provider driver implements
> >>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
> >>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
> >>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
> >>>> +value.
> >>>> +
> >>>> +GPIO GTE
> >>>> +--------
> >>>> +
> >>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
> >>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
> >>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
> >>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
> >>>> +controller as it requires very specific bits to be set in GPIO config register.
> >>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
> >>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
> >>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
> >>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
> >>>> +subsystem and GPIO GTE for in kernel consumers.
> >>>> +
> >>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
> >>>> +
> >>>> +	To enable HTE on given GPIO line.
> >>>> +
> >>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
> >>>> +
> >>>> +	To retrieve hardwre timestamp in nano seconds.
> >>>> +
> >>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
> >>>> +
> >>>> +	To query if HTE is enabled on the given GPIO.
> >>>> +
> >>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
> >>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
> >>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
> >>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
> >>>> +in nano second.
> >>>> +
> >>> <snip>
> >>>
> >>>> +
> >>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
> >>>> +{
> >>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
> >>>> +	u64 tsc;
> >>>> +	int dir;
> >>>> +	struct hte_ts_data el;
> >>>> +
> >>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
> >>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
> >>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
> >>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
> >>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
> >>>> +		tsc = (((u64)tsh << 32) | tsl);
> >>>> +
> >>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
> >>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
> >>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
> >>>> +
> >>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
> >>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
> >>>> +		acv = pv ^ cv;
> >>>> +		while (acv) {
> >>>> +			bit_index = __builtin_ctz(acv);
> >>>> +			if ((pv >> bit_index) & BIT(0))
> >>>> +				dir = HTE_EVENT_RISING_EDGE;
> >>>> +			else
> >>>> +				dir = HTE_EVENT_FALLING_EDGE;
> >>>> +
> >>>> +			line_id = bit_index + (slice << 5);
> >>>> +			el.dir = dir;
> >>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
> >>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
> >>>> +					      sizeof(el));
> >>>> +			acv &= ~BIT(bit_index);
> >>>> +		}
> >>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
> >>>> +	}
> >>>> +}
> >>> What happens when the hte_push_ts_ns_atomic() fails?
> >>> The timestamp will be quietly dropped?
> >>> What happens when the interrupt corresponding to that dropped timestamp
> >>> asks for it?  The irq handler thread will block until it can get a
> >>> timestamp from the subsequent interrupt?
> >> Two things happen, 1) at the push, HTE core increments seq counter
> >>
> >> 2) If the consumer has provided callback, it will either call that callback
> >>
> >> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
> >>
> >> view of dropped ts. However, I see the problem with the consumers not
> >>
> >> providing callback, in that case, push_ts* API just wakes up process without
> >>
> >> indicating why (assuming notify variable is true or else there is a chance for
> >>
> >> the thread to block forever). One easy approach I can think of for now is to
> >>
> >> make callback mandatory (which is optional right now), I will have to rethink
> >>
> >> that scenario and will push corrected version next RFC version.
> >>
> >> Thanks for pointing out.
> >>
> > I'm not sure you understood my question, which was intended to
> > demonstrate how an overflow here would break your gpio integration, but I
> > am certain that I don't understand your answer.
> >
> > Using the callback to signal fifo overflow to the consumer is crazy.
> > If the consumer is too busy to service the fifo then they probably wont
> > be prepared to deal with the callback either. And the primary purpose of
> > the fifo is to decouple the producer and consumer, so requiring a callback
> > defeats the whole purpose of having the fifo there in the first place.
> >
> >>> Which brings me back to the concern I have with the approach used in
> >>> the hte/gpiolib integration - how do you guarantee that the timestamp
> >>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
> >>> being handled, particularly in the face of errors such as:
> >>>  - overflows of the timestamp FIFO in the chip
> >> I currently do not have any indication mechanism as the providers
> >>
> >> I am dealing with right now does not have overflow hardware detection
> >>
> >> support. If the chip supports, it should be easy to integrate that feature.
> >>
> >> I will provide some hook function or change in push_* API to accommodate
> >>
> >> this in next version of RFC.
> >>
> >>>  - overflows of software FIFOs as here
> >> HTE core records sequence counter as well it callsback the consumer with
> >>
> >> HTE_TS_DROPPED.
> >>
> >>>  - lost interupts (if the hw generates interrupts faster than the CPU
> >>>    can service them)
> >> For this, I have no idea unless hardware supports some sort of mechanism
> >>
> >> to catch that. For the current providers, as soon as it detects changes on lines
> >>
> >> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
> >>
> >> set in that hw fifo. This interrupt is different than the lines of actual device
> >>
> >> that is why I said I have no idea how we can tackle that. Let me know if there
> >>
> >> is any idea or reference of the codes which does tackle this.
> >>
> > As far as I am aware there is no solution, given your suggested
> > architecture.
> >
> > Your architecture is inherently fragile, as you try to use one stream
> > of data (the timestamp fifo) to provide supplementary info for another
> > (the physical irq).  Guaranteeing that the two are synchronised is
> > impossible - even if you can get them synced at some point, they can
> > fall out of sync without any indication.
> > That is a recipe for Ingenuity flight 6.
> >
> > My solution would be to use the hte timestamp fifo as the event source,
> > rather than the physical irq.  With only one event source the 
> > synchronisation problem disappears.  As to how to implement that,
> > gpiolib-cdev would request a line from the hte subsystem and register
> > and event handler for it, much as it does currently with the irq
> > subsystem. That event handler would translate the hte events into gpio
> > events.
> 
> I have to circle back to here regarding the event handler thing. I
> 
> surely did not understand fifo as event source rather than physical irq
> 
> part? I believe you are suggesting to have somekind of buffer abstraction
> 
> layer for the hardware fifo similar to what I have with software buffer and
> 
> register handler to that buffer, right?
> 

No, what is the purpose of that buffering and watermarking in software?
Just pass the timestamped edge event direct to the consumer.
Let the consumer do any buffering if necessary, as Jonathon Cameron
also suggested in the 02/11 thread.

> 
> The current implementation I have (not with gpiolib/HTE integration)
> 
> is partially simlar to event handler mechanism except that it does not send data
> 
> with it. See hte-tegra194-irq-test.c in this patch.
> 
> 
> Coming back to gpiolib/hte integration part and your suggestion about
> 
> providing event handler during hte registration. I have below doubts:
> 
> 1. When HTE calls this provided hte_handler, will it store data into
> 
> hte->timestamp_ns directly, I am guessing yes.
> 

This is implementation detail of the hte/gpiolib interface that I leave
for you to suggest.  Work something out.

> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
> 
> hte_handler, for the worst case scenario as below?
> 

No.  If hardware timestamp is selected then no irq is requested from the
irq subsystem for that line - only from the hte subsystem instead.
So there will be no edge_irq_handler call for that line, so no possible race.

> 2.a edge_irq_handler runs first, checks some kind of flag to see if
> 
> we are using hardware clock and if yes, directly accesses timestamp_ns
> 
> instead of calling line_event_timestamp.
> 
> 2.b finds timestamp_ns to be invalid since it ran first before hte event handler did.
> 
> 2.c returns and invokes edge_irq_thread.
> 
> 2.c.1 Here, should edge_irq_thread wait in cdev till hte handler to be called? If yes,
> 
> Doesn't it have case where it will wait forever till hte handler gets called, also not
> 
> to mention keeping irq line disabled since IRQF_ONESHOT is specified, could be possible
> 
> when provider has gone rogue?
> 
> 3. I am guessing there will not be dropped event in this suggestion since are
> 
> directly sending data without buffering in HTE, that is the good part I believe.
> 
> 
> >
> > You still have to deal with possible fifo overflows, but if the fifo
> > overflows result in discarding the oldest event, rather than the most
> > recent, then everything comes out in the wash.  If not then the final
> > event in a burst may not correspond to the actual state so you need
> > some additional mechanism to address that.
> > Either way the consumer needs to be aware that events may be lost - but
> > with the event seqno for consumers to detect those lost events we
> > already have that covered.
> 
> Correct (for the seqno part), you already have seqno, cdev does not need
> 
> struct hte_ts_data's u64 seq counter.
> 
> 
> On similar note, I was looking at the linereq_put_event
> 
> function and I have below doubts:
> 
> 1. IIUC, you are discarding oldest data when fifo is full, right?
> 

Correct.

> 2. There is no indication to waiting client if overflow is true beside pr_debug print.
> 
> 2.a Does this not block waiting client infinitely since there is no wake_up_poll call
> 
> in case of overflow == 1?
> 

No - there already was a wake_up_poll call for the entry discarded by
the kfifo_skip().

You dropped 2.b intentionally, right?  Buffer overflow perhaps??

> 2.c If I have missed, what current mechanism cdev provides to client beside seqno
> 
> to indicate there is a drop and if there is a drop, what it does to re-sync?
> 

Just seqno.  Overflows in the cdev event buffer discard the oldest
events, so the final event that the client reads will correspond to
current state. There is an event waiting for the client that, due to
the seqno jump, indicates the overflow.  What else do they need?
And what is there to resync?

Not sorry if I'm getting short with you here - I'm getting really tired
of this subject as we're clearly not communicating well and are repeatedly
covering the same ground.

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  4:52             ` Dipen Patel
@ 2021-08-07  4:51               ` Kent Gibson
  2021-08-07  5:35                 ` Dipen Patel
  0 siblings, 1 reply; 75+ messages in thread
From: Kent Gibson @ 2021-08-07  4:51 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Aug 06, 2021 at 09:52:54PM -0700, Dipen Patel wrote:
> 
> On 8/6/21 8:07 PM, Kent Gibson wrote:
> > On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
> >> On 7/31/21 8:43 AM, Kent Gibson wrote:
> >>> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
> >>>> Thanks Kent for the review comment. My responses inline.
> >>>>

<snip>

> >
> >> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
> >>
> >> hte_handler, for the worst case scenario as below?
> >>
> > No.  If hardware timestamp is selected then no irq is requested from the
> > irq subsystem for that line - only from the hte subsystem instead.
> > So there will be no edge_irq_handler call for that line, so no possible race.
> 
> That is not possible for certain providers, for example the one I am dealing
> 
> with which requires GPIO line to be requested as input and IRQ needs to
> 
> be enabled on them.
> 

So, for your hte subsystem to work, the consumer has to also request
a line from the irq subsystem?  That makes sense to you?
Have hte do that, rather than the consumer.

And another reason it makes sense to integrate this with irq...

Cheers,
Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  3:07           ` Kent Gibson
@ 2021-08-07  4:52             ` Dipen Patel
  2021-08-07  4:51               ` Kent Gibson
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-08-07  4:52 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 8/6/21 8:07 PM, Kent Gibson wrote:
> On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
>> On 7/31/21 8:43 AM, Kent Gibson wrote:
>>> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
>>>> Thanks Kent for the review comment. My responses inline.
>>>>
>>>> On 7/1/21 7:21 AM, Kent Gibson wrote:
>>>>> On Fri, Jun 25, 2021 at 04:55:24PM -0700, Dipen Patel wrote:
>>>>>> Tegra194 device has multiple HTE instances also known as GTE
>>>>>> (Generic hardware Timestamping Engine) which can timestamp subset of
>>>>>> SoC lines/signals. This provider driver focuses on IRQ and GPIO lines
>>>>>> and exposes timestamping ability on those lines to the consumers
>>>>>> through HTE subsystem.
>>>>>>
>>>>>> Also, with this patch, added:
>>>>>> - documentation about this provider and its capabilities at
>>>>>> Documentation/hte.
>>>>>> - Compilation support in Makefile and Kconfig
>>>>>>
>>>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
>>>>>> ---
>>>>>>  Documentation/hte/index.rst        |  21 ++
>>>>>>  Documentation/hte/tegra194-hte.rst |  65 ++++
>>>>>>  Documentation/index.rst            |   1 +
>>>>>>  drivers/hte/Kconfig                |  12 +
>>>>>>  drivers/hte/Makefile               |   1 +
>>>>>>  drivers/hte/hte-tegra194.c         | 554 +++++++++++++++++++++++++++++
>>>>>>  6 files changed, 654 insertions(+)
>>>>>>  create mode 100644 Documentation/hte/index.rst
>>>>>>  create mode 100644 Documentation/hte/tegra194-hte.rst
>>>>>>  create mode 100644 drivers/hte/hte-tegra194.c
>>>>>>
>>>>>> diff --git a/Documentation/hte/index.rst b/Documentation/hte/index.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..f311ebec6b47
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/hte/index.rst
>>>>>> @@ -0,0 +1,21 @@
>>>>>> +.. SPDX-License-Identifier: GPL-2.0
>>>>>> +
>>>>>> +============================================
>>>>>> +The Linux Hardware Timestamping Engine (HTE)
>>>>>> +============================================
>>>>>> +
>>>>>> +The HTE Subsystem
>>>>>> +=================
>>>>>> +
>>>>>> +.. toctree::
>>>>>> +   :maxdepth: 1
>>>>>> +
>>>>>> +   hte
>>>>>> +
>>>>>> +HTE Tegra Provider
>>>>>> +==================
>>>>>> +
>>>>>> +.. toctree::
>>>>>> +   :maxdepth: 1
>>>>>> +
>>>>>> +   tegra194-hte
>>>>>> \ No newline at end of file
>>>>>> diff --git a/Documentation/hte/tegra194-hte.rst b/Documentation/hte/tegra194-hte.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..c23eaafcf080
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/hte/tegra194-hte.rst
>>>>>> @@ -0,0 +1,65 @@
>>>>>> +HTE Kernel provider driver
>>>>>> +==========================
>>>>>> +
>>>>>> +Description
>>>>>> +-----------
>>>>>> +The Nvidia tegra194 chip has many hardware timestamping engine (HTE) instances
>>>>>> +known as generic timestamping engine (GTE). This provider driver implements
>>>>>> +two GTE instances 1) GPIO GTE and 2) IRQ GTE. The both GTEs instances get the
>>>>>> +timestamp from the system counter TSC which has 31.25MHz clock rate, and the
>>>>>> +driver converts clock tick rate to nano seconds before storing it as timestamp
>>>>>> +value.
>>>>>> +
>>>>>> +GPIO GTE
>>>>>> +--------
>>>>>> +
>>>>>> +This GTE instance help timestamps GPIO in real time, for that to happen GPIO
>>>>>> +needs to be configured as input and IRQ needs to ba enabled as well. The only
>>>>>> +always on (AON) gpio controller instance supports timestamping GPIOs in
>>>>>> +realtime and it has 39 GPIO lines. There is also a dependency on AON GPIO
>>>>>> +controller as it requires very specific bits to be set in GPIO config register.
>>>>>> +It in a way creates cyclic dependency between GTE and GPIO controller. The GTE
>>>>>> +GPIO functionality is accessed from the GPIOLIB. It can support both the in
>>>>>> +kernel and userspace consumers. In the later case, requests go through GPIOLIB
>>>>>> +CDEV framework. The below APIs are added in GPIOLIB framework to access HTE
>>>>>> +subsystem and GPIO GTE for in kernel consumers.
>>>>>> +
>>>>>> +.. c:function:: int gpiod_hw_timestamp_control( struct gpio_desc *desc, bool enable )
>>>>>> +
>>>>>> +	To enable HTE on given GPIO line.
>>>>>> +
>>>>>> +.. c:function:: u64 gpiod_get_hw_timestamp( struct gpio_desc *desc, bool block )
>>>>>> +
>>>>>> +	To retrieve hardwre timestamp in nano seconds.
>>>>>> +
>>>>>> +.. c:function:: bool gpiod_is_hw_timestamp_enabled( const struct gpio_desc *desc )
>>>>>> +
>>>>>> +	To query if HTE is enabled on the given GPIO.
>>>>>> +
>>>>>> +There is hte-tegra194-gpio-test.c, located in ``drivers/hte/`` directory, test
>>>>>> +driver which demonstrates above APIs for the Jetson AGX platform. For userspace
>>>>>> +consumers, GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE flag must be specifed during
>>>>>> +IOCTL calls, refer ``tools/gpio/gpio-event-mon.c``, which returns the timestamp
>>>>>> +in nano second.
>>>>>> +
>>>>> <snip>
>>>>>
>>>>>> +
>>>>>> +static void tegra_hte_read_fifo(struct tegra_hte_soc *gs)
>>>>>> +{
>>>>>> +	u32 tsh, tsl, src, pv, cv, acv, slice, bit_index, line_id;
>>>>>> +	u64 tsc;
>>>>>> +	int dir;
>>>>>> +	struct hte_ts_data el;
>>>>>> +
>>>>>> +	while ((tegra_hte_readl(gs, HTE_TESTATUS) >>
>>>>>> +		HTE_TESTATUS_OCCUPANCY_SHIFT) &
>>>>>> +		HTE_TESTATUS_OCCUPANCY_MASK) {
>>>>>> +		tsh = tegra_hte_readl(gs, HTE_TETSCH);
>>>>>> +		tsl = tegra_hte_readl(gs, HTE_TETSCL);
>>>>>> +		tsc = (((u64)tsh << 32) | tsl);
>>>>>> +
>>>>>> +		src = tegra_hte_readl(gs, HTE_TESRC);
>>>>>> +		slice = (src >> HTE_TESRC_SLICE_SHIFT) &
>>>>>> +			    HTE_TESRC_SLICE_DEFAULT_MASK;
>>>>>> +
>>>>>> +		pv = tegra_hte_readl(gs, HTE_TEPCV);
>>>>>> +		cv = tegra_hte_readl(gs, HTE_TECCV);
>>>>>> +		acv = pv ^ cv;
>>>>>> +		while (acv) {
>>>>>> +			bit_index = __builtin_ctz(acv);
>>>>>> +			if ((pv >> bit_index) & BIT(0))
>>>>>> +				dir = HTE_EVENT_RISING_EDGE;
>>>>>> +			else
>>>>>> +				dir = HTE_EVENT_FALLING_EDGE;
>>>>>> +
>>>>>> +			line_id = bit_index + (slice << 5);
>>>>>> +			el.dir = dir;
>>>>>> +			el.tsc = tsc << HTE_TS_NS_SHIFT;
>>>>>> +			hte_push_ts_ns_atomic(gs->chip, line_id, &el,
>>>>>> +					      sizeof(el));
>>>>>> +			acv &= ~BIT(bit_index);
>>>>>> +		}
>>>>>> +		tegra_hte_writel(gs, HTE_TECMD, HTE_TECMD_CMD_POP);
>>>>>> +	}
>>>>>> +}
>>>>> What happens when the hte_push_ts_ns_atomic() fails?
>>>>> The timestamp will be quietly dropped?
>>>>> What happens when the interrupt corresponding to that dropped timestamp
>>>>> asks for it?  The irq handler thread will block until it can get a
>>>>> timestamp from the subsequent interrupt?
>>>> Two things happen, 1) at the push, HTE core increments seq counter
>>>>
>>>> 2) If the consumer has provided callback, it will either call that callback
>>>>
>>>> with HTE_TS_DROPPED or HTE_TS_AVAIL. The seq counter gives indirect
>>>>
>>>> view of dropped ts. However, I see the problem with the consumers not
>>>>
>>>> providing callback, in that case, push_ts* API just wakes up process without
>>>>
>>>> indicating why (assuming notify variable is true or else there is a chance for
>>>>
>>>> the thread to block forever). One easy approach I can think of for now is to
>>>>
>>>> make callback mandatory (which is optional right now), I will have to rethink
>>>>
>>>> that scenario and will push corrected version next RFC version.
>>>>
>>>> Thanks for pointing out.
>>>>
>>> I'm not sure you understood my question, which was intended to
>>> demonstrate how an overflow here would break your gpio integration, but I
>>> am certain that I don't understand your answer.
>>>
>>> Using the callback to signal fifo overflow to the consumer is crazy.
>>> If the consumer is too busy to service the fifo then they probably wont
>>> be prepared to deal with the callback either. And the primary purpose of
>>> the fifo is to decouple the producer and consumer, so requiring a callback
>>> defeats the whole purpose of having the fifo there in the first place.
>>>
>>>>> Which brings me back to the concern I have with the approach used in
>>>>> the hte/gpiolib integration - how do you guarantee that the timestamp
>>>>> returned by gpiod_get_hw_timestamp() corresponds to the irq interrupt
>>>>> being handled, particularly in the face of errors such as:
>>>>>  - overflows of the timestamp FIFO in the chip
>>>> I currently do not have any indication mechanism as the providers
>>>>
>>>> I am dealing with right now does not have overflow hardware detection
>>>>
>>>> support. If the chip supports, it should be easy to integrate that feature.
>>>>
>>>> I will provide some hook function or change in push_* API to accommodate
>>>>
>>>> this in next version of RFC.
>>>>
>>>>>  - overflows of software FIFOs as here
>>>> HTE core records sequence counter as well it callsback the consumer with
>>>>
>>>> HTE_TS_DROPPED.
>>>>
>>>>>  - lost interupts (if the hw generates interrupts faster than the CPU
>>>>>    can service them)
>>>> For this, I have no idea unless hardware supports some sort of mechanism
>>>>
>>>> to catch that. For the current providers, as soon as it detects changes on lines
>>>>
>>>> it captures TS in its hw fifo. Its interrupt gets generated based on threshold
>>>>
>>>> set in that hw fifo. This interrupt is different than the lines of actual device
>>>>
>>>> that is why I said I have no idea how we can tackle that. Let me know if there
>>>>
>>>> is any idea or reference of the codes which does tackle this.
>>>>
>>> As far as I am aware there is no solution, given your suggested
>>> architecture.
>>>
>>> Your architecture is inherently fragile, as you try to use one stream
>>> of data (the timestamp fifo) to provide supplementary info for another
>>> (the physical irq).  Guaranteeing that the two are synchronised is
>>> impossible - even if you can get them synced at some point, they can
>>> fall out of sync without any indication.
>>> That is a recipe for Ingenuity flight 6.
>>>
>>> My solution would be to use the hte timestamp fifo as the event source,
>>> rather than the physical irq.  With only one event source the 
>>> synchronisation problem disappears.  As to how to implement that,
>>> gpiolib-cdev would request a line from the hte subsystem and register
>>> and event handler for it, much as it does currently with the irq
>>> subsystem. That event handler would translate the hte events into gpio
>>> events.
>> I have to circle back to here regarding the event handler thing. I
>>
>> surely did not understand fifo as event source rather than physical irq
>>
>> part? I believe you are suggesting to have somekind of buffer abstraction
>>
>> layer for the hardware fifo similar to what I have with software buffer and
>>
>> register handler to that buffer, right?
>>
> No, what is the purpose of that buffering and watermarking in software?
> Just pass the timestamped edge event direct to the consumer.
> Let the consumer do any buffering if necessary, as Jonathon Cameron
> also suggested in the 02/11 thread.
>
>> The current implementation I have (not with gpiolib/HTE integration)
>>
>> is partially simlar to event handler mechanism except that it does not send data
>>
>> with it. See hte-tegra194-irq-test.c in this patch.
>>
>>
>> Coming back to gpiolib/hte integration part and your suggestion about
>>
>> providing event handler during hte registration. I have below doubts:
>>
>> 1. When HTE calls this provided hte_handler, will it store data into
>>
>> hte->timestamp_ns directly, I am guessing yes.
>>
> This is implementation detail of the hte/gpiolib interface that I leave
> for you to suggest.  Work something out.
>
>> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
>>
>> hte_handler, for the worst case scenario as below?
>>
> No.  If hardware timestamp is selected then no irq is requested from the
> irq subsystem for that line - only from the hte subsystem instead.
> So there will be no edge_irq_handler call for that line, so no possible race.

That is not possible for certain providers, for example the one I am dealing

with which requires GPIO line to be requested as input and IRQ needs to

be enabled on them.

>
>> 2.a edge_irq_handler runs first, checks some kind of flag to see if
>>
>> we are using hardware clock and if yes, directly accesses timestamp_ns
>>
>> instead of calling line_event_timestamp.
>>
>> 2.b finds timestamp_ns to be invalid since it ran first before hte event handler did.
>>
>> 2.c returns and invokes edge_irq_thread.
>>
>> 2.c.1 Here, should edge_irq_thread wait in cdev till hte handler to be called? If yes,
>>
>> Doesn't it have case where it will wait forever till hte handler gets called, also not
>>
>> to mention keeping irq line disabled since IRQF_ONESHOT is specified, could be possible
>>
>> when provider has gone rogue?
>>
>> 3. I am guessing there will not be dropped event in this suggestion since are
>>
>> directly sending data without buffering in HTE, that is the good part I believe.
>>
>>
>>> You still have to deal with possible fifo overflows, but if the fifo
>>> overflows result in discarding the oldest event, rather than the most
>>> recent, then everything comes out in the wash.  If not then the final
>>> event in a burst may not correspond to the actual state so you need
>>> some additional mechanism to address that.
>>> Either way the consumer needs to be aware that events may be lost - but
>>> with the event seqno for consumers to detect those lost events we
>>> already have that covered.
>> Correct (for the seqno part), you already have seqno, cdev does not need
>>
>> struct hte_ts_data's u64 seq counter.
>>
>>
>> On similar note, I was looking at the linereq_put_event
>>
>> function and I have below doubts:
>>
>> 1. IIUC, you are discarding oldest data when fifo is full, right?
>>
> Correct.
>
>> 2. There is no indication to waiting client if overflow is true beside pr_debug print.
>>
>> 2.a Does this not block waiting client infinitely since there is no wake_up_poll call
>>
>> in case of overflow == 1?
>>
> No - there already was a wake_up_poll call for the entry discarded by
> the kfifo_skip().
>
> You dropped 2.b intentionally, right?  Buffer overflow perhaps??
>
>> 2.c If I have missed, what current mechanism cdev provides to client beside seqno
>>
>> to indicate there is a drop and if there is a drop, what it does to re-sync?
>>
> Just seqno.  Overflows in the cdev event buffer discard the oldest
> events, so the final event that the client reads will correspond to
> current state. There is an event waiting for the client that, due to
> the seqno jump, indicates the overflow.  What else do they need?
> And what is there to resync?
>
> Not sorry if I'm getting short with you here - I'm getting really tired
> of this subject as we're clearly not communicating well and are repeatedly
> covering the same ground.
Sure, no problem...
>
> Cheers,
> Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  4:51               ` Kent Gibson
@ 2021-08-07  5:35                 ` Dipen Patel
  2021-08-07  5:42                   ` Kent Gibson
  2021-08-07  5:47                   ` Dipen Patel
  0 siblings, 2 replies; 75+ messages in thread
From: Dipen Patel @ 2021-08-07  5:35 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 8/6/21 9:51 PM, Kent Gibson wrote:
> On Fri, Aug 06, 2021 at 09:52:54PM -0700, Dipen Patel wrote:
>> On 8/6/21 8:07 PM, Kent Gibson wrote:
>>> On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
>>>> On 7/31/21 8:43 AM, Kent Gibson wrote:
>>>>> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
>>>>>> Thanks Kent for the review comment. My responses inline.
>>>>>>
> <snip>
>
>>>> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
>>>>
>>>> hte_handler, for the worst case scenario as below?
>>>>
>>> No.  If hardware timestamp is selected then no irq is requested from the
>>> irq subsystem for that line - only from the hte subsystem instead.
>>> So there will be no edge_irq_handler call for that line, so no possible race.
>> That is not possible for certain providers, for example the one I am dealing
>>
>> with which requires GPIO line to be requested as input and IRQ needs to
>>
>> be enabled on them.
>>
> So, for your hte subsystem to work, the consumer has to also request
> a line from the irq subsystem?

Yes

>   That makes sense to you?
Its not me, its peculiarity of the hardware that I am dealing with.
> Have hte do that, rather than the consumer.

Sure, for cdev it would mean to duplicate (most of) the edge* or line_create

code in HTE. For such hardware, my initial doubt remains the same about

the worst case scenario between two handlers, but perhaps that's

implementation details for hte to handle.

>
> And another reason it makes sense to integrate this with irq...

Alright, will explore this route as well. I remember both Thierry[1] and

Marc[2] raised some doubts (time to revive that discussion).


[1]: https://lore.kernel.org/lkml/YFm9r%2FtFkzVlYDEp@orome.fritz.box/

[2]: https://lore.kernel.org/lkml/87h7l1k9yi.wl-maz@kernel.org/

>
> Cheers,
> Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  5:35                 ` Dipen Patel
@ 2021-08-07  5:42                   ` Kent Gibson
  2021-08-07  5:47                   ` Dipen Patel
  1 sibling, 0 replies; 75+ messages in thread
From: Kent Gibson @ 2021-08-07  5:42 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt

On Fri, Aug 06, 2021 at 10:35:10PM -0700, Dipen Patel wrote:
> 
> On 8/6/21 9:51 PM, Kent Gibson wrote:
> > On Fri, Aug 06, 2021 at 09:52:54PM -0700, Dipen Patel wrote:
> >> On 8/6/21 8:07 PM, Kent Gibson wrote:
> >>> On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
> >>>> On 7/31/21 8:43 AM, Kent Gibson wrote:
> >>>>> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
> >>>>>> Thanks Kent for the review comment. My responses inline.
> >>>>>>
> > <snip>
> >
> >>>> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
> >>>>
> >>>> hte_handler, for the worst case scenario as below?
> >>>>
> >>> No.  If hardware timestamp is selected then no irq is requested from the
> >>> irq subsystem for that line - only from the hte subsystem instead.
> >>> So there will be no edge_irq_handler call for that line, so no possible race.
> >> That is not possible for certain providers, for example the one I am dealing
> >>
> >> with which requires GPIO line to be requested as input and IRQ needs to
> >>
> >> be enabled on them.
> >>
> > So, for your hte subsystem to work, the consumer has to also request
> > a line from the irq subsystem?
> 
> Yes
> 
> >   That makes sense to you?
> Its not me, its peculiarity of the hardware that I am dealing with.

My point is that the peculiarities of the hardware should be hidden from
the hte API user, especially if it is only necessary for some hardware.

> > Have hte do that, rather than the consumer.
> 
> Sure, for cdev it would mean to duplicate (most of) the edge* or line_create
> 
> code in HTE.

And your current way every other hte user will have to duplicate the
gpiolib-cdev code....

> For such hardware, my initial doubt remains the same about
> 
> the worst case scenario between two handlers, but perhaps that's
> 
> implementation details for hte to handle.
> 

Indeed.

Cheers,
Kent.

> >
> > And another reason it makes sense to integrate this with irq...
> 
> Alright, will explore this route as well. I remember both Thierry[1] and
> 
> Marc[2] raised some doubts (time to revive that discussion).
> 
> 
> [1]: https://lore.kernel.org/lkml/YFm9r%2FtFkzVlYDEp@orome.fritz.box/
> 
> [2]: https://lore.kernel.org/lkml/87h7l1k9yi.wl-maz@kernel.org/
> 
> >
> > Cheers,
> > Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 03/11] hte: Add tegra194 HTE kernel provider
  2021-08-07  5:35                 ` Dipen Patel
  2021-08-07  5:42                   ` Kent Gibson
@ 2021-08-07  5:47                   ` Dipen Patel
  1 sibling, 0 replies; 75+ messages in thread
From: Dipen Patel @ 2021-08-07  5:47 UTC (permalink / raw)
  To: Kent Gibson
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, devicetree, linux-doc, robh+dt


On 8/6/21 10:35 PM, Dipen Patel wrote:
> On 8/6/21 9:51 PM, Kent Gibson wrote:
>> On Fri, Aug 06, 2021 at 09:52:54PM -0700, Dipen Patel wrote:
>>> On 8/6/21 8:07 PM, Kent Gibson wrote:
>>>> On Fri, Aug 06, 2021 at 07:41:09PM -0700, Dipen Patel wrote:
>>>>> On 7/31/21 8:43 AM, Kent Gibson wrote:
>>>>>> On Wed, Jul 28, 2021 at 04:59:08PM -0700, Dipen Patel wrote:
>>>>>>> Thanks Kent for the review comment. My responses inline.
>>>>>>>
>> <snip>
>>
>>>>> 2. Does hte handler solution create race between two handlers? i.e. edge_irq_handler and
>>>>>
>>>>> hte_handler, for the worst case scenario as below?
>>>>>
>>>> No.  If hardware timestamp is selected then no irq is requested from the
>>>> irq subsystem for that line - only from the hte subsystem instead.
>>>> So there will be no edge_irq_handler call for that line, so no possible race.
>>> That is not possible for certain providers, for example the one I am dealing
>>>
>>> with which requires GPIO line to be requested as input and IRQ needs to
>>>
>>> be enabled on them.
>>>
>> So, for your hte subsystem to work, the consumer has to also request
>> a line from the irq subsystem?
> Yes
>
>>   That makes sense to you?
> Its not me, its peculiarity of the hardware that I am dealing with.
>> Have hte do that, rather than the consumer.
> Sure, for cdev it would mean to duplicate (most of) the edge* or line_create
>
> code in HTE.

Ignore code duplicate comment, shouldn't be big deal.


> For such hardware, my initial doubt remains the same about
>
> the worst case scenario between two handlers, but perhaps that's
>
> implementation details for hte to handle.
>
>> And another reason it makes sense to integrate this with irq...
> Alright, will explore this route as well. I remember both Thierry[1] and
>
> Marc[2] raised some doubts (time to revive that discussion).
>
>
> [1]: https://lore.kernel.org/lkml/YFm9r%2FtFkzVlYDEp@orome.fritz.box/
>
> [2]: https://lore.kernel.org/lkml/87h7l1k9yi.wl-maz@kernel.org/
>
>> Cheers,
>> Kent.

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 09/11] tools: gpio: Add new hardware clock type
  2021-07-31  6:16       ` Kent Gibson
@ 2021-08-11  9:11         ` Linus Walleij
  0 siblings, 0 replies; 75+ messages in thread
From: Linus Walleij @ 2021-08-11  9:11 UTC (permalink / raw)
  To: Kent Gibson
  Cc: Dipen Patel, thierry.reding, Jon Hunter, linux-kernel,
	linux-tegra, open list:GPIO SUBSYSTEM, Bartosz Golaszewski,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Linux Doc Mailing List, Rob Herring

On Sat, Jul 31, 2021 at 8:16 AM Kent Gibson <warthog618@gmail.com> wrote:
> On Thu, Jul 29, 2021 at 08:17:22PM -0700, Dipen Patel wrote:
> >
> > On 6/27/21 4:36 AM, Linus Walleij wrote:
> > > On Sat, Jun 26, 2021 at 1:48 AM Dipen Patel <dipenp@nvidia.com> wrote:
> > >
> > >> gpiolib-cdev is extended to support hardware clock type, this
> > >> patch reflects that fact.
> > >>
> > >> Signed-off-by: Dipen Patel <dipenp@nvidia.com>
> > > (...)
> > >>                 case 'w':
> > >>                         config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_REALTIME;
> > >>                         break;
> > >> +               case 't':
> > >> +                       config.flags |= GPIO_V2_LINE_FLAG_EVENT_CLOCK_HARDWARE;
> > >> +                       break;
> > > After the checking of the command line options we need a small sanity
> > > check so we don't try to enable both realtime and hardware clock
> > > at the same time, we will only be able to request one of them.
> >
> > This will any way fail at gpiolib-cdev layer. Do we want to add it here
> >
> > as well?
> >
>
> I can't speak for Linus, but I'm fine with it as is as it allows the tool
> to be used to exercise the sanity check in the kernel.

Fair enough, that sounds useful. Go ahead with this as-is.

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-08-01 16:13       ` Jonathan Cameron
@ 2021-09-14  5:43         ` Dipen Patel
  2021-09-26 15:42           ` Jonathan Cameron
  0 siblings, 1 reply; 75+ messages in thread
From: Dipen Patel @ 2021-09-14  5:43 UTC (permalink / raw)
  To: Jonathan Cameron
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

Hi Jonathan,

I got some time to implement RFC version 2 while doing so I have a follow up comment

inline regarding clock source comment of yours.

Best Regards,

Dipen Patel

On 8/1/21 9:13 AM, Jonathan Cameron wrote:
> On Tue, 27 Jul 2021 21:38:45 -0700
> Dipen Patel <dipenp@nvidia.com> wrote:
>
>> On 7/4/21 1:15 PM, Jonathan Cameron wrote:
>>> On Fri, 25 Jun 2021 16:55:23 -0700
>>> Dipen Patel <dipenp@nvidia.com> wrote:
>>>  
>>>> Some devices can timestamp system lines/signals/Buses in real-time
>>>> using the hardware counter or other hardware means which can give
>>>> finer granularity and help avoid jitter introduced by software means
>>>> of timestamping. To utilize such functionality there has to be
>>>> framework where such devices can register themselves as producers or
>>>> providers so that the consumers or clients devices can request specific
>>>> line from the providers. This patch introduces such subsystem as
>>>> hardware timestamping engine (HTE).
>>>>
>>>> It provides below APIs for the provider:
>>>> - hte_register_chip() -- To register the HTE chip.
>>>> - hte_unregister_chip() -- To unregister the HTE chip.
>>>> - hte_push_ts_ns_atomic() -- To push timestamp data into HTE subsystem.
>>>>
>>>> It provides below APIs for the consumer:
>>>> - of_hte_request_ts() -- To request timestamp functionality.
>>>> - devm_of_hte_request_ts() -- Managed version of the above.
>>>> - hte_req_ts_by_dt_node() -- To request timestamp functionality by
>>>> using HTE provider dt node.
>>>> - devm_hte_release_ts() -- The managed version to release timestamp
>>>> functionality and associated resources.
>>>> - hte_retrieve_ts_ns() -- To retrieve timestamps.
>>>> - hte_retrieve_ts_ns_wait() -- Same as above but blocking version.
>>>> - hte_enable_ts() -- To disable timestamp functionality.
>>>> - hte_disable_ts() -- To enable timestamp functionality.
>>>> - hte_available_ts() -- To query available timestamp data.
>>>> - hte_release_ts() -- To release timestamp functionality and its
>>>> associated resources.
>>>> - hte_get_clk_src_info() -- To query clock source information from
>>>> the provider
>>>>
>>>> It provides centralized software buffer management per requested id to
>>>> store the timestamp data for the consumers as below:
>>>> - hte_set_buf_len() -- To set the buffer length.
>>>> - hte_get_buf_len() -- To get the buffer length.
>>>> - hte_set_buf_watermark() -- To set the software threshold/watermark.
>>>> - hte_get_buf_watermark() -- To get the software threshold/watermark.
>>>>
>>>> The detail about parameters and API usage are described in each
>>>> functions definitions in drivers/hte/hte.c file.
>>>>
>>>> The patch adds compilation support in Makefile and menu options in
>>>> Kconfig.
>>>>
>>>> Signed-off-by: Dipen Patel <dipenp@nvidia.com>  
>>> Hi Dipen, this isn't a particularly thorough review as I'm still getting my head
>>> around what this is doing + it is an RFC :)  
>> Thanks for the review comments. My responses inline.
> You are welcome, some follow up responses inline.
> I've tried to crop this down a bit so only kept the bits we are discussing.
>
>>>> +
>>>> +static int hte_ts_dis_en_common(struct hte_ts_desc *desc, bool en)
>>>> +{
>>>> +	u32 ts_id;
>>>> +	struct hte_device *gdev;
>>>> +	struct hte_ts_info *ei;
>>>> +	int ret;
>>>> +
>>>> +	if (!desc)
>>>> +		return -EINVAL;
>>>> +
>>>> +	ei = (struct hte_ts_info *)desc->data_subsys;  
>>> As above, no need to cast - though it rather implies the type of data_subsys
>>> should not be void *.  
>> desc is public facing structure, I wanted to make subsystem related
>>
>> information opaque that is why I had it void *.
>>
> you can keep it opaque, just have a forwards definition of
> struct hte_ts_desc;
> which just means it is defined somewhere.  You can have that in the header with
> the definition hidden away.
>
> It will only need to have a visible complete definition when you dereference it inside
> the the core.
>
> Mind you, I'm suggesting allowing it to be embedded in another structure anyway which
> would require you to have it exposed.  Perhaps this desire to keep it opaque is
> a reason to not take that suggestion but it isn't relevant for this one.
>
>
>>>> + */
>>>> +struct hte_ts_desc *devm_of_hte_request_ts(struct device *dev,
>>>> +					   const char *label,
>>>> +					   void (*cb)(enum hte_notify n))
>>>> +{
>>>> +
>>>> +	struct hte_ts_desc **ptr, *desc;
>>>> +
>>>> +	ptr = devres_alloc(__devm_hte_release_ts, sizeof(*ptr), GFP_KERNEL);  
>>> Superficially looks like you might get way with just calling dev_add_action_or_reset() in here
>>> and avoid this boilerplate.  A lot of cases that looked like this got cleaned up in the
>>> last kernel cycle.  
>> I based my patches from linux-next/master. Not sure if that has
>>
>> dev_add_action_or_reset
> typo on my part was meant to be
>
> devm_add_action_or_reset()
>
>>>> +
>>>> +/**
>>>> + * hte_req_ts_by_dt_node() - Request entity to monitor by passing HTE device
>>>> + * node directly, where meaning of the entity is provider specific, for example
>>>> + * lines, signals, GPIOs, buses etc...
>>>> + *
>>>> + * @of_node: HTE provider device node.
>>>> + * @id: entity id to monitor, this id belongs to HTE provider of_node.
>>>> + * @cb: Optional callback to notify.
>>>> + *
>>>> + * Context: Holds mutex lock, can not be called from atomic context.  
>>> What mutex and why?  If it is one you can check is held even better.  
>> ___hte_req_ts holds the mutex lock to serialize multiple consumers
>>
>> requesting same entity.
> Add that detail to the comment.
>
>>>  
>>>> + * Returns: ts descriptor on success or error pointers.
>>>> + */
>>>> +struct hte_ts_desc *hte_req_ts_by_dt_node(struct device_node *of_node,
>>>> +					  unsigned int id,
>>>> +					  void (*cb)(enum hte_notify n))
>>>> +{
>>>> +	struct hte_device *gdev;
>>>> +	struct hte_ts_desc *desc;
>>>> +	int ret;
>>>> +	u32 xlated_id;
>>>> +
>>>> +	gdev = of_node_to_htedevice(of_node);
>>>> +	if (IS_ERR(gdev))
>>>> +		return ERR_PTR(-ENOTSUPP);
>>>> +
>>>> +	if (!gdev->chip || !gdev->chip->ops)
>>>> +		return ERR_PTR(-ENOTSUPP);
>>>> +
>>>> +	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
>>>> +	if (!desc) {
>>>> +		ret = -ENOMEM;
>>>> +		goto out_put_device;
>>>> +	}  
>>> Pass a desc pointer into this function rather than allocating the structure
>>> in here.  That lets the caller embed that structure inside one of it's own
>>> structures if it wants to, resulting in fewer small allocations which is always good.
>>>
>>> It's far from obvious that the caller needs to free desc.  
>> Are you suggesting to shift burden of allocation/deallocation (static or dynamic)
>>
>> at client/consumer side?
> It's been a while so I've forgotten how this works, but 'probably' yes...
> If a function creates some sort of record (of fixed known size and type) then
> letting that be passed in + filled in by the function is normally more efficient
> than having an allocation in here.   Chances are the consumer will just have
> it embedded in an existing state structure and not need to do any explicit
> allocation / deallocation.  Disadvantage is you can't keep it opaque.
>
>>>  
>>>> +
>>>> +	desc->con_id = id;
>>>> +	ret = gdev->chip->xlate(gdev->chip, NULL, desc, &xlated_id);
>>>> +	if (ret < 0) {
>>>> +		dev_err(gdev->chip->dev,
>>>> +			"failed to xlate id: %d\n", id);
>>>> +		goto out_free_desc;
>>>> +	}
>>>> +
>>>> +	ret = ___hte_req_ts(gdev, desc, xlated_id, cb);
>>>> +	if (ret < 0) {
>>>> +		dev_err(gdev->chip->dev,
>>>> +			"failed to request id: %d\n", id);
>>>> +		goto out_free_desc;
>>>> +	}
>>>> +
>>>> +	return desc;
>>>> +
>>>> +out_free_desc:
>>>> +	kfree(desc);
>>>> +
>>>> +out_put_device:
>>>> +	return ERR_PTR(ret);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(hte_req_ts_by_dt_node);
>>>> +
>>>> +/**
>>>> + * hte_get_clk_src_info() - Consumer calls this API to query clock source
>>>> + * information of the desc.
>>>> + *
>>>> + * @desc: ts descriptor, same as returned from request API.
>>>> + *
>>>> + * Context: Any context.
>>>> + * Returns: 0 on success else negative error code on failure.
>>>> + */
>>>> +int hte_get_clk_src_info(const struct hte_ts_desc *desc,
>>>> +			 struct hte_clk_info *ci)
>>>> +{
>>>> +	struct hte_chip *chip;
>>>> +	struct hte_ts_info *ei;
>>>> +
>>>> +	if (!desc || !desc->data_subsys || !ci) {
>>>> +		pr_debug("%s:%d\n", __func__, __LINE__);
>>>> +		return -EINVAL;
>>>> +	}
>>>> +
>>>> +	ei = desc->data_subsys;
>>>> +	if (!ei || !ei->gdev || !ei->gdev->chip)
>>>> +		return -EINVAL;
>>>> +
>>>> +	chip = ei->gdev->chip;
>>>> +	if (!chip->ops->get_clk_src_info)
>>>> +		return -ENOTSUPP;
>>>> +
>>>> +	return chip->ops->get_clk_src_info(chip, ci);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(hte_get_clk_src_info);
>>>> +
>>>> +static inline void hte_add_to_device_list(struct hte_device *gdev)
>>>> +{
>>>> +	struct hte_device *prev;  
>>> Needs to take an appropriate lock as you may have concurrent calls.  
>> There is spin_lock held from register API from where this gets
>> called.
> Great. I'd missed that.
>
>>>  
>>>> +
>>>> +	if (list_empty(&hte_devices)) {
>>>> +		list_add_tail(&gdev->list, &hte_devices);  
>>> Needs a comment. I've no idea why you might want to only add it if there were
>>> no other hte_devices already there.
>>>  
>>>> +		return;
>>>> +	}
>>>> +
>>>> +	prev = list_last_entry(&hte_devices, struct hte_device, list);  
>>> Why woud you do this?  
>> Thanks for pointing out. I definitely missed cleaning this up. Now, I will
>>
>> remove this function in next RFC version as one line can be added directly
>>
>> in register API.
>>
>>>  
>>>> +	list_add_tail(&gdev->list, &hte_devices);
>>>> +}
>>>> +
>>>> +/**
>>>> + * hte_push_ts_ns_atomic() - Used by the provider to push timestamp in nano
>>>> + * seconds i.e data->tsc will be in ns, it is assumed that provider will be
>>>> + * using this API from its ISR or atomic context.
>>>> + *
>>>> + * @chip: The HTE chip, used during the registration.
>>>> + * @xlated_id: entity id understood by both subsystem and provider, usually this
>>>> + * is obtained from xlate callback during request API.
>>>> + * @data: timestamp data.
>>>> + * @n: Size of the data.
>>>> + *
>>>> + * Context: Atomic.
>>>> + * Returns: 0 on success or a negative error code on failure.
>>>> + */
>>>> +int hte_push_ts_ns_atomic(const struct hte_chip *chip, u32 xlated_id,
>>>> +			  struct hte_ts_data *data, size_t n)
>>>> +{
>>>> +	unsigned int ret;
>>>> +	bool notify;
>>>> +	size_t el_avail;
>>>> +	struct hte_ts_buf *buffer;
>>>> +	struct hte_ts_info *ei;
>>>> +
>>>> +	if (!chip || !data || !chip->gdev)
>>>> +		return -EINVAL;
>>>> +
>>>> +	if (xlated_id > chip->nlines)
>>>> +		return -EINVAL;
>>>> +
>>>> +	ei = &chip->gdev->ei[xlated_id];
>>>> +
>>>> +	if (!test_bit(HTE_TS_REGISTERED, &ei->flags) ||
>>>> +	    test_bit(HTE_TS_DISABLE, &ei->flags)) {
>>>> +		dev_dbg(chip->dev, "Unknown timestamp push\n");
>>>> +		return -EINVAL;
>>>> +	}
>>>> +
>>>> +	/* timestamp sequence counter, start from 0 */
>>>> +	data->seq = ei->seq++;
>>>> +
>>>> +	buffer = ei->buf;
>>>> +	el_avail = buffer->access->el_available(buffer);
>>>> +	ret = buffer->access->store(buffer, data, n);  
>>> If we are doing this from the hte core, why is buffer definition in the scope of the
>>> drivers rather than the core?  That seems backwards to me.  
>> I do not understand this comment. The buffer definition is in scope of hte core
>>
>> as it is the only entity that manages it.
> I think I figured that out later and forgot to come back and edit this comment.
> However...
>
> In that case, why is it an ops function?  Don't introduce abstraction
> until you need it. Will be simpler and easier to review if you just
> call those functions directly for now. e.g.
>
> 	ret = hs_ts_store_to_buf(buffer, data, n);
>
> Chances are you'll never introduce another buffer choice.
> For a long time I thought we'd have both fifo and ring options in IIO
> but it turned out no one really cared. We do have an ops structure, but
> that's because in IIO the buffer interface is used for two things:
> 1) Pushing to a kfifo that is going to userspace.
> 2) Pushing to a callback function owned by a consumer.
> and there is a rather fiddly data demux on the front end to ensure each
> of those only gets the data requested via that path - at least with timestamps
> there is only one type of data!
>
> Hmm, thinking about this raises an interesting question.
> Why do we want a kfifo here at all for HTE?  You could
> just call a callback function registered by the consumer of that
> kfifo directly.  If that consumer then wants to buffer then of
> course it can, but it not (perhaps it only cares about the latest
> value and will drop the rest) then it can chose not to.  Maybe
> it's just gathering stats rather than caring about individual
> timestamps?  Probably lots of other things that might happen in
> the consumer that I've not thought of.  We need a buffer if
> userspace becomes involved, but here IIRC that's not (yet) true.
>
>>>  
>>>> +	if (ret != n) {
>>>> +		atomic_inc(&ei->dropped_ts);
>>>> +		if (ei->cb)
>>>> +			ei->cb(HTE_TS_DROPPED);
>>>> +		return -ENOMEM;
>>>> +	}
> ...
>
>>>  
>>>> +
>>>> +/**
>>>> + * struct hte_ts_data - HTE timestamp data.
>>>> + * The provider uses and fills timestamp related details during push_timestamp
>>>> + * API call. The consumer uses during retrieve_timestamp API call.
>>>> + *
>>>> + * @tsc: Timestamp value.
>>>> + * @seq: Sequence counter of the timestamps.
>>>> + * @dir: Direction of the event at the time of timestamp.
>>>> + */
>>>> +struct hte_ts_data {
>>>> +	u64 tsc;
>>>> +	u64 seq;
>>>> +	int dir;
>>>> +};
>>>> +
>>>> +/**
>>>> + * struct hte_clk_info - Clock source info that HTE provider uses.
>>>> + * The provider uses hardware clock as a source to timestamp real time. This
>>>> + * structure presents the clock information to consumers. 
>>>> + *
>>>> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
>>>> + * @type: Clock type. CLOCK_* types.  
>>> So this is something we got a it wrong in IIO. It's much better to define
>>> a subset of clocks that can be potentially used.  There are some that make
>>> absolutely no sense and consumers really don't want to have to deal with them.  
>> Is there anything I have to change here?
> Yes - specify which clocks would make sense.  You might not need to explicitly
> allow only those, but that might also be worthwhile. Otherwise, the chances are
> you'll end up with a bunch of special purpose code in consumers on the basis
> they might get CLOCK_TAI or similar and have to deal with it.
> As for exactly which clocks do make sense, that's one which may take some figuring
> out. Probably REALTIME, MONOTONIC and BOOTTIME depending on whether you care
> what happens when the time of the system gets adjusted, or whether it carries
> on measuring time across suspend.   Very application dependent but there are some
> you can definitely rule out. Don't repeat my mistake of leaving it vague
> (which incidentally was a follow up to picking a silly clock to use for timestamps
>  before we allowed it to be configured).

I believe your comment is under assumption that providers have choice in selecting

clock source to timestamp in turns clients have it as well. For now, the provider

I have implemented has single clock source and hence I only implemented get_clock*

hook that provider implement and client can retrieve that information. I guess I can

always implement set_clock* hook as well for the future providers which support

multiple clock sources. Please let me if I missed your point.

>>>    
>>>> + */
>>>> +struct hte_clk_info {
>>>> +	u64 hz;
>>>> +	clockid_t type;
>>>> +};
>>>> +
>>>> +/**
>>>> + * HTE subsystem notifications for the consumers.
>>>> + *
>>>> + * @HTE_TS_AVAIL: Timestamps available notification.
>>>> + * @HTE_TS_DROPPED: Timestamps dropped notification.  
>>> Something I've missed so far is whether drops are in a kfifo or a ring
>>> fashion.  I'm guess that's stated somewhere, but it might be useful to have
>>> it here.  
>> Dropped are from kfifo if kfifo does not have space.
> Ok, perhaps expand the comment?
>
> ...
>
>>>  
>>>> + *
>>>> + * xlated_id parameter is used to communicate between HTE subsystem and the
>>>> + * providers. It is the same id returned during xlate API call and translated
>>>> + * by the provider. This may be helpful as both subsystem and provider locate
>>>> + * the requested entity in constant time, where entity could be anything from
>>>> + * lines, signals, events, buses etc.. that providers support.
>>>> + */
>>>> +struct hte_ops {
>>>> +	int (*request)(struct hte_chip *chip, u32 xlated_id);
>>>> +	int (*release)(struct hte_chip *chip, u32 xlated_id);
>>>> +	int (*enable)(struct hte_chip *chip, u32 xlated_id);
>>>> +	int (*disable)(struct hte_chip *chip, u32 xlated_id);
>>>> +	int (*get_clk_src_info)(struct hte_chip *chip,
>>>> +				struct hte_clk_info *ci);
>>>> +};
>>>> +
>>>> +/**
>>>> + * struct hte_chip - Abstract HTE chip structure.
>>>> + * @name: functional name of the HTE IP block.
>>>> + * @dev: device providing the HTE.  
>>> Unclear naming.  Is this the parent device, or one associated with the HTE itself?
>>> I'm guessing today you don't have one associated with the HTE, but it is plausible you
>>> might gain on in future to make it fit nicely in the device model as a function of another
>>> device.  
>> This is provider's device, could be &pdev->dev or any dev provider deems fit hence the
>>
>> generic name.
> Ok, for now this works as a name, but I wonder if you will end up growing another
> layer in the device model as would happen for majority of subsystems.
> You may end up doing so when adding support to query the provider via a handle
> in the dt of the consumer.  It could probably be avoided, but throwing this into
> a class might make your life easier as you can use more standard infrastructure.
>
>>>  
>>>> + * @ops: callbacks for this HTE.
>>>> + * @nlines: number of lines/signals supported by this chip.
>>>> + * @xlate: Callback which translates consumer supplied logical ids to
>>>> + * physical ids, return from 0 for the success and negative for the
>>>> + * failures. It stores (0 to @nlines) in xlated_id parameter for the success.
>>>> + * @of_hte_n_cells: Number of cells used to form the HTE specifier.
>>>> + * @gdev: HTE subsystem abstract device, internal to the HTE subsystem.
>>>> + * @data: chip specific private data.
>>>> + */
>>>> +struct hte_chip {
>>>> +	const char *name;
>>>> +	struct device *dev;
>>>> +	const struct hte_ops *ops;
>>>> +	u32 nlines;
>>>> +	int (*xlate)(struct hte_chip *gc,
>>>> +		     const struct of_phandle_args *args,
>>>> +		     struct hte_ts_desc *desc, u32 *xlated_id);
>>>> +	u8 of_hte_n_cells;
>>>> +
>>>> +	/* only used internally by the HTE framework */
>>>> +	struct hte_device *gdev;
>>>> +	void *data;
>>>> +};
> ...
>
> Jonathan

^ permalink raw reply	[flat|nested] 75+ messages in thread

* Re: [RFC 02/11] drivers: Add HTE subsystem
  2021-09-14  5:43         ` Dipen Patel
@ 2021-09-26 15:42           ` Jonathan Cameron
  0 siblings, 0 replies; 75+ messages in thread
From: Jonathan Cameron @ 2021-09-26 15:42 UTC (permalink / raw)
  To: Dipen Patel
  Cc: thierry.reding, jonathanh, linux-kernel, linux-tegra, linux-gpio,
	linus.walleij, bgolaszewski, warthog618, devicetree, linux-doc,
	robh+dt

On Mon, 13 Sep 2021 22:43:02 -0700
Dipen Patel <dipenp@nvidia.com> wrote:

> Hi Jonathan,
> 
> I got some time to implement RFC version 2 while doing so I have a follow up comment
> 
> inline regarding clock source comment of yours.
> 
> Best Regards,
> 
> Dipen Patel
> 
...

> >>>> +/**
> >>>> + * struct hte_clk_info - Clock source info that HTE provider uses.
> >>>> + * The provider uses hardware clock as a source to timestamp real time. This
> >>>> + * structure presents the clock information to consumers. 
> >>>> + *
> >>>> + * @hz: Clock rate in HZ, for example 1KHz clock = 1000.
> >>>> + * @type: Clock type. CLOCK_* types.    
> >>> So this is something we got a it wrong in IIO. It's much better to define
> >>> a subset of clocks that can be potentially used.  There are some that make
> >>> absolutely no sense and consumers really don't want to have to deal with them.    
> >> Is there anything I have to change here?  
> > Yes - specify which clocks would make sense.  You might not need to explicitly
> > allow only those, but that might also be worthwhile. Otherwise, the chances are
> > you'll end up with a bunch of special purpose code in consumers on the basis
> > they might get CLOCK_TAI or similar and have to deal with it.
> > As for exactly which clocks do make sense, that's one which may take some figuring
> > out. Probably REALTIME, MONOTONIC and BOOTTIME depending on whether you care
> > what happens when the time of the system gets adjusted, or whether it carries
> > on measuring time across suspend.   Very application dependent but there are some
> > you can definitely rule out. Don't repeat my mistake of leaving it vague
> > (which incidentally was a follow up to picking a silly clock to use for timestamps
> >  before we allowed it to be configured).  
> 
> I believe your comment is under assumption that providers have choice in selecting
> 
> clock source to timestamp in turns clients have it as well. For now, the provider
> 
> I have implemented has single clock source and hence I only implemented get_clock*
> 
> hook that provider implement and client can retrieve that information. I guess I can
> 
> always implement set_clock* hook as well for the future providers which support
> 
> multiple clock sources. Please let me if I missed your point.

I'll be honest I can't really remember :(  too many sleeps.

Sorry - if it is still relevant perhaps it'll come back to me on v2.

Thanks,

Jonathan

^ permalink raw reply	[flat|nested] 75+ messages in thread

end of thread, other threads:[~2021-09-26 15:38 UTC | newest]

Thread overview: 75+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-25 23:55 [RFC 00/11] Intro to Hardware timestamping engine Dipen Patel
2021-06-25 23:55 ` [RFC 01/11] Documentation: Add HTE subsystem guide Dipen Patel
2021-07-04 18:55   ` Jonathan Cameron
2021-07-27 23:44     ` Dipen Patel
2021-08-01 15:24       ` Jonathan Cameron
2021-06-25 23:55 ` [RFC 02/11] drivers: Add HTE subsystem Dipen Patel
2021-06-27 17:40   ` Randy Dunlap
2021-07-04 20:15   ` Jonathan Cameron
2021-07-04 20:45     ` Jonathan Cameron
2021-07-28  5:12       ` Dipen Patel
2021-08-01 16:48         ` Jonathan Cameron
2021-07-28  4:38     ` Dipen Patel
2021-08-01 16:13       ` Jonathan Cameron
2021-09-14  5:43         ` Dipen Patel
2021-09-26 15:42           ` Jonathan Cameron
2021-07-05  7:30   ` Greg KH
2021-07-28  0:34     ` Dipen Patel
2021-06-25 23:55 ` [RFC 03/11] hte: Add tegra194 HTE kernel provider Dipen Patel
2021-07-01 14:21   ` Kent Gibson
2021-07-28 23:59     ` Dipen Patel
2021-07-30  7:01       ` Dipen Patel
2021-07-31 15:43       ` Kent Gibson
2021-08-03 22:40         ` Dipen Patel
2021-08-03 23:02           ` Kent Gibson
2021-08-07  2:41         ` Dipen Patel
2021-08-07  3:07           ` Kent Gibson
2021-08-07  4:52             ` Dipen Patel
2021-08-07  4:51               ` Kent Gibson
2021-08-07  5:35                 ` Dipen Patel
2021-08-07  5:42                   ` Kent Gibson
2021-08-07  5:47                   ` Dipen Patel
2021-07-04 20:27   ` Jonathan Cameron
2021-07-29  2:42     ` Dipen Patel
2021-07-08 23:33   ` Michał Mirosław
2021-07-29  2:43     ` Dipen Patel
2021-06-25 23:55 ` [RFC 04/11] dt-bindings: Add HTE bindings Dipen Patel
2021-06-27 10:56   ` Linus Walleij
2021-07-30  1:32     ` Dipen Patel
2021-07-01 14:02   ` Rob Herring
2021-07-30  1:56     ` Dipen Patel
2021-07-01 15:54   ` Rob Herring
2021-07-30  1:58     ` Dipen Patel
2021-06-25 23:55 ` [RFC 05/11] hte: Add Tegra194 IRQ HTE test driver Dipen Patel
2021-06-27 17:42   ` Randy Dunlap
2021-06-25 23:55 ` [RFC 06/11] gpiolib: Add HTE support Dipen Patel
2021-06-27 11:41   ` Linus Walleij
2021-07-01 14:24   ` Kent Gibson
2021-07-30  2:25     ` Dipen Patel
2021-07-31  5:13       ` Kent Gibson
2021-06-25 23:55 ` [RFC 07/11] gpio: tegra186: Add HTE in gpio-tegra186 driver Dipen Patel
2021-06-25 23:55 ` [RFC 08/11] gpiolib: cdev: Add hardware timestamp clock type Dipen Patel
2021-06-27 11:38   ` Linus Walleij
2021-06-27 11:49   ` Linus Walleij
2021-07-30  3:16     ` Dipen Patel
2021-07-01 14:24   ` Kent Gibson
2021-07-30  3:07     ` Dipen Patel
2021-07-31  6:05       ` Kent Gibson
2021-08-03 22:41         ` Dipen Patel
2021-08-03 22:38           ` Kent Gibson
2021-07-09  8:30   ` Jon Hunter
2021-07-30  2:33     ` Dipen Patel
2021-08-03 16:42       ` Jon Hunter
2021-08-03 22:51         ` Dipen Patel
2021-08-03 23:09           ` Kent Gibson
2021-06-25 23:55 ` [RFC 09/11] tools: gpio: Add new hardware " Dipen Patel
2021-06-27 11:36   ` Linus Walleij
2021-07-30  3:17     ` Dipen Patel
2021-07-31  6:16       ` Kent Gibson
2021-08-11  9:11         ` Linus Walleij
2021-06-25 23:55 ` [RFC 10/11] hte: Add tegra GPIO HTE test driver Dipen Patel
2021-06-27 17:43   ` Randy Dunlap
2021-06-25 23:55 ` [RFC 11/11] MAINTAINERS: Added HTE Subsystem Dipen Patel
2021-06-27 13:07 ` [RFC 00/11] Intro to Hardware timestamping engine Andy Shevchenko
2021-06-27 14:40   ` Linus Walleij
2021-06-28 12:02     ` Andy Shevchenko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).